A video banner text detection method and system

Abstract

The application provides a video banner text detection method and system, which represents a text instance by a set of key points instead of text box contour points, so as to adapt to text instances of different shapes, and selects a center point in the key points to represent the text instance, thereby avoiding the problem that a text instance is detected as multiple text instances due to the text aspect ratio being too large. In the inter-frame text information processing stage, the image block obtained through video encoding is matched with the key points obtained through the intra-frame text information processing, the matched key points are matched with the key points of a reference frame, the key points of different frames are accurately matched through the double matching mechanism, the key point information is supplemented in the case that the key points are lost due to text masking, and then the banner text is accurately detected, so that the detection accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of text detection in natural scenes, and more specifically, to a method and system for detecting text in video banners. Background Technology

[0002] With the rapid development of deep learning, deep learning-based methods have been widely applied in the field of natural text detection. This paper designs a video image text detection network using text detection technology to detect banner text in videos. By recognizing the content of the banners, it determines whether a mass incident will affect public safety, providing early warnings before the situation worsens, ensuring public safety, maintaining social order, and promoting harmonious social development.

[0003] However, natural scene text detection technology performs poorly when detecting banner text in videos, especially in scenes where the text is distorted or partially obscured.

[0004] Research on the banner's images and related videos revealed that the banner had issues such as a large aspect ratio of the text, text distortion, and partial text obscuring. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing a method and system for detecting video banner text.

[0006] According to a first aspect of the present invention, a method for detecting video banner text is provided, comprising:

[0007] The acquired banner video is encoded to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image;

[0008] Extract key point information from each frame of the image, and use the key point information of the frame image with independent encoding as the key point information of the reference frame image;

[0009] Match the key point information of the current frame image with the position of each macroblock in the current frame image to find the key point information of each macroblock, and perform an offset operation on the key point information of each macroblock based on the displacement vector information of each macroblock in the current frame image.

[0010] The key point information in the current frame image is matched with the key point information in the reference frame image. Based on the key point information in the reference frame image, the key point information in the current frame image is supplemented to obtain the complete key point information of the current frame image.

[0011] Generate a banner video text box based on the complete key point information in the current frame image.

[0012] According to a second aspect of the present invention, a video banner text detection system is provided, comprising:

[0013] The encoding module is used to encode the acquired banner video to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image;

[0014] The extraction module is used to extract key point information of each frame image, and uses the key point information of the independently encoded frame image as the key point information of the reference frame image.

[0015] The first matching module is used to match the key point information of the current frame image with the position of each macroblock in the current frame image, find the key point information to which each macroblock belongs, and perform an offset operation on the key point information to which each macroblock belongs based on the displacement vector information of each macroblock in the current frame image.

[0016] The second matching module is used to match the key point information in the current frame image with the key point information in the reference frame image, and supplement the key point information in the current frame image based on the key point information in the reference frame image to obtain the complete key point information in the current frame image.

[0017] The generation module is used to generate a banner video text box based on the complete key point information in the current frame image.

[0018] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the processor is configured to implement the steps of a video banner text detection method when executing a computer management program stored in the memory.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer management class program stored thereon, which, when executed by a processor, implements the steps of a video banner text detection method.

[0020] The present invention provides a video banner text detection method and system. In the intra-frame text information processing stage, a set of key points are used to replace the text box outline points to represent text instances, thereby adapting to text instances of different shapes. At the same time, a center point is selected from the key points to represent the text instance, avoiding the problem of detecting one text instance as multiple text instances due to the large aspect ratio of the text.

[0021] To address the issue of inaccurate detection of banner text due to partial occlusion in video text detection, a dual-matching mechanism is employed during the inter-frame text information processing stage. This mechanism matches macroblock positions obtained from video decoding with keypoints derived from intra-frame text information processing. The matched keypoints are then compared with keypoints in a reference frame. This dual-matching mechanism accurately matches keypoints from different frames and supplements keypoint information in cases where text occlusion leads to missing keypoints, thereby improving the accuracy of banner text detection. Attached Figure Description

[0022] Figure 1 A flowchart of a video banner text detection method provided by the present invention;

[0023] Figure 2 This is a schematic diagram of a network for extracting key points from video frame images;

[0024] Figure 3 A schematic diagram illustrating the overall process of the video banner text detection method;

[0025] Figure 4 This is a schematic diagram of the structure of a video banner text detection system provided by the present invention;

[0026] Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0027] Figure 6 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0029] By studying banner images and related videos, it was found that banners suffer from problems such as large text aspect ratios, text distortion, and partial text occlusion. To address these issues, this application designs a keypoint-based text representation method. This method uses a certain number of keypoints to represent text boxes. Each keypoint can generate two boundary points, and connecting adjacent keypoints generates a text box, thus accommodating different types of text instances. Simultaneously, a point is selected from the keypoints as the center point to represent the text instance, avoiding the problem of detecting multiple texts due to large text aspect ratios. When the text in the current frame is partially occluded, this application designs a detection method that fuses text box keypoint information with video coding block information. First, the input video is encoded to obtain frame-by-frame video images, along with the macroblock positions and corresponding motion vectors for each frame. Intra-frame text information processing is performed on each frame to obtain keypoints detected using the aforementioned method. These keypoints are stored as reference frames. The keypoints of the current frame are then matched against macroblocks. If a perfect match is found, no further processing is needed, and the keypoint information is directly input to the next step. If there is no perfect match, the keypoints are offset according to the corresponding displacement vector of their respective macroblocks, and the keypoint information is then input to the next step. The keypoints in the current frame are then matched against the keypoints in the reference frame. If a perfect match is found, a text box is directly generated based on the keypoint label generation method. If there is no perfect match, the current frame is supplemented with keypoint information from the reference frame. Finally, a text box is generated using the keypoint image, thus achieving text detection for the video banner.

[0030] Figure 1 A flowchart of a video banner text detection method provided by the present invention is shown below. Figure 1 As shown, the method includes:

[0031] S1 encodes the acquired banner video to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image.

[0032] Understandably, the banner video to be detected is encoded to obtain frame-by-frame video images, as well as the position of each macroblock containing text content and the motion vector information of each macroblock in each frame, and the macroblock position and motion vector information of each frame are stored.

[0033] S2, extract the key point information of each frame image, and use the key point information of the independently encoded frame image as the key point information of the reference frame image.

[0034] Understandably, for each frame of an image, key point information is extracted. As an example, extracting key point information for each frame includes: extracting deep and shallow feature information from any frame of the image using a ResNet backbone network, and fusing the deep and shallow feature information based on an FPN network to generate a fused feature map; inputting the fused feature map into a segmentation branch network to obtain a center feature map and a region feature map, and inputting the fused feature map into a detection network to obtain a key point probability map; binarizing the center feature map and the region feature map respectively to obtain a binarized center feature map. The keypoint probability map is initially filtered using the binarized center feature map as a mask. The maximum peak point in each mask region of the filtered keypoint probability map is taken as the center point. The keypoint probability map is then filtered using the binarized region feature map as a mask. The keypoints in the keypoint probability map are divided into k categories. The maximum peak point of each category of keypoints in different mask regions is extracted as the keypoint position of that category of keypoints in that mask region. The offset vector between adjacent keypoints in the same mask region is calculated, and the distance from the keypoint to the boundary of the set threshold is obtained based on the set threshold as the width information of the keypoint.

[0035] Understandably, for text information processing within each frame of an image, the method for extracting keypoint information is as follows: The frame to be detected is input into the model, and deep and shallow feature information is extracted using a ResNet+FPN feature extraction network. Then, the deep and shallow information are fused using a feature fusion module to obtain a fused feature map. This fused feature map is then placed into the segmentation branch to obtain a center feature map and a region feature map. Simultaneously, the fused feature map is placed into the detection branch to obtain a keypoint probability map. The center and region feature maps are binarized according to a given threshold. Then, the binarized center feature map is used as a mask to initially filter the keypoint probability map. The maximum peak point in each mask region of the filtered keypoint probability map is taken as the center point. Then, the binarized region feature map is used as a mask to further filter the keypoint probability map. Keypoints are strictly divided into k categories ({P0, P1, ..., P...). k-1 The maximum peak value of each category of keypoints in different mask regions is extracted as the keypoint position of that category in that mask region. The offset vector between adjacent keypoints within the same mask region is calculated. The resulting independently coded frame image is used as the reference frame keypoint image.

[0036] Among them, see Figure 2 Here is a network diagram for extracting key point information from a frame image. The steps for extracting key point information are as follows:

[0037] (1) Feature extraction:

[0038] like Figure 2 As shown, a typical model (ResNet-50) is used as the backbone network. Then, a borrowed channel reduction strategy is employed to reduce computational cost. Specifically, four feature maps (denoted as C2, C3, C4, and C5) generated by the conv2, conv3, conv4, and conv5 stages of the backbone are used for channel reconstruction. It is worth noting that the strides of the four feature maps relative to the input image are 4, 8, 16, and 32 pixels, respectively. Finally, by using 1×1 convolutional layers (including batch normalization (BN) and corrected linear units (ReLU)), the number of channels in each feature map is reduced to 256. The reduced feature maps are denoted as C′2, C′3, C′4, and C′5.

[0039] (2) Feature fusion:

[0040] FPN is employed to fuse feature maps from different levels, from top to bottom. First, four new feature maps (named P2, P3, P4, and P5) are calculated, each with 256 channels. The calculation method is as follows:

[0041] P5 = C′5;

[0042] P4 = 0.5 × C′4 + 0.5Up ×2 (P5);

[0043] P3 = 0.5 × C′3 + 0.5Up ×2 (P4);

[0044] P2 = 0.5 × C′2 + 0.5Up ×2 (P3);

[0045] Among them, Up ×2 (·) represents two upsampling operations performed using the nearest neighbor interpolation algorithm. The top-down path and lateral connection strategy generated some feature maps {P2, P3, P4, P5}, which are more informative than {C′2, C′3, C′4, C′5}.

[0046] To further integrate the semantic features between low-level and high-level features, feature map F is calculated, which is described as follows:

[0047] F = P2∪Up ×2 P3∪Up ×4 P4∪Up ×8 P5;

[0048] Where ∪ represents the channel connection, Up ×4 (·) and Up ×8 (·) represents 4 and 8 upsampling times, respectively. Then, a 3×3 convolutional layer (with BN and ReLU) is used to reduce the number of channels of F to 256.

[0049] (3) Detection:

[0050] The detection branch inputs the fused feature map into the probability map generation module, which generates a keypoint probability map through a Gaussian filter. The distance from the keypoint to the threshold boundary is obtained based on a certain threshold, which is used as the width information of the keypoint.

[0051] (4) Division:

[0052] Use split branches to compute the center feature map and the region feature map, such as Figure 2 As shown, using the fused feature map, two upsampling operations are first performed. Then, a 3×3 convolutional layer with a sigmoid activation function is used. Finally, two output feature maps are obtained: a center feature map and a region feature map.

[0053] (5) Binarization:

[0054] The center feature map and the region feature map are first passed through a threshold t. c and t k Binarization is performed; in this invention, t c and t k Set to t c =0.5, t k =0.5.

[0055] (6) Key point purification:

[0056] Binary region mapping is used as a mask to filter the keypoint probability map. Specifically, the binarized center feature map is used as a mask to initially filter the keypoint probability map, and the maximum peak point in each mask region of the filtered keypoint probability map is taken as the center point. Then, the binarized region feature map is used as a mask to filter the keypoint probability map, and the keypoints are strictly divided into k categories ({P0, P1, ..., P...). k-1 Extract the maximum peak value of each category of keypoint in different mask regions as the keypoint position of that category in that mask region, and calculate the offset of different types of keypoints from the center point within the same mask region.

[0057] The above processing is performed on each frame of image to extract key point information. The key point information of the frame with independent encoding is used as the key point information of the reference frame and stored for later use.

[0058] S3: Match the key point information of the current frame image with the position of each macroblock in the current frame image to find the key point information of each macroblock, and perform an offset operation on the key point information of each macroblock based on the displacement vector information of each macroblock in the current frame image.

[0059] Understandably, step S2 extracts keypoint information for each frame of the image, and step S1 obtains the position and movement vector information of each macroblock in each frame of the image. If it is necessary to identify banner text from the current frame image, the keypoint information of the current frame image is matched with the position of each macroblock in the current frame image, so that the keypoint coordinates and the position of the macroblock they belong to correspond one-to-one, that is, to find the keypoint information to which each macroblock belongs. For example, by using the coordinates of the keypoints and the coordinates of each macroblock, it is determined which macroblock each keypoint belongs to. The position of each macroblock in the current frame image may not be consistent with the position of each macroblock in the reference frame image. Therefore, it is necessary to offset the keypoint information in each macroblock of the current frame image based on the movement vectors of each macroblock in the current frame image and the movement vectors of each macroblock in the reference frame image, and then correct the position of the keypoint information in each macroblock of the current frame image to obtain the offset keypoint information of each macroblock in the current frame image.

[0060] S4. Match the key point information in the current frame image with the key point information in the reference frame image, and supplement the key point information in the current frame image based on the key point information in the reference frame image to obtain the complete key point information of the current frame image.

[0061] Understandably, video frame images may be occluded, causing key point information extraction to fail. Therefore, the key point information in the current frame image is matched with the key point information in the reference frame image. If they match completely, it means that the key points in the current frame image are not occluded; if they do not match completely, the key point information in the current frame image is supplemented based on the key point information in the reference frame image to obtain the complete key point information in the current frame image.

[0062] S5 generates a banner video text box based on the supplemented key point information in the current frame image.

[0063] Understandably, after obtaining the supplemented key point information in the current frame image, the key points are combined with the previously obtained offset vector and width information. Using the offset vector as the bottom line and the width as the vertical line, two boundary points are obtained. The boundary points obtained by connecting adjacent key points are used to obtain a polygon composed of the connected boundary points. This polygon is used as the text box of the banner image to realize text detection of the video banner.

[0064] See Figure 3This is an overall flowchart of a video banner text detection method provided by the present invention. The banner video is input into the designed banner detection algorithm, and the input video undergoes video decoding to obtain frames of images to be processed, along with corresponding macroblock and displacement vector information. The image blocks and motion vector information are stored. Each frame is input into the intra-frame text information processing module, where feature extraction and feature fusion are performed. The fused feature map is transformed into a keypoint probability map using Gaussian filtering, and then segmented into a center feature map and a region feature map. Both feature maps are binarized, and the binarized feature map is used as a mask for filtering to obtain the center point and keypoints. The obtained independently coded frame image is used as the reference frame keypoint image, and the current frame keypoint image is matched with macroblocks in the same frame to ensure a one-to-one correspondence between keypoint coordinates and their macroblock positions. The keypoints in the current frame are matched with the keypoints in the reference frame. If a complete match is found, the keypoints in the current frame are offset according to the motion vectors obtained above. If the match is not perfect, the key points are first processed using the steps for perfect matching described above. Then, the key point information from the reference frame is used to supplement the current frame. Finally, the supplemented and complete key points are used to generate a text box to achieve text detection for the video banner.

[0065] Figure 4 A structural diagram of a video banner text detection system provided in an embodiment of the present invention is shown below. Figure 4 As shown, a video banner text detection system includes an encoding module 401, an extraction module 402, a first matching module 403, a second matching module 404, and a generation module 405, wherein:

[0066] The encoding module 401 is used to encode the acquired banner video to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image.

[0067] The extraction module 401 is used to extract key point information of each frame image, and uses the key point information of the independently encoded frame image as the key point information of the reference frame image.

[0068] The first matching module 403 is used to match the key point information of the current frame image with the position of each macroblock of the current frame image, find the key point information to which each macroblock belongs, and perform an offset operation on the key point information to which each macroblock belongs based on the displacement vector information of each macroblock of the current frame image.

[0069] The second matching module 404 is used to match the key point information in the current frame image with the key point information in the reference frame image, and supplement the key point information in the current frame image based on the key point information in the reference frame image to obtain the complete key point information in the current frame image.

[0070] The generation module 405 is used to generate a banner video text box based on the complete key point information in the current frame image.

[0071] It is understood that the video banner text detection system provided by the present invention corresponds to the video banner text detection method provided in the foregoing embodiments. The relevant technical features of the video banner text detection system can be referred to the relevant technical features of the video banner text detection method, and will not be repeated here.

[0072] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, it implements the steps of the video banner text detection method.

[0073] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, it implements the steps of a video banner text detection method.

[0074] The video banner text detection method and system provided in this invention have the following beneficial effects:

[0075] 1. To address the issues of large aspect ratio and text distortion in banner text, a key-point-based label generation method is designed during the intra-frame text information processing stage. This method uses a set of key points to replace the text box outline points to represent text instances, thus adapting to text instances of different shapes. At the same time, a center point is selected from the key points to represent the text instance, avoiding the problem of detecting one text instance as multiple text instances due to excessively large text aspect ratio.

[0076] 2. To address the issue of inaccurate detection of banner text due to partial occlusion in video text detection, a method for fusing video-encoded image block information is designed during the inter-frame text information processing stage. This involves matching macroblock positions obtained from video decoding with keypoints derived from intra-frame text information processing. The matched keypoints are then compared with keypoints in a reference frame. This dual-matching mechanism accurately matches keypoints from different frames, supplementing keypoint information in cases where text occlusion causes keypoint loss, thereby improving the accuracy of banner text detection.

[0077] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0078] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0083] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting text on video banners, characterized in that, include: The acquired banner video is encoded to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image; Extract key point information from each frame of the image, and use the key point information of the frame image with independent encoding as the key point information of the reference frame image; Match the key point information of the current frame image with the position of each macroblock in the current frame image to find the key point information of each macroblock, and perform an offset operation on the key point information of each macroblock based on the displacement vector information of each macroblock in the current frame image. The key point information in the current frame image is matched with the key point information in the reference frame image. Based on the key point information in the reference frame image, the key point information in the current frame image is supplemented to obtain the complete key point information of the current frame image. Generate a banner video text box based on the complete key point information in the current frame image; The extraction of key point information for each frame of image includes: Deep and shallow feature information of any frame image is extracted using the ResNet backbone network, and the deep and shallow feature information is fused based on the FPN network to generate a fused feature map. The fused feature map is input into the segmentation branch network to obtain the center feature map and the region feature map, and the fused feature map is input into the detection network to obtain the key point probability map; The center feature map and the region feature map are binarized respectively to obtain a binarized center feature map and a binarized region feature map; The binarized center feature map is used as a mask to initially filter the key point probability map. The maximum peak point in each mask region of the filtered key point probability map is the center point. The binarized region feature map is used as a mask to filter the key point probability map. The key points in the key point probability map are divided into k categories. The maximum peak value of each category of key points in different mask regions is extracted as the key point position of that category of key points in that mask region. The vector between adjacent key points in the same mask region is calculated. The step of inputting the fused feature map into the detection network to obtain a keypoint probability map further includes: The distance from the key point to the boundary of the set threshold is obtained based on the set threshold, and is used as the width information of the key point; The step of generating a banner video text box based on complete key point information in the current frame image includes: Using the vector between adjacent key points as the baseline, and drawing a perpendicular line from the width of the key points to obtain two boundary points, the boundary points obtained by connecting the adjacent key points are used to form a polygon. This polygon is then used as the text box for the banner image.

2. The video banner text detection method according to claim 1, characterized in that, The ResNet backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer. The extraction of deep and shallow feature information from any frame of image using the ResNet backbone network includes: Four feature maps, C2, C3, C4, and C5, are generated for any frame image through the second, third, fourth, and fifth convolutional layers. The number of channels in each feature map is reduced to 256 using a 1×1 convolutional layer. The reduced feature map is denoted as... , , .

3. The video banner text detection method according to claim 2, characterized in that, The FPN-based method fuses the deep and shallow feature information to obtain a fused feature map, including: Feature maps are processed using an FPN network. , , Calculations were performed, resulting in four new feature maps. , , Each new feature map has 256 channels, calculated as follows: ； ； ； ； in, This involves two upsampling operations. Calculate the fused feature map F: Where ∪ represents a channel connection, and The number of channels in F was reduced to 256 by upsampling 4 times and 8 times respectively, using a 3×3 convolutional layer.

4. The video banner text detection method according to claim 1, characterized in that, The fused feature map is input into the segmentation branch network to obtain a center feature map and a region feature map, including: The segmentation branch network performs two upsampling operations on the fused feature map and uses a 3×3 convolutional layer with a Sigmoid activation function to output the center feature map and the region feature map; Accordingly, the central feature map and the region feature map are binarized respectively to obtain a binarized central feature map and a binarized region feature map, including: The center feature map and the region feature map are respectively subjected to thresholding. and Binarization is performed to obtain a binarized center feature map and a binarized region feature map, where... , .

5. A video banner text detection system, characterized in that, include: The encoding module is used to encode the acquired banner video to obtain continuous video frame images, as well as the position and displacement vector information of each macroblock in each frame image; The extraction module is used to extract key point information of each frame image, and uses the key point information of the independently encoded frame image as the key point information of the reference frame image. The first matching module is used to match the key point information of the current frame image with the position of each macroblock in the current frame image, find the key point information to which each macroblock belongs, and perform an offset operation on the key point information to which each macroblock belongs based on the displacement vector information of each macroblock in the current frame image. The second matching module is used to match the key point information in the current frame image with the key point information in the reference frame image, and supplement the key point information in the current frame image based on the key point information in the reference frame image to obtain the complete key point information in the current frame image. The generation module is used to generate a banner video text box based on the complete key point information in the current frame image; The extraction of key point information for each frame of image includes: Deep and shallow feature information of any frame image is extracted using the ResNet backbone network, and the deep and shallow feature information is fused based on the FPN network to generate a fused feature map. The fused feature map is input into the segmentation branch network to obtain the center feature map and the region feature map, and the fused feature map is input into the detection network to obtain the key point probability map; The center feature map and the region feature map are binarized respectively to obtain a binarized center feature map and a binarized region feature map; The binarized center feature map is used as a mask to initially filter the key point probability map. The maximum peak point in each mask region of the filtered key point probability map is the center point. The binarized region feature map is used as a mask to filter the key point probability map. The key points in the key point probability map are divided into k categories. The maximum peak value of each category of key points in different mask regions is extracted as the key point position of that category of key points in that mask region. The vector between adjacent key points in the same mask region is calculated. The step of inputting the fused feature map into the detection network to obtain a keypoint probability map further includes: The distance from the key point to the boundary of the set threshold is obtained based on the set threshold, and is used as the width information of the key point; The step of generating a banner video text box based on complete key point information in the current frame image includes: Using the vector between adjacent key points as the baseline, and drawing a perpendicular line from the width of the key points to obtain two boundary points, the boundary points obtained by connecting the adjacent key points are used to form a polygon. This polygon is then used as the text box for the banner image.

6. An electronic device, characterized in that, The device includes a memory and a processor, wherein the processor is used to execute computer management programs stored in the memory to implement the steps of the video banner text detection method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer management program, which, when executed by a processor, implements the steps of the video banner text detection method as described in any one of claims 1-4.

Images