A method for efficiently overlaying text labels based on video object classification

Abstract

This invention belongs to the field of video target monitoring systems, specifically relating to a method for efficiently overlaying text labels based on video target classification. The method includes the following steps: acquiring video stream data; decoding the video stream and outputting video frame data; placing the video frame data into a display queue for further processing; performing target recognition inference on the video frames, outputting the target's category name, confidence level information, and the target's position information in the image; querying the target's category name and confidence level information in a label cache; if a corresponding text label image exists, it is directly reused; otherwise, a corresponding text label image is rendered and stored in the label cache; after the query is completed, the images are stitched together to form a complete target label image; the complete target label image is overlaid on the video frames for annotation; and the annotated video frames are encoded and pushed.

Description

Technical Field

[0001] This invention belongs to the field of video target monitoring systems, specifically relating to a method for efficiently overlaying text tags based on video target classification. Background Technology

[0002] Existing video target monitoring systems typically identify monitored targets via video streams and display the results on images or video frames. However, in practical applications, this technology requires real-time text rendering of the identified target category names. When the category names are in Chinese or are long, the rendering process is time-consuming, leading to a decrease in the video processing frame rate and affecting the system's real-time performance. In particular, Chinese text contains many complex characters, and the font parsing and rendering process is more time-consuming than English text. When identifying and labeling multiple targets simultaneously, existing technologies need to pre-generate images for all possible category names and confidence combinations and load all label images into memory, consuming a large amount of space. This results in repetitive calculations and repeated use of video memory / RAM resources. Text rendering and overlay operations can easily become the performance bottleneck of the system, making it difficult to meet the real-time display requirements of high frame rates or high resolution videos under the needs of long-term continuous monitoring or large-scale deployment. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a method for efficiently overlaying text labels based on video target classification. This method can significantly improve the processing performance of video target monitoring systems while ensuring visual quality, meeting the processing requirements of real-time video streams, and is particularly suitable for the rendering optimization of Chinese category names.

[0004] The specific technical solution adopted in this invention is as follows:

[0005] A method for efficiently overlaying text labels based on video object classification, the key of which includes the following steps:

[0006] S1. Obtain video stream data from the server through the video access module;

[0007] S2. Use a decoder to decode the video stream and output video frame data;

[0008] S3. The video stream processing module receives the video frame data output by the decoder and puts the video frame data into the display queue for processing.

[0009] S4. The inference thread pool performs target recognition inference on the video frames and outputs the target detection results. The target detection results include the category name, confidence information and position information of each target in the image.

[0010] S5. Establish a label cache library. Based on the target detection results, the monitoring result drawing module performs cache queries on the category name and confidence information of each target in the label cache library. If a text label image corresponding to the target exists in the label cache library, it is directly reused. If it does not exist, the corresponding text label image is rendered and stored in the label cache library for subsequent reuse. After the query is completed, the category name text label image and the confidence text label image of each target are stitched together to form a complete target label image.

[0011] S6. The monitoring result drawing module overlays the complete target label image onto the video frame to complete the labeling and display of each target;

[0012] S7. The encoder encodes the labeled video frames, and the streaming module pushes the encoded video to the server to realize the transmission of the labeled video stream.

[0013] The process of creating the text label image in step S5 includes dynamically calculating the size of the label image based on the text content to be drawn, creating a semi-transparent background for the label image, and drawing the target category name or confidence level text on the background image to generate the text label image.

[0014] In step S2, the decoder uses hardware acceleration when decoding the video stream.

[0015] In step S3, the video stream processing module also completes data synchronization between multiple threads.

[0016] In step S3, the video stream processing module adopts a dual-queue asynchronous scheduling mechanism, with the display queue and processing queue running independently. The processing queue is used to cache video frame data that needs to be preprocessed, and the display queue is used to cache video frame data that has completed preprocessing.

[0017] In step S4, the inference thread pool calls the YOLO object detection model to perform object classification inference on multiple video frames in parallel.

[0018] The beneficial effects of this invention are:

[0019] This invention establishes a tag cache library. Based on the target detection results, it creates and stores text tag images of category names and confidence scores that do not exist in the tag cache library for subsequent reuse. This avoids repeated font parsing and glyph rendering operations, significantly reducing text rendering time from milliseconds to sub-milliseconds, and improving overall text rendering performance by more than 95%.

[0020] Targets of the same category share cached text images, avoiding repeated rendering. In multi-target, high-frequency annotation scenarios, it can still maintain a stable video processing frame rate of over 25fps, significantly reducing frame rate fluctuations, improving system real-time performance and screen smoothness, and also greatly reducing overall annotation time.

[0021] The tag cache library only stores the category text images actually used. Compared with the pre-generated image library scheme, it significantly reduces memory and storage space usage, while reducing CPU resource consumption, achieving a good balance between performance and resource utilization. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the overall architecture of a video target monitoring system.

[0023] Figure 2 This is a flowchart of the process for step S5; Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0025] In a specific embodiment, the present invention relates to a method for efficiently overlaying text tags based on video target classification. This method is based on a video target monitoring system, such as... Figure 1 As shown, the video target monitoring system includes a video access module, a decoder, a video stream processing module, an inference thread pool, a monitoring result rendering module, an encoder, and a streaming module. Crucially, it includes the following steps:

[0026] S1. Obtain H.264 / H.265 video stream data from the RTSP server through the video access module.

[0027] S2. Use a hardware-accelerated decoder to decode the H.264 / H.265 video stream and output YUV format video frame data.

[0028] S3. The video stream processing module receives video frame data output from the decoder, puts the video frame data into the display queue for processing, and manages data synchronization between multiple threads.

[0029] The video stream processing module adopts a dual-queue asynchronous scheduling mechanism, with the display queue and processing queue running independently. The processing queue is used to buffer video frame data that needs to be preprocessed, and the display queue is used to buffer video frame data that has been preprocessed, thereby realizing data synchronization among multiple threads of the decoder, processing queue, and display queue.

[0030] S4. The inference thread pool that manages the YOLO object detection model processes object detection tasks of multiple video frames in parallel. The inference thread pool performs object recognition inference on the video frames and outputs object detection results. The object detection results include the category name, confidence information and position information of each object in the image. At the same time, the YUV format video frame data is converted into RGB format.

[0031] S5. Establish a label cache library. Based on the target detection results, the monitoring result plotting module performs cache queries in the label cache library for the category name and confidence information of each target. Figure 2 As shown:

[0032] A buffered query is performed on the target category name. If the corresponding text label image exists in the label cache library, it is directly retrieved from the label cache library for reuse. If it does not exist, the corresponding text label image is generated based on the category name text and stored in the label cache library for subsequent reuse.

[0033] The confidence value (within the range of 0-100%) is cached and queried. If the label image with the corresponding confidence text already exists in the label cache library, it is directly retrieved from the label cache library for reuse. If it does not exist, the confidence value is converted into text form, the corresponding label image is rendered and stored in the label cache library.

[0034] The total number of tag images stored in the tag cache library includes images of numbers 0-9, images of percentage signs, and images of each category name. When there are 100 identified categories, and each category has 100 confidence values ​​(0-100%), the traditional storage method requires pre-storing 100 × 100 = 10,000 images. However, this invention stores 100 category names, 10 number images, and one percentage sign image, for a total of 111 images. Compared with the traditional storage method, this invention can greatly reduce the system memory usage.

[0035] After the query is completed, the text label images of the category names of each target and the text label images of the confidence scores are stitched together to form a complete target label image.

[0036] S6, the monitoring result drawing module overlays the complete target label image onto the RGB video frame to complete the labeling and display of each target.

[0037] S7. The encoder encodes the labeled video frames, and the streaming module pushes the encoded video to the RTMP server to realize the transmission of the labeled video stream.

[0038] The process of creating the text label image in step S5 includes dynamically calculating the size of the label image based on the text content to be drawn, creating a semi-transparent black background for the label image, and drawing white text of the target category name or confidence level on the background image to generate the text label image. The font, color, size and other styles of the text can be flexibly configured as needed.

[0039] This invention establishes a label cache library. Based on the target detection results, it creates and stores text label images of category names and confidence scores that are not present in the label cache library for subsequent reuse. This avoids repeated font parsing and glyph rendering operations, significantly reducing text rendering time from milliseconds to sub-milliseconds. The overall text rendering performance is improved by more than 95%. Targets of the same category share cached text images, avoiding repeated rendering. Even in multi-target, high-frequency annotation scenarios, it can still stably maintain a video processing frame rate of more than 25fps, significantly reducing frame rate fluctuations, improving system real-time performance and screen smoothness, and also greatly reducing the overall annotation time.

[0040] This method adopts an on-demand caching strategy. The tag cache library only stores the category text images that are actually used. Compared with the pre-generated image library scheme, it significantly reduces memory and storage space usage, while reducing CPU resource consumption, achieving a good balance between performance and resource utilization.

[0041] Since the complex font parsing and rendering process of Chinese text only needs to be executed once, and the cache can be fully reused afterwards, it effectively solves the problems of high time consumption and low efficiency in rendering Chinese text, and is particularly suitable for scenarios with dense Chinese labels such as aquaculture and agricultural monitoring.

[0042] The tag caching library supports dynamically adding new category names at runtime without predefining or regenerating resources, and has good scalability.

[0043] The rendering and production of text label images in this invention is based on pure CPU software implementation, without relying on GPU hardware. The monitoring system has low hardware dependency, is suitable for embedded devices, edge computing devices and cloud servers, and can be used in conjunction with existing hardware acceleration units to further improve the overall system performance.

Claims

1. A method for efficiently overlaying text labels based on video object classification, characterized in that, Includes the following steps: S1. Obtain video stream data from the server through the video access module; S2. Use a decoder to decode the video stream and output video frame data; S3. The video stream processing module receives the video frame data output by the decoder and puts the video frame data into the display queue for processing. S4. The inference thread pool performs target recognition inference on the video frames and outputs the target detection results. The target detection results include the category name, confidence information and position information of each target in the image. S5. Establish a label cache library. Based on the target detection results, the monitoring result drawing module performs cache queries on the category name and confidence information of each target in the label cache library. If a text label image corresponding to the target exists in the label cache library, it is directly reused. If it does not exist, the corresponding text label image is rendered and stored in the label cache library for subsequent reuse. After the query is completed, the category name text label image and the confidence text label image of each target are stitched together to form a complete target label image. S6. The monitoring result drawing module overlays the complete target label image onto the video frame to complete the labeling and display of each target; S7. The encoder encodes the labeled video frames, and the streaming module pushes the encoded video to the server to realize the transmission of the labeled video stream.

2. The method for efficiently overlaying text tags based on video object classification according to claim 1, characterized in that: The process of creating the text label image in step S5 includes dynamically calculating the size of the label image based on the text content to be drawn, creating a semi-transparent background for the label image, and drawing the target category name or confidence level text on the background image to generate the text label image.

3. The method for efficiently overlaying text tags based on video object classification according to claim 1, characterized in that: In step S2, the decoder uses hardware acceleration when decoding the video stream.

4. The method for efficiently overlaying text tags based on video target classification according to claim 1, characterized in that: In step S3, the video stream processing module also completes data synchronization between multiple threads.

5. The method for efficiently overlaying text tags based on video target classification according to claim 4, characterized in that: In step S3, the video stream processing module adopts a dual-queue asynchronous scheduling mechanism, with the display queue and processing queue running independently. The processing queue is used to cache video frame data that needs to be preprocessed, and the display queue is used to cache video frame data that has completed preprocessing.

6. The method for efficiently overlaying text tags based on video object classification according to claim 1, characterized in that: In step S4, the inference thread pool calls the YOLO object detection model to perform object classification inference on multiple video frames in parallel.

Images