Image target detection method, video stream compression method and apparatus

Abstract

This invention relates to an image target detection method, a video stream compression method, and an apparatus, belonging to the field of video stream compression technology. It addresses the problem of existing technologies lacking optimization of traffic monitoring video streams based on advanced coding and artificial intelligence technologies. The video stream is split into multiple images; an improved YOLOv8 algorithm performs target detection on the region of interest (ROI) of one image; a final binarized mask is generated for each image; the final binarized mask extracts the image of interest (ROI) and background image from the image containing the ROI; this process is repeated until ROI and background images are extracted from multiple images, then the process continues; multiple ROI images are combined to form the ROI video stream, and multiple background images are combined to form the background video stream; the ROI video stream and background video stream are compressed using different compression levels, and the resulting compressed video stream is obtained by merging the compressed ROI video stream and background video stream.

Description

Technical Field

[0001] This invention relates to the field of video stream compression technology, specifically to image target detection methods, video stream compression methods, and apparatus. Background Technology

[0002] With the development of artificial intelligence, intelligent transportation systems have become an important trend in modern urban development, and the storage and processing of traffic monitoring video streams are key components of these systems. As a core part of intelligent transportation systems, the real-time performance and accuracy of traffic monitoring video streams directly impact the effectiveness of traffic management and urban traffic safety. Intelligent surveillance cameras can capture real-time traffic conditions on the road, including traffic density, speed, and vehicle type, providing real-time traffic status data to traffic management departments. This data can not only be used for real-time traffic control but also, through historical data analysis, guide traffic planning and road construction, improving urban transportation efficiency. Simultaneously, traffic monitoring video streams can promptly detect and handle traffic accidents and congestion, providing strong support for urban traffic safety. However, the characteristics of traffic monitoring video streams require corresponding hardware to store massive amounts of data, but the large storage space and high-resolution imaging equipment are very expensive. For example, the acquired YUV (video color encoding system) video resolution is 1080p, and a medium-sized city has at least 5,000 traffic monitoring cameras. At a bitrate of 4 Mbps, the concurrently written bitstream can reach 1648 TB per day. Furthermore, due to the unique nature of traffic surveillance video streams, they typically need to be stored for a certain number of days. Faced with such massive video streams, improving encoding efficiency, accelerating transmission speed, reducing storage costs, and processing and analyzing video stream information in a timely and effective manner have become a series of significant challenges.

[0003] Currently, advanced video coding standards such as H.265 / HEVC (High-Efficiency Video Coding) are widely adopted. Compared to existing coding standards, H.265 / HEVC can reduce the video stream file size by approximately 50% while maintaining the same video stream quality. However, even so, it still cannot well adapt to the characteristics of traffic monitoring video streams. Due to the complexity of road environments and the variability of traffic conditions, traffic monitoring video streams often contain a large number of static areas and repetitive information. Existing coding standards have limited effectiveness in compressing this information, leading to wasted resources and low processing efficiency, failing to meet the needs of intelligent transportation system construction.

[0004] With the development of artificial intelligence technology, deep learning-based video coding technologies are constantly emerging. Among them, traffic monitoring video stream compression based on regions of interest (ROIs) has become a research direction of great interest. By identifying and extracting ROIs from traffic monitoring video streams, the size of video stream files can be effectively reduced, lowering storage and transmission costs while ensuring the effective capture and transmission of traffic information. This compression technology can process video stream data in a targeted manner according to the characteristics of the actual monitoring scenario, avoiding repeated encoding of static and repetitive information, and improving compression efficiency and resource utilization. Furthermore, the extracted ROIs can be further developed at higher levels, such as adding functions like hazard warnings, traffic flow monitoring, license plate recognition, and vehicle violation monitoring, making intelligent video surveillance systems more flexible and efficient, and improving the monitoring and analysis effects in high-risk areas.

[0005] In conclusion, optimizing the data processing and storage of traffic monitoring video streams through advanced coding and artificial intelligence technologies has become a crucial research topic in intelligent transportation systems. This will help improve the efficiency and safety of urban traffic management and provide strong technical support for the development of intelligent transportation systems. Summary of the Invention

[0006] This invention solves the problem of existing technologies lacking optimization of traffic monitoring video streams based on advanced coding and artificial intelligence technologies.

[0007] The improved YOLOv8 image target detection method described in this invention involves sequentially inputting an image into the YOLOv8 backbone network, the YOLOv8 neck network, and the YOLOv8 head network for image target detection. Specifically:

[0008] The images are input into the CSC module in the backbone network and the CSC module in the neck network of YOLOv8 respectively for feature extraction and output feature maps;

[0009] In the CSC module, the image is processed by the CBS module to extract features and output a feature map. The feature map is split into two branches. One branch feature map is directly input into Concat, and the other branch feature map is processed by multiple SC-Block modules to fuse features and output a fused feature map. After the two branches feature maps are concatted, they are processed again by the CBS module to output a feature map.

[0010] In the SC-Block module, the feature map of another branch is fused through CAA and star operations to output the fused feature map.

[0011] Furthermore, in one embodiment of the present invention, the other branch feature map is fused using CAA and star operations, specifically as follows:

[0012] ;

[0013] in, The feature map of another branch is fused using CAA and star operations. For adaptive weighting coefficients, To extract a specific location from an image Local features, For a certain position The corresponding global context information.

[0014] Furthermore, in one embodiment of the present invention, the other branch feature map is fused using CAA and star operations to output the fused feature map, specifically as follows:

[0015] The other branch feature map is processed by CAA to extract features and output a feature map. The feature map is then processed by DW-Conv to divide it into a first branch and a second branch. The feature maps of the first branch and the second branch are respectively input into the FC layer for operation to generate feature maps again. The star operation merges the feature maps of the first branch and the second branch to generate an updated feature map. The updated feature map is then input into the FC layer and DW-Conv in sequence to generate a fused feature map.

[0016] Furthermore, in one embodiment of the present invention, the updated feature map specifically refers to:

[0017] ;

[0018] in, For the updated feature map, For feature maps, For adaptive weights, For logical operators.

[0019] The video stream compression method of this invention, implemented using the improved YOLOv8 image target detection method described above, includes the following steps:

[0020] Step S1: The video stream is split into multiple images;

[0021] Step S2: Target detection is performed on the region of interest of an image based on the improved YOLOv8 to form an image with the region of interest;

[0022] Step S3: A preliminary binary mask image is generated from an image. The preliminary binary mask image is then subjected to erosion, dilation, and Gaussian blur operations in sequence to generate the final binary mask image.

[0023] Step S4: The final binarized mask image extracts the image of interest and the background image from an image with a region of interest.

[0024] Step S5: Repeat steps S2 to S4 until the image of interest and the background image are extracted from multiple images, then proceed to step S6.

[0025] Step S6: Multiple images of interest are combined to form a video stream of interest, and multiple background images are combined to form a background video stream;

[0026] Step S7: The video stream of interest and the background video stream are merged and compressed using different compression levels, and the compressed video stream is obtained based on the merged and compressed video stream of interest and background video stream.

[0027] Furthermore, in one embodiment of the present invention, the erosion operation on the preliminary binary mask image in step S3 specifically includes:

[0028] ;

[0029] in, To perform erosion on the initial binarized mask image, For the first One image, The structuring element used when performing erosion on the initial binarized mask image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. These are the pixel coordinates of the image. This is a preliminary binarized mask image;

[0030] The dilation operation on the initial binarized mask image is specifically as follows:

[0031] ;

[0032] in, To perform a dilation operation on the initial binarized mask image. The structuring element used when performing dilation operations on the initial binarized mask image.

[0033] Furthermore, in one embodiment of the present invention, in step S3, the final binarized mask image specifically comprises:

[0034] ;

[0035] in, For the final binarized mask image, The pixel coordinates of the image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. For Gaussian blur, Let the standard deviation of the kernel function of the Gaussian blur be the standard deviation. This is to perform a dilation operation on the initial binarized mask image.

[0036] Furthermore, in one embodiment of the present invention, in step S6, the video stream of interest specifically refers to:

[0037] ;

[0038] in, For video streams of interest, For the final binarized mask image, For the first One image, These are the pixel coordinates of the image. A sequence of images containing a region of interest;

[0039] The background video stream mentioned above is specifically:

[0040] ;

[0041] in, For background video stream, This is the inverse mask of the final binarized mask image.

[0042] The traffic monitoring video compression device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the video stream compression method described above.

[0043] This invention addresses the problem of existing technologies lacking advanced encoding and artificial intelligence technologies for optimizing traffic monitoring video streams. Specific beneficial effects include:

[0044] 1. The improved YOLOv8 image target detection method described in this invention addresses the challenges of existing technologies where video streams are susceptible to various interferences and noises in complex and ever-changing environments. These interferences directly impact the accuracy of image target detection. Furthermore, video streams are typically generated continuously, resulting in massive amounts of data, requiring the system to efficiently process and analyze the video streams within a short timeframe. To address these technical problems, this invention designs an improved YOLOv8 system. This improved YOLOv8 maintains high efficiency and accuracy in complex environments while simultaneously responding to and processing massive video streams in real-time, ensuring real-time performance in practical applications.

[0045] 2. The improved YOLOv8 image target detection method described in this invention introduces a newly designed SC-Block module through the proposed CSC module. The SC-Block module significantly reduces computational costs through DW-Conv and star operations. Specifically, DW-Conv effectively reduces the amount of computation and parameters, while star operations optimize the dynamic processing of feature maps, enabling the improved YOLOv8 to better handle features at different scales while maintaining high efficiency.

[0046] 3. The improved YOLOv8 image target detection method described in this invention incorporates CAA into the SC-Block module. This mechanism can obtain contextual information of distant targets, thereby improving the accuracy and robustness of detecting small distant targets in video streams.

[0047] 4. The video stream compression method of this invention addresses the shortcomings of existing video coding algorithms, which typically apply a uniform compression standard to the entire image without differentiating based on image content. This leads to resource waste and low processing efficiency. To solve these problems, this invention utilizes an improved YOLOv8 algorithm to automatically identify and extract the region of interest (ROI) from the video stream, generating a final binarized mask image. The video stream is then divided into two parts: the ROI video stream and the background video stream, each using a different compression strategy. The ROI video stream, containing more important information, receives higher compression quality, while the background video stream receives lower compression quality, effectively reducing the overall size of the video file. This avoids the problems of resource waste and low processing efficiency.

[0048] 5. The traffic monitoring video compression device of this invention lacks technical means based on advanced coding and artificial intelligence technologies to optimize traffic monitoring videos. To solve the above technical problem, this invention uses an improved YOLOv8 for image target detection, identifies and marks the region of interest in the video stream, and then preprocesses each image to generate a final binarized mask image. The final mask image is used to separate the video stream into two parts: a video stream of interest and a background video stream. Different compression levels are applied to each part, thereby significantly reducing the video stream file size while ensuring the effective capture of key traffic monitoring video stream information.

[0049] The video stream compression method described in this invention, when applied to the analysis of key areas, performs better than existing uniform compression methods. Attached Figure Description

[0050] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 This is a diagram of the improved YOLOv8 structure described in Implementation Method 1;

[0052] Figure 2 This is a structural diagram of the CSC module as described in Implementation Method 1;

[0053] Figure 3 This is a structural diagram of the SC-Block module as described in Implementation Method 1;

[0054] Figure 4 This is the video stream separation diagram described in Implementation Method 2;

[0055] Figure 5 This is a comparison chart of various video stream quality evaluation indicators between the existing uniform compression method described in Embodiment 3 and the traffic monitoring video compression device described in this embodiment. Detailed Implementation

[0056] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0057] Implementation Method 1: The improved YOLOv8 image target detection method described in this implementation method involves sequentially inputting the image into the YOLOv8 backbone network, the YOLOv8 neck network, and the YOLOv8 head network for image target detection. Specifically:

[0058] The images are input into the CSC module in the backbone network and the CSC module in the neck network of YOLOv8 respectively for feature extraction and output feature maps;

[0059] In the CSC module, the image is processed by the CBS module to extract features and output a feature map. The feature map is split into two branches. One branch feature map is directly input into Concat, and the other branch feature map is processed by multiple SC-Block modules to fuse features and output a fused feature map. After the two branches feature maps are concatted, they are processed again by the CBS module to output a feature map.

[0060] In the SC-Block module, the feature map of another branch is fused through CAA and star operations to output the fused feature map.

[0061] In this embodiment, the other branch feature map undergoes feature fusion via CAA and star operations, specifically as follows:

[0062] ;

[0063] in, The feature map of another branch is fused using CAA and star operations. For adaptive weighting coefficients, To extract a specific location from an image Local features, For a certain position The corresponding global context information.

[0064] In this embodiment, the other branch feature map is fused using CAA and star operations to output the fused feature map, specifically as follows:

[0065] The other branch feature map is processed by CAA to extract features and output a feature map. The feature map is then processed by DW-Conv to divide it into a first branch and a second branch. The feature maps of the first branch and the second branch are respectively input into the FC layer for operation to generate feature maps again. The star operation merges the feature maps of the first branch and the second branch to generate an updated feature map. The updated feature map is then input into the FC layer and DW-Conv in sequence to generate a fused feature map.

[0066] In this embodiment, the updated feature map is specifically as follows:

[0067] ;

[0068] in, For the updated feature map, For feature maps, For adaptive weights, For logical operators.

[0069] In complex and ever-changing environments, video streams can be affected by various interferences and noises, such as changes in lighting, fluctuations in weather conditions (e.g., rain, snow, fog, haze), object occlusion, and camera shake. These interference factors directly affect the accuracy of image target detection. Furthermore, video streams are typically generated continuously, resulting in massive amounts of data, requiring the system to process and analyze the video stream efficiently within a short timeframe. To meet this requirement, YOLOv8 (YOLO version 8) not only needs high accuracy to precisely identify image targets and delineate the boundaries of regions of interest, but also extremely high processing speed to respond to and process massive video streams in real time, ensuring real-time performance in practical applications.

[0070] To solve the above technical problems, such as Figure 1 As shown, this embodiment uses YOLOv8 as the base model to design an improved YOLOv8 image target detection method, specifically as follows:

[0071] Video streams are typically formed by continuously capturing images. Therefore, splitting a video stream into multiple images allows for object detection within a single image:

[0072] First, in the backbone network of YOLOv8, a processed image is input into the Conv layer (convolutional layer) to reduce the dimensionality of the image. After passing through the Conv layer, the image passes through four cascaded Conv layers and the CSC module (Star Anchor Context Feature Extraction Module) to output a feature map. The feature map is then fed into the SPPF layer (spatial pooling layer) for feature extraction again, and a feature map is output.

[0073] In the existing technology, although the C2f module (feature extraction module) in the backbone network of YOLOv8 improves the feature extraction capability of the network to a certain extent, its structure is highly complex and computationally expensive. Furthermore, when processing consecutive frames of a video stream, it suffers from problems such as repeated feature calculation and insufficient real-time performance.

[0074] To address the aforementioned technical problems, this embodiment includes a CSC module, such as... Figure 2 As shown, in the CSC module, the feature map first undergoes preliminary feature extraction and transformation using a CBS structure (convolution, batch normalization, SiLU activation function (adaptive activation function)) to output a feature map. Next, the feature map is split into two branches for processing. One branch's feature map is directly passed to the Concat layer (feature fusion layer), while the other branch's feature map is processed by multiple SC-Block modules (anchor context modules). Finally, the feature maps from the two branches are concatenated along the channel dimension to achieve feature fusion.

[0075] The CSC module not only maintains the feature extraction capabilities of the original C2f module, but also further optimizes the computation path, reduces redundant computations, and significantly improves real-time processing speed through structural simplification.

[0076] In the SC-Block module, such as Figure 3 As shown, firstly, the feature map utilizes CAA (Context Anchor Attention) to capture contextual information of distant targets and enhances the expressive power of the central features during feature extraction, thereby outputting a feature map. Next, the extracted feature map is input into DW-Conv (Depthwise Separable Convolution) for convolution operations, splitting into a first branch and a second branch. The feature maps from the first and second branches are then input into a fully connected (FC) layer for further operations to generate new feature maps. Finally, the feature maps undergo intelligent merging via star operations and are sequentially input into FC layers and DW-Conv to generate a fused feature map.

[0077] CAA effectively captures the contextual information of distant targets and, through the interaction between star operations and CAA, weightedly fuses the contextual information of distant targets with central features, enabling the model to maintain high accuracy when processing small, distant targets. This solves the technical challenge of processing the contextual information and central features of distant targets and overcomes the shortcomings of existing attention mechanisms in long-distance dependency modeling.

[0078] Specifically, the effects of CAA and star operations are achieved through feature fusion using the following formula:

[0079] ;

[0080] in, The feature map of another branch is fused using CAA and star operations. For adaptive weighting coefficients, To extract a specific location from an image Local features, For a certain position The corresponding global context information.

[0081] Building upon this, star operations optimize the dynamic processing of the feature map through point-by-point weighting. Assume the input feature map is... ,in, The real number field represents the range of each pixel value in the feature map. For the number of channels, For height, Width. Star operation on feature map. Apply point-by-point adaptive weights Generate updated feature maps:

[0082] ;

[0083] in, For the updated feature map, For logical operators.

[0084] The star operation achieves adaptive weighting of feature maps through pointwise multiplication, enabling YOLOv8 to dynamically adjust the importance of each feature point based on the feature distribution, thereby optimizing the sensitivity of local features to targets at different scales. Furthermore, this operation addresses the issue of insufficient feature response after the introduction of CAA, resulting in more refined feature map processing.

[0085] After combining the CAA-enhanced feature map with the star operation-optimized feature map, the final generated feature map is output using the following formula:

[0086] ;

[0087] in, It is a non-linear activation function. This is the final generated feature map.

[0088] Therefore, star operations and CAA solve the problems of feature redundancy and local information loss while maintaining low computational complexity and improving adaptability to targets of different scales.

[0089] Secondly, in the neck network of YOLOv8, such as Figure 1 As shown, the feature map first passes through an Upsample layer, and then, together with the feature map extracted by the CSC module in the backbone network, it is input into the Concat layer to output a feature map. This feature map is then fed into the CSC module for feature extraction, outputting another feature map. This feature map again passes through the Upsample layer, the CSC module, and the CSC module to output the first-dimensional feature map. Subsequently, the first-dimensional feature map is sent to the Conv layer for dimensionality reduction. The dimensionality-reduced feature map is then compared with the feature map extracted by the CSC module. Figure 1 The input is then fed into the Concat layer, which outputs a feature map. This feature map is then processed by the CSC module to extract features, outputting a second-sized feature map. This feature map is then passed through the Conv layer for dimensionality reduction. The dimensionality-reduced feature map is then compared with the feature map extracted by the SPPF layer. Figure 1 The input is then fed into the Concat layer, which outputs a feature map. This feature map is then processed by the CSC module to extract features and output a third-sized feature map.

[0090] Finally, in the YOLOv8 head network, the feature maps of the first, second, and third dimensions are all input into the Detect head for image object detection.

[0091] Therefore, the improved YOLOv8 designed in this embodiment not only has high accuracy, capable of accurately identifying target objects and marking the boundaries of regions of interest, but also must have extremely high processing speed, capable of responding to and processing massive amounts of video data in real time, to ensure the real-time requirements in practical application scenarios.

[0092] Implementation Method 2: The video stream compression method described in this implementation method is implemented using the improved YOLOv8 image target detection method described in Implementation Method 1, and includes the following steps:

[0093] Step S1: The video stream is split into multiple images;

[0094] Step S2: Target detection is performed on the region of interest of an image based on the improved YOLOv8 to form an image with the region of interest;

[0095] Step S3: A preliminary binary mask image is generated from an image. The preliminary binary mask image is then subjected to erosion, dilation, and Gaussian blur operations in sequence to generate the final binary mask image.

[0096] Step S4: The final binarized mask image extracts the image of interest and the background image from an image with a region of interest.

[0097] Step S5: Repeat steps S2 to S4 until the image of interest and the background image are extracted from multiple images, then proceed to step S6.

[0098] Step S6: Multiple images of interest are combined to form a video stream of interest, and multiple background images are combined to form a background video stream;

[0099] Step S7: The video stream of interest and the background video stream are merged and compressed using different compression levels, and the compressed video stream is obtained based on the merged and compressed video stream of interest and background video stream.

[0100] In this embodiment, the erosion operation on the preliminary binary mask image in step S3 specifically includes:

[0101] ;

[0102] in, To perform erosion on the initial binarized mask image, For the first One image, The structuring element used when performing erosion on the initial binarized mask image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. These are the pixel coordinates of the image. This is a preliminary binarized mask image;

[0103] The dilation operation on the initial binarized mask image is specifically as follows:

[0104] ;

[0105] in, To perform a dilation operation on the initial binarized mask image. The structuring element used when performing dilation operations on the initial binarized mask image.

[0106] In this embodiment, the final binarized mask image in step S3 is specifically:

[0107] ;

[0108] in, For the final binarized mask image, These are the pixel coordinates of the image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. For Gaussian blur, Let the standard deviation of the kernel function of the Gaussian blur be the standard deviation. This is to perform a dilation operation on the initial binarized mask image.

[0109] In this embodiment, the video stream of interest in step S6 specifically refers to:

[0110] ;

[0111] in, For video streams of interest, For the final binarized mask image, For the first One image, These are the pixel coordinates of the image. A sequence of images containing a region of interest;

[0112] The background video stream mentioned above is specifically:

[0113] ;

[0114] in, For background video stream, This is the inverse mask of the final binarized mask image.

[0115] In video streams, there are typically unchanging background regions and moving foreground regions. Background regions occupy the majority of the image area; these regions usually change little and contain a large amount of redundant information. Moving foreground regions, while smaller in area, contain more important information than the background regions, such as dynamic objects like vehicles and pedestrians. These moving foreground regions are often key to the video stream because they provide crucial data about the video stream. However, existing video coding algorithms have limited effectiveness in handling these characteristics. Existing algorithms do not fully utilize the redundancy of background regions, leading to repeated encoding of static backgrounds and wasting significant storage space and transmission bandwidth. Furthermore, because these algorithms typically use a uniform compression standard for the entire image without differentiating processing based on image content, they result in resource waste and low processing efficiency.

[0116] To address the aforementioned technical problems, this embodiment presents a video stream compression method, comprising the following steps:

[0117] Step S1, Extraction of Region of Interest from Video Stream:

[0118] Video streams captured by cameras are typically formed by continuously capturing images; therefore, a video stream is split into multiple images, denoted as... , Indicates the first Multiple images. After being acquired, multiple images are typically stored and transmitted in YUV video encoding format. To process and analyze these multiple images, they usually need to be decoded into the standard RGB image format first. Based on the improved YOLOv8 described in Implementation Method 1, target detection is performed on the regions of interest (ROIs) of the multiple images, resulting in multiple images with ROIs.

[0119] Step S2: Generate the final binarized mask image.

[0120] To generate a preliminary binary mask for an image. .in, Represents the pixel coordinates of the image, iterating through each pixel. When at least one bounding box exists... ,in, For the first A set of bounding boxes for regions of interest in an image, such that pixels When located within this bounding box, then The value of is 1, and otherwise 0. Thus, This results in a binary image where each pixel is either 1 (belonging to any region of interest) or 0 (not belonging to any region of interest). The specific formula is as follows: ;

[0121] in, After nonmaximum suppression, the first The set of all bounding boxes in the frame. It is The coordinates of the top left and bottom right corners of the bounding box.

[0122] After generating the initial binary mask image, further post-processing is required. These post-processing operations mainly include morphological operations such as dilation and erosion. The main purpose of dilation is to expand the foreground region, i.e., the area with a value of 1 in the mask. This can fill small holes in the foreground region and expand the boundaries of foreground objects outward. The main purpose of erosion is to shrink the foreground region, i.e., the area with a value of 1 in the mask. This can remove edge noise of foreground objects and shrink the boundaries of foreground objects inward. The initial binary mask image is first eroded and then dilated. as well as These are the structuring elements used during erosion and expansion operations, respectively.

[0123] ;

[0124] ;

[0125] in, To perform erosion on the initial binarized mask image, For the first One image, For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. These are the pixel coordinates of the image. This is to perform a dilation operation on the initial binarized mask image.

[0126] The initial binarized mask image, after post-processing, improves its quality and accuracy.

[0127] Assume the kernel function of Gaussian blur is ,in, Generate the final binarized mask image by calculating the standard deviation of the kernel function for Gaussian blur. It can be represented as:

[0128] ;

[0129] Gaussian blur helps reduce boundary discontinuities, making the initial binarized mask more suitable for subsequent image processing steps. Furthermore, the edges of the initial binarized mask after Gaussian blurring become softer and smoother, resulting in less sharp boundaries and more natural transitions.

[0130] Step S3, video stream separation:

[0131] like Figure 4 As shown, the final binarized mask image is used. Extract the image of interest and the background image from multiple images with regions of interest. Specifically, for each pixel location... Perform a pixel-wise bitwise AND operation, and use the final binarized mask image to extract multiple images of interest from multiple images with regions of interest. That is, retain pixels with a value of 1 in the mask image, and set other pixels to 0. Similarly, for each pixel position... A pixel-wise bitwise AND operation is performed. The inverse mask of the final binarized mask image is used to extract multiple background images from multiple images containing regions of interest (ROIs). This means pixels with a value of 1 in the inverse mask image are retained, while other pixels are set to 0. Multiple ROI images are combined to form the ROI video stream, and multiple background images are combined to form the background video stream. Assume the sequence of images containing ROIs is... The video stream of interest and the background video stream can be represented as follows:

[0132] ;

[0133] ;

[0134] in, For video streams of interest, For background video stream, This is the inverse mask of the final binarized mask image.

[0135] Step S4, layer-by-layer compression and merging of the video stream:

[0136] like Figure 4As shown, the two video streams, the video stream of interest and the background video stream, are then processed using different compression levels. Specifically, the existing H.264 / H.265 codec framework is selected for compression of both the video stream of interest and the background video stream. The resulting video stream compression method is called CA-X264 / CA-X265, and the CRF (Constant Rate Factor) is used to control the quality of the video stream. In CRF mode, the encoder automatically adjusts the quantization value to ensure that the output video stream reaches the user-specified quality level. Specifically, when a smaller CRF value is set, the encoder maintains higher quality, but the bit rate increases accordingly. That is, a smaller CRF value is set for the video stream of interest to ensure the integrity of critical information. Conversely, when a larger CRF value is set, the encoder compresses at a higher bit rate. That is, a larger CRF value is set for the background video stream to reduce image quality while reducing storage and transmission costs. By adjusting the CRF value, the quality of the video stream can be easily controlled without manually adjusting more complex parameters.

[0137] The two video streams are then added together to obtain the final compressed video stream.

[0138] Therefore, this implementation utilizes an improved YOLOv8 to automatically identify and extract regions of interest (ROIs) from the video stream, generating a final binarized mask image. The video stream is then divided into two parts: the ROI stream and the background stream, and different compression strategies are employed. The ROI stream, containing more important information, receives higher compression quality, while the background stream uses lower compression quality, effectively reducing the overall size of the video file.

[0139] Implementation Method 3: The traffic monitoring video compression device described in this implementation method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the video stream compression method as described in Implementation Method 2.

[0140] This embodiment applies the video stream compression method described in Embodiment 2 to a traffic monitoring video compression device, and includes the following steps:

[0141] Step S1, marking the region of interest:

[0142] First, video stream data from real traffic monitoring scenarios was collected. Using random frame extraction, 4000 monitoring images were saved as a dataset, encompassing different weather conditions, intersections, and lighting conditions. Then, annotation software was used to annotate these images; the annotated content represents the desired region of interest. In this implementation, the annotation categories selected were non-motorized vehicles, pedestrians, ordinary vehicles, and special-purpose vehicles.

[0143] Step S2, improve YOLOv8:

[0144] In both the backbone and neck network of YOLOv8, CSC modules are designed to extract features and output feature maps. After the improvement, YOLOv8 is trained using the previously labeled dataset.

[0145] Step S3, Extraction of Region of Interest from Video Stream:

[0146] On the computer, the `cv2.VideoCapture` function from OpenCV (a cross-platform computer vision library) is used to retrieve a video stream using the RTSP (Real-Time Streaming Protocol). Then, the `cap.read()` function reads images frame by frame from the video stream. The `cv2.cvtColor` and `cv2.resize` functions are then used to decode and preprocess the read images. A pre-trained ONNX (Open Neural Network Exchange) model is used for object detection. The ONNX Runtime library provides the functionality to load and infer ONNX model formats. The model is loaded using `onnxruntime.InferenceSession`, and inference is performed using `session.run` to obtain the object detection results, including bounding boxes for multiple object regions and their corresponding confidence scores. Post-processing is then applied to the detection results, using Non-Maximum Suppression (NMS) to remove overlapping bounding boxes, resulting in a final list containing coordinate information of the regions of interest.

[0147] Step S4: Generate the final binarized mask image and video stream separation:

[0148] The final binarized mask is generated based on the bounding boxes after NMS. Each pixel value in the final binarized mask is either 1 or 0. The detailed process is as follows: Initialize a zero-matrix of the same size as the image as the binarized mask. Iterate through all bounding boxes, setting the pixel values ​​within each bounding box to 1. Perform erosion and dilation operations on the initialized binarized mask sequentially to remove noise and enhance the region of interest. Use Gaussian blur to smooth and generate the final binarized mask. Then, perform a bitwise AND operation between the smoothed final binarized mask and its inverse mask with the video stream, separating the video stream into two paths: one containing only the video stream of interest, and the other containing only the background video stream.

[0149] Step S5: Layered compression and merging of the video stream:

[0150] The separated video streams are compressed. Different compression quality parameters are used for different video streams. Compression of the video stream of interest uses a higher compression quality parameter, i.e., a lower CRF value, to retain more detail. Compression of the background video stream uses a lower compression quality parameter, i.e., a higher CRF value, to reduce file size. Finally, the two compressed video streams are added element-wise to obtain the processed video stream.

[0151] To comprehensively evaluate the performance differences between the traffic monitoring video compression device described in this embodiment and the existing CA-Comp (uniform compression method and content-aware compression) method in video stream processing, this embodiment will conduct a detailed video stream compression study using FFmpeg's X.264 and X.265 codecs, i.e., the existing uniform compression method. The focus will be on comparing the performance of these two compression methods at different CRF values, analyzing their specific impact on video stream quality and compression efficiency. In this study, 23 and 28 will be used as the default CRF values ​​in H.264 and H.265, respectively. These CRF values ​​correspond to compression ratios of approximately 1:250 and 1:375.

[0152] Subsequently, various video stream quality assessment metrics, including PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity), will be used to objectively evaluate the quality of the compressed video stream. In addition, VAMF (Video Quality Assessment Method) will be used for subjective evaluation of the video stream. Using binarized mask images generated by Mask_PSNR (an image quality assessment metric) and Mask_SSIM (an image quality assessment metric), the PSNR and SSIM values ​​of the region of interest in each frame will be calculated. All evaluation metrics will be calculated using the professional tool MSU Video Quality Measurement Tool.

[0153] like Figure 5 The diagram shows the quality of reconstructed frames after compression of a traffic monitoring video stream using the traffic monitoring video compression device described in this embodiment and an existing uniform compression method. The test video stream is a segment of video stream pulled in real time from traffic monitoring. "H.26..." indicates the use of an existing uniform compression method, "CA-X26..." indicates the use of the traffic monitoring video compression device described in this embodiment, ROI represents the video stream of interest, and N-ROI represents the background video stream.

[0154] For the traffic monitoring video compression device described in this embodiment and existing uniform compression methods, PSNR, SSIM, and VAMF all decrease significantly with increasing CRF values. The quality degradation is particularly pronounced when the CRF value reaches 28 or higher.

[0155] Furthermore, at the same compression ratio, for example, for numbers C and D, and numbers I and J, the existing uniform compression methods achieve higher PSNR and SSIM values ​​than the traffic monitoring video compression device described in this embodiment. This is because the background video stream accounts for a large proportion of the test video stream, approximately 70%, while the traffic monitoring video compression device described in this embodiment uses a higher CRF value for the background video stream. This leads to a significant decrease in the quality of the background video stream, thereby lowering the overall video stream's PSNR and SSIM values. Although the video stream of interest maintains higher quality, consistent with its design intent, the widespread presence of the background video stream results in a trade-off in overall video stream quality. This is particularly evident in the results of VAMF, Mask_PSNR, and Mask_SSIM, three metrics that focus on the quality of the video stream of interest. The traffic monitoring video compression device described in this embodiment outperforms the existing uniform compression methods in all three metrics.

[0156] It can be concluded that the advantages of the traffic monitoring video compression device described in this embodiment are mainly reflected in applications with clearly defined video streams of interest. If the importance of the video stream of interest is much higher than that of the background video stream, the traffic monitoring video compression device described in this embodiment is obviously a better choice. Conversely, when the importance of each part in the video stream is relatively uniform, existing uniform compression methods may still be a better choice because they can provide more consistent overall quality.

[0157] In summary, the traffic monitoring video compression device described in this embodiment achieves a higher compression ratio by sacrificing the quality of the background video stream while maintaining the quality of the video stream of interest. This method may outperform existing uniform compression methods in specific applications, such as key area analysis in surveillance videos or lesion detection in medical images.

[0158] In summary, by using the final binarized mask image to separate the video stream into two parts—the video stream of interest and the background video stream—and applying different compression levels to each part using the H.264 / H.265 codec framework, the video stream file size is significantly reduced while ensuring the effective capture of critical traffic monitoring information. Experimental results show that the traffic monitoring video compression device described in this embodiment outperforms existing uniform compression methods in terms of compression efficiency and video quality.

[0159] The image target detection method, video stream compression method, and apparatus proposed in this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A video stream compression method, characterized in that, Includes the following steps: Step S1: The video stream is split into multiple images; Step S2: Target detection is performed on the region of interest of an image based on the improved YOLOv8 to form an image with the region of interest; Step S3: Generate a preliminary binarized mask image for the image described in step S2, and perform erosion, dilation and Gaussian blur operations on the preliminary binarized mask image in sequence to generate the final binarized mask image. Step S4, the final binarized mask image described in step S3 extracts the image of interest and the background image from an image with a region of interest described in step S2; Step S5: Repeat steps S2 to S4 until the image of interest and the background image are extracted from multiple images, then proceed to step S6. Step S6: Multiple images of interest are combined to form a video stream of interest, and multiple background images are combined to form a background video stream; Step S7: The video stream of interest and the background video stream described in step S6 are merged and compressed using different compression levels, and a compressed video stream is obtained based on the merged and compressed video stream of interest and background video stream. Step S2 involves performing target detection on the region of interest (ROI) of an image based on an improved YOLOv8 algorithm. The image is sequentially input into the YOLOv8 backbone network, the YOLOv8 neck network, and the YOLOv8 head network for target detection. Specifically: The image is sequentially input into the YOLOv8 backbone network and the YOLOv8 neck network's Star Anchor Context Feature Extraction (CSC) module for feature extraction, outputting a feature map. In the CSC module, the image is processed by the convolutional batch normalization SiLU activation function module CBS module to extract features and output feature maps. The feature maps are split into two branches. One branch feature map is directly input into Concat, and the other branch feature map is processed by multiple SC-Block modules to fuse features and output fused feature maps. After the feature maps of the two branches are concatted, they are processed again by the CBS module to output feature maps. In the SC-Block module, the feature map of another branch is fused through CAA and star operations to output the fused feature map; The other branch feature map is fused using CAA and star operations to output the fused feature map, specifically: The other branch feature map is processed by CAA to extract features and output a feature map. The feature map is then processed by DW-Conv to divide it into a first branch and a second branch. The feature maps of the first branch and the second branch are respectively input into the FC layer for operation to generate feature maps again. The star operation merges the feature maps of the first branch and the second branch to generate an updated feature map. The updated feature map is then input into the FC layer and DW-Conv in sequence to generate a fused feature map. The other branch feature map is fused using CAA and star operations, specifically as follows: ； in, The feature map of another branch is fused using CAA and star operations. For adaptive weighting coefficients, To extract a specific location from an image Local features, For a certain position The corresponding global context information; The updated feature map is specifically as follows: ； in, For the updated feature map, For feature maps, For adaptive weights, For logical operators.

2. The video stream compression method according to claim 1, characterized in that, In step S3, the erosion operation on the preliminary binarized mask image specifically involves: ； in, To perform erosion on the initial binarized mask image, For the first One image, The structuring element used when performing erosion on the initial binarized mask image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. These are the pixel coordinates of the image. This is a preliminary binarized mask image; The dilation operation on the initial binarized mask image is specifically as follows: ； in, To perform a dilation operation on the initial binarized mask image. The structuring element used when performing dilation operations on the initial binarized mask image.

3. The video stream compression method according to claim 1, characterized in that, In step S3, the final binarized mask image is specifically as follows: ； in, For the final binarized mask image, The pixel coordinates of the image. For structural elements relative to pixels Horizontal offset For structural elements relative to pixels The vertical offset. For Gaussian blur, Let the standard deviation of the kernel function of the Gaussian blur be the standard deviation. This is to perform a dilation operation on the initial binarized mask image.

4. The video stream compression method according to claim 1, characterized in that, In step S6, the video stream of interest specifically refers to: ； in, For video streams of interest, For the final binarized mask image, For the first One image, These are the pixel coordinates of the image. A sequence of images containing a region of interest; The background video stream is specifically as follows: ； in, For background video stream, This is the inverse mask of the final binarized mask image.

5. A traffic monitoring video compression device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the video stream compression method as described in any one of claims 1 to 4.

Images