Video region of interest detection method based on YOLO model

Abstract

This paper presents a video region of interest (ROI) detection method based on the YOLO model, belonging to the field of video object detection. Addressing the issue that the detection bounding boxes output by the YOLO model cannot completely cover the ROI, a YOLO-based method for video ROI detection is designed. First, the original video is converted from formats such as RMVB, AVI, or MP4 to YUV format, and then video YUV decomposition is performed. Second, a pre-trained YOLO model is used to obtain an initial bounding box set, followed by bounding box fusion and expansion operations to obtain a final expanded bounding box. Finally, the ROI is extracted using a mask matrix, and the bounding boxes containing the ROI are then... Y Components and original U , V The components are combined to form the detection result video. Experimental results and method analysis show that the new method has good detection performance and can completely cover the region of interest, laying the foundation for subsequent operations such as video encryption.

Description

Technical Field

[0001] This invention relates to the field of target detection, and more specifically to a method for detecting regions of interest in videos. Background Technology

[0002] With the widespread application of video data, security threats during its generation, storage, processing, and transmission have become increasingly prominent. Due to the significant characteristics of video data, such as large data volume, strong inter-pixel correlation, and high redundancy, traditional text encryption algorithms exhibit obvious lag when processing high-resolution images. On the one hand, the enormous computational overhead of full video encryption cannot meet the demands of real-time transmission in the 5G era; on the other hand, videos transmitted over public networks or third-party cloud platforms are highly vulnerable to attacks such as illegal interception, malicious tampering, and privacy theft. In recent years, security incidents caused by video privacy breaches have occurred frequently. How to ensure the security of sensitive video information in a high-time-sensitivity environment has become a critical issue that urgently needs to be addressed in the field of cyberspace security.

[0003] Traditional video encryption algorithms often employ a "full-disk encryption" strategy, treating the entire video as an equally important binary stream. However, in real-world applications, the information value carried by different regions of a video frame varies significantly. For example, in a medical ultrasound video frame, the lesion area is the core basis for the doctor's diagnosis and is extremely sensitive privacy information, while background noise and edge areas contain less useful information. In a surveillance video, facial features and license plates are crucial for privacy protection, while background buildings are publicly visible. This uneven distribution of information value has prompted researchers to consider whether it is necessary to perform indiscriminate deep encryption on the entire image. "Full-disk encryption" not only wastes a lot of computational resources but also causes the video to completely lose its visual characteristics, making it impossible for users to perform preliminary retrieval, classification, or preview. Based on this, region-of-interest (ROI) video encryption technology has emerged. It advocates "on-demand encryption" and "selective encryption," identifying key regions through image segmentation or object detection techniques and performing strong encryption, while keeping non-ROI regions public or subjecting them to lightweight processing. This approach, which balances security and flexibility, is becoming a hot research direction in multimedia information security.

[0004] Object detection, a core task in computer vision, aims to locate objects in videos or images and identify their categories. In content-based secure encryption systems, the accuracy of object detection directly determines the quality of region-of-interest extraction. However, due to the complex morphological evolution of objects, their varied poses, and uncontrollable factors such as lighting fluctuations and occlusion during imaging, object detection remains a challenging area in computer vision. Summary of the Invention

[0005] The purpose of this invention is to address the problem that the detection bounding boxes output by the YOLO model cannot completely cover the region of interest. This invention proposes a video region of interest detection method based on the YOLO model, laying the foundation for subsequent operations such as video encryption.

[0006] The technical solution of this invention: To achieve the above-mentioned objective, a video region of interest detection method based on the YOLO model is designed, the steps of which are detailed below:

[0007] Step 1: Decoding the original video: Decode the original video P 1 Depend on k Frame size is m × n The image composition will P 1 Convert from formats such as RMVB, AVI, or MP4 to the original video data format YUV. The converted video is... P 2 ; Step 2: Video YUV Decomposition: From P 2 Decomposed from Y , U and V The components, their sizes are respectively m × n × k、 ( m / 2)×( n / 2)× k and( m / 2)×( n / 2)× k ; Step 3: Initial Border Generation: Generate the initial border of a video frame to be processed. Y Components are input into a pre-trained YOLO model to obtain an initial set of bounding boxes. B ={ b i}, i= 1 , 2 , …, c Test results b i It is a rectangular frame. b i =(x i , y i , w i , h i ), (1) in,( x i ,y i () represents the coordinates of the top-left corner of the rectangle. w i and h i These are the width and height of the rectangle, respectively; and also, provide... b i Category prediction and confidence score; Step 4: Confidence Ranking: Let P To predict the set of class probabilities, formula (2) is used to... B Sort by confidence level in descending order. [ A , D ]=sort( P ), (2) Where sort(·) is the descending sorting function. A This is the sorted confidence sequence. D For the new index sequence; Step 5: Calculate the border area: According to formula (3), calculate b i area S i , S i = w i × h i , i= 1 , 2 , …, c , (3) in, S i The area of ​​the two bounding boxes; Step 6: Border Blending: Let a 1∈ A The corresponding border is the main border. Using formula (4), for A The borders in the middle are blended to ensure the final borders are blended. F Able to fully cover potential areas of interest, r 1i =( S 1∩ S i ) / ( S 1∪ S i ), i= 2 ,3, …, c , (4) in, r 1i For two bounding boxes S1 and S i The intersection and union ratio; let t For the intersection-union ratio threshold, if r 1i ＞t Then update the current bounding box to S 1 and S i Find the smallest bounding rectangle; otherwise, delete the border. S i ; Step 7: Border expansion: Calculate the scale factor using formula (5). f Using formula (6), the border is oriented outwards from the center point. F Expanding, we get a size of m 1× n The final extended border of 1 K , f =1- p , (5) K =(1+ f )× F , (6) in, p= max( P ), max(·) is the maximum value function; Step 8: Mask Matrix Generation: Generate a mask matrix. m × n binary mask matrix M The covered pixel positions are marked as 1, and the remaining positions are marked as 0; Step 9: Region of Interest Extraction: Using formula (7), extract the region of interest. Y Perform masking operation to extract Y Region of interest of the component E , E = Y .* M , (7) Where .* indicates element-wise multiplication of two matrices; Step 10: Composite Video: Combine the video with the region of interest border. Y Components and original U , V Components combined into video P 3 .

[0008] Beneficial Effects: This invention addresses the problem that the detection bounding boxes output by the YOLO model cannot completely cover the region of interest (ROI), proposing a video ROI detection method based on the YOLO model. The main contribution is: utilizing the YOLO detection model, a video ROI detection method is proposed. Experimental results and method analysis show that the new method effectively eliminates the risk of edge privacy pixel leakage, ensuring the integrity of the extracted video ROI. Attached Figure Description

[0009] Figure 1 Flowchart of video region of interest detection based on the YOLO model; Figure 2 : Original video frames; Figure 3 : Video frames with detection frames. Implementation

[0010] The implementation process of the present invention will be further described in detail below with reference to specific accompanying drawings and examples.

[0011] Figure 1 This is a flowchart of the video region of interest detection process based on the YOLO model.

[0012] The programming software used was Matlab R2025b. A video frame of 160×120 pixels from the original video file traffic.avi was selected as the experimental object. Figure 2 As shown.

[0013] The video region of interest detection method based on the YOLO model is described in detail below:

[0014] Step 1: Decoding the original video: Decode the original video P 1 Composed of 119 frames of images, each 160×120 pixels, P 1 Convert from formats such as RMVB, AVI, or MP4 to the original video data format YUV. The converted video is... P 2 ; Step 2: Video YUV Decomposition: From P 2 Decomposed from Y , U and V The components are 160×120×119. 、 80×60×119 and 80×60×119; Step 3: Initial Border Generation: Generate the initial border of a video frame to be processed. YComponents are input into a pre-trained YOLO model to obtain an initial set of bounding boxes. B ={ b i}, i= 1 , 2 , …, 10. The specific test results are as follows: b 1 = (441, 161, 110, 164), b 2 = (441, 161, 109, 164), b 3 = (441, 161, 109, 163), b 4 = (441, 160, 109, 164) b 5 = (441, 160, 109, 164) b 6 = (442, 160, 109, 164) b 7 = (441, 161, 109, 164) b 8 = (442, 161, 109, 164) b 9 = (442, 161, 109, 163) b 10 =(442,160,108,164), b i The predicted categories are all truck. b i The confidence scores are as follows: p I = 0.64130676 p 2 = 0.62201208, p 3 = 0.75625849 p 4 = 0.69260561, p 5 = 0.71598440 p 6 = 0.73906982 p 7 = 0.72019571 p 8 = 0.66216379 p 9 = 0.66119695 p 10 =0.50976294; Step 4: Confidence Ranking: Predicted Class Probabilities Set P ={ p i}, i= 1 , 2 , …, 10. Using formula (2), for B Sort by confidence level in descending order. A This is the sorted confidence sequence. DFor the new index sequence; Step 5: Calculate the border area: According to formula (3), calculate b 1, b 2, …, b 10 The areas are respectively: S 1 = 18040 S 2 = 17876, S 3 = 17767 S 4 = 17876 S 5 = 17876 S 6 = 17876 S 7 = 17876 S 8 = 17876 S 9 = 17767 S 10 =17712; Step 6: Border Blending: Let a 1∈ A The corresponding border is the main border. Using formula (4), for A The borders in the middle are blended to ensure the final borders are blended. F Able to fully cover potential areas of interest, r 12 =0.9759, r 13 =0.9790, r 14 =0.9849, r 15 =0.9849, r 16 =0.9849, r 17 =0.9789, r 18 =0.9939, r 19 =0.9849, r 110 =0.9759, let the crossover ratio threshold be... t =0.5, therefore we know r 12 , r 13 , …, r 110 All greater than t Then update the current bounding box to S 1 and S i The minimum bounding rectangle, i = 1, 2 , …, 10; Finally, the final border can be obtained. F =(110,40,28,42); Step 7: Border Expansion: Based on p= max( P From this, we can know that p = p 3 = 0.75625849; Calculate the scaling factor using formula (5). f =0.24374151; Using formula (6), the center point of the border is calculated to be (61, 124). The border is then extended outwards from the center point. F Expand to obtain the final expanded border. K= (108,37,32,47); Step 8: Mask Matrix Generation: Generate a 160×120 binary mask matrix. M The covered pixel positions are marked as 1, and the remaining positions are marked as 0; Step 9: Region of Interest Extraction: Using formula (7), extract the region of interest. Y Perform masking operation to extract Y Region of interest of the component E ; Step 10: Composite Video: Combine the video with the region of interest border. Y Components and original U , V Components combined into video P 3 .

Claims

1. A video region of interest detection method based on the YOLO model, characterized in that, The testing process includes the following steps: Step 1: Decoding the original video: Decode the original video P 1 Depend on k Frame size is m × n The image composition will P 1 Convert from formats such as RMVB, AVI, or MP4 to the original video data format YUV. The converted video is... P 2 ; Step 2: Video YUV Decomposition: From P 2 Decomposed from Y , U and V The components, their sizes are respectively m × n × k、 ( m / 2)×( n / 2)× k and( m / 2)×( n / 2)× k ; Step 3: Initial Border Generation: Generate the initial border of a video frame to be processed. Y Components are input into a pre-trained YOLO model to obtain an initial set of bounding boxes. B ={ b i }, i= 1 , 2 , …, c Test results b i It is a rectangular frame. b i =(x i , y i , w i , h i )， （1） in,( x i , y i () represents the coordinates of the top-left corner of the rectangle. w i and h i These represent the width and height of the rectangle, respectively; Additionally, provide... b i Category prediction and confidence score; Step 4: Confidence Ranking: Let P To predict the set of class probabilities, formula (2) is used to... B Sort by confidence level in descending order. [ A , D ]=sort( P ), (2) Where sort(·) is the descending sorting function. A This is the sorted confidence sequence. D For the new index sequence; Step 5: Calculate the border area: According to formula (3), calculate b i area S i , S i = w i × h i ， i= 1 , 2 , …, c ， （3） in, S i The area of ​​the two bounding boxes; Step 6: Border Blending: Let a 1∈ A The corresponding border is the main border. Using formula (4), for A The borders in the middle are blended to ensure the final borders are blended. F Able to fully cover potential areas of interest, r 1i =( S 1∩ S i ) / ( S 1∪ S i )， i= 2 ,3, …, c ， （4） in, r 1i For two bounding boxes S 1 and S i The intersection and union ratio; let t For the intersection-union ratio threshold, if r 1i ＞t Then update the current bounding box to S 1 and S i Find the smallest bounding rectangle; otherwise, delete the border. S i ; Step 7: Border expansion: Calculate the scale factor using formula (5). f Using formula (6), the border is oriented outwards from the center point. F Expanding, we get a size of m 1× n The final extended border of 1 K , f =1- p ， （5） K =(1+ f )× F ， （6） in, p= max( P ), max(·) is the maximum value function; Step 8: Mask Matrix Generation: Generate a mask matrix. m × n binary mask matrix M The covered pixel positions are marked as 1, and the remaining positions are marked as 0; Step 9: Region of Interest Extraction: Using formula (7), extract the region of interest. Y Perform masking operation to extract Y Region of interest of the component E , E = Y .* M ， （7） Where .* indicates element-wise multiplication of two matrices; Step 10: Composite Video: Combine the video with the region of interest border. Y Components and original U , V Components combined into video P 3 .

Images