Residual frame video processing convolution method based on dynamic gating

Abstract

The invention relates to the technical field of computer vision and video processing, and provides a residual frame video processing convolution method based on dynamic gating, which comprises the following steps of: carrying out gray conversion and Gaussian filtering processing on a current frame image and a previous frame image; calculating an inter-frame difference image and carrying out binarization processing to obtain a motion area mask; performing morphological expansion operation on the mask image; extracting the contour of the motion area and calculating a minimum bounding rectangle to obtain a bounding box; carrying out merging processing on the overlapped bounding boxes; updating the synthesized background frame based on the combined bounding box, and copying the pixels of the motion area of the current frame to the corresponding position of the background frame; and outputting the optimized bounding box set and the updated synthesized background frame. According to the method, the calculation efficiency and the resource utilization rate of video processing are improved, and meanwhile, the timing sequence continuity and the space consistency of a processing result are ensured through a strategy of keeping the static region unchanged and only updating the motion region.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and video processing technology, and in particular to a convolution method for residual frame video processing based on dynamic gating. Background Technology

[0002] Video processing technology is an important branch of computer vision, widely used in key application scenarios such as video surveillance, intelligent analysis, real-time transmission, and multimedia processing. Video data is essentially a sequence of images composed of consecutive temporal frames. Its unique temporal dimension and inter-frame correlation provide unique optimization space for efficient processing. However, traditional video processing methods typically treat video as a simple sequence of static images, performing complete convolutional neural network calculations frame by frame, failing to fully utilize the inherent temporal redundancy and inter-frame correlation information of video data.

[0003] In existing technologies, video processing mainly employs deep convolutional neural networks to perform global feature extraction and semantic understanding for each frame, achieving significant success in single-frame image processing tasks. However, existing methods do not adequately consider the inherent physical relationship between the temporal characteristics of video data and the convolutional computation pattern. They struggle to organically integrate objective temporal sparsity constraints with actual spatial locality features, failing to fundamentally reconstruct the convolutional computation method to directly utilize the sparsity features of video residuals. Instead, they rely on external auxiliary modules or heuristic strategies, leading to additional complexity, error propagation risks, and deployment difficulties. Summary of the Invention

[0004] In view of this, this invention proposes a convolutional method for residual frame video processing based on dynamic gating. This method addresses the shortcomings of existing methods in considering the inherent physical relationship between the temporal characteristics of video data and the convolutional calculation mode. These methods fail to organically integrate objective temporal sparsity constraints with actual spatial locality features and do not fundamentally reconstruct the convolutional calculation method to directly utilize the sparsity characteristics of video residuals. Instead, they rely on external auxiliary modules or heuristic strategies, which leads to additional complexity, error propagation risks, and deployment difficulties.

[0005] The technical solution of this invention is implemented as follows: This invention provides a convolution method for residual frame video processing based on dynamic gating, comprising the following steps: The current frame image and the previous frame image are obtained, and grayscale conversion processing is performed on the current frame image and the previous frame image respectively to obtain the current frame grayscale image and the previous frame grayscale image; Gaussian blur filtering is applied to the current frame grayscale image and the previous frame grayscale image respectively to obtain the current frame filtered image and the previous frame filtered image; Calculate the absolute difference between the current frame filtered image and the previous frame filtered image to obtain the inter-frame difference image; The inter-frame difference image is subjected to threshold binarization to obtain a binary mask image of the motion region; A morphological dilation operation is performed on the binary mask image of the motion region to obtain a dilated mask image; Contour detection is performed on the inflated mask image to extract the contours of the moving regions and calculate the minimum bounding rectangle of each contour to obtain an initial bounding box set. The overlapping bounding boxes in the initial bounding box set are merged to obtain a merged bounding box set; The composite background frame is updated based on the merged bounding box set. The pixel regions in the current frame image located within the merged bounding box set are copied to the corresponding positions in the composite background frame, while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, thus obtaining the updated composite background frame. Output the merged bounding box set and the updated composite background frame. Based on the above technical solutions, preferably, the step of calculating the absolute difference between the current frame filtered image and the previous frame filtered image to obtain the inter-frame difference image includes: The absolute difference between the pixel values ​​at corresponding positions in the current frame filtered image and the previous frame filtered image is calculated pixel by pixel to obtain the original difference matrix; For each pixel position in the original difference matrix, a threshold judgment is performed. When the pixel difference is greater than a preset motion threshold, the corresponding pixel position is marked as a moving pixel. When the pixel difference is less than or equal to the preset motion threshold, the corresponding pixel position is marked as a static pixel. The labeling results of the moving pixels and the static pixels are combined to obtain an inter-frame difference image.

[0006] Based on the above technical solutions, preferably, the pixel-by-pixel calculation includes: Obtain the grayscale pixel values ​​at the same coordinate positions in the current frame filtered image and the previous frame filtered image, and record them as the current pixel value and the previous pixel value, respectively; Calculate the absolute value of the difference between the current pixel value and the previous pixel value to obtain the pixel difference at the current pixel position. Set the preset motion threshold to 2. When the pixel difference is greater than 2, set the pixel value at that position to 255 and mark it as a moving pixel. When the pixel difference is less than or equal to 2, the pixel value at that location is set to 0 and marked as a static pixel; Iterate through all pixel positions in the image until the inter-frame difference image is generated.

[0007] Based on the above technical solutions, preferably, the step of merging overlapping bounding boxes in the initial bounding box set to obtain a merged bounding box set includes: The bounding boxes in the initial bounding box set are sorted in ascending order according to the horizontal coordinate value of the top left corner to obtain a sorted bounding box sequence. Traverse the sorted bounding box sequence and calculate the ratio of the overlapping area of ​​the two bounding boxes to the smaller area of ​​the two bounding boxes to obtain the overlap ratio; When the overlap ratio is greater than the preset overlap threshold, the two bounding boxes are merged to obtain a merged bounding box. When the overlap ratio is less than or equal to the preset overlap threshold, the two bounding boxes are kept independently until the merging judgment of all adjacent bounding boxes is completed, and then a merged bounding box set is obtained.

[0008] Based on the above technical solutions, preferably, the fusion process includes: Extract the top-left and bottom-right coordinates of the first bounding box, and extract the top-left and bottom-right coordinates of the second bounding box. When the overlap ratio is greater than a preset overlap threshold, determine the top-left coordinate of the fused bounding box by calculating the minimum x-coordinate and minimum y-coordinate values ​​of all coordinate points of the two bounding boxes, and determine the bottom-right coordinate of the fused bounding box by calculating the maximum x-coordinate and maximum y-coordinate values ​​of all coordinate points of the two bounding boxes. Generate the fused bounding box, and remove the original two bounding boxes from the sorted bounding box sequence.

[0009] Based on the above technical solutions, preferably, the step of updating the composite background frame based on the merged bounding box set, which involves copying the pixel regions in the current frame image located within the merged bounding box set to the corresponding positions in the composite background frame while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, to obtain the updated composite background frame, includes: Traverse each bounding box in the merged bounding box set, extract pixel region data within the bounding box range from the current frame image, map the extracted pixel region data to the corresponding region position of the synthesized background frame according to the coordinate position, perform pixel value replacement operation, and keep the pixel values ​​of static regions outside the range of the merged bounding box set in the synthesized background frame unchanged, to obtain the updated synthesized background frame, which includes motion region update information and static region history information.

[0010] Based on the above technical solutions, preferably, the pixel value replacement operation includes: Obtain the coordinates of the top-left and bottom-right corners of the bounding box, determine the range of pixel coordinates covered by the bounding box, traverse each pixel position within the range of pixel coordinates, read the RGB pixel value of each pixel position from the current frame image, and directly assign the read RGB pixel value to the same pixel coordinate position in the synthesized background frame, until the update operation of all pixel positions within the bounding box area is completed, and obtain the hybrid background frame structure. The hybrid background frame structure includes using the current frame pixels in the dynamic motion area and retaining historical pixels in the static background area.

[0011] Based on the above technical solutions, preferably, the step of performing threshold binarization processing on the inter-frame difference image to obtain a motion region binary mask image includes: A binarization threshold is determined for the pixel value of each pixel in the inter-frame difference image. When the pixel value is greater than the preset binarization threshold, the corresponding pixel is set as the foreground pixel value. When the pixel value is less than or equal to the preset binarization threshold, the corresponding pixel is set as the background pixel value, thus obtaining the initial binarized image. The initial binary image is subjected to pixel value normalization processing, with the foreground pixel values ​​being uniformly normalized to 255 and the background pixel values ​​being uniformly normalized to 0, to obtain a binary mask image of the motion region.

[0012] Based on the above technical solutions, preferably, the step of performing contour detection on the expanded mask image, extracting the contours of the moving regions, and calculating the minimum bounding rectangle of each contour to obtain an initial bounding box set includes: Contour detection is performed on the inflated mask image to extract all external contours and calculate the bounding area of ​​each external contour. External contours with bounding areas greater than a preset minimum area threshold are marked as valid contours to obtain a set of valid contours. Traverse each valid contour in the set of valid contours, calculate the coordinates of the four vertices of the minimum bounding rectangle of the valid contour, mark the coordinates of the top left corner and the bottom right corner respectively, construct a bounding box format rectangle based on the coordinates of the top left corner and the bottom right corner, and obtain the initial bounding box set.

[0013] Based on the above technical solutions, preferably, the step of outputting the merged bounding box set and the updated composite background frame includes: The merged bounding box set is subjected to structured data encapsulation processing to generate bounding box output data, resulting in structured bounding box data. The bounding box output data includes bounding box coordinates, identification information, and timing information. The bounding box coordinates include the coordinates of the upper left corner and the lower right corner. The identification information includes a unique identifier for the bounding box. The timing information includes the current frame number. The updated synthetic background frame is formatted by encoding the pixel data according to the number of image channels and pixel bit depth to generate synthetic background frame output data, thus obtaining formatted synthetic background frame data.

[0014] The residual frame video processing convolution method based on dynamic gating of the present invention has the following advantages over the prior art: (1) By integrating inter-frame difference analysis and dynamic gating mechanism, Gaussian filtering and threshold binarization algorithm are used to accurately identify motion regions. Contour detection and bounding box merging technology are combined to dynamically determine the update region range. Background frames are selectively updated according to the distribution of motion regions, which improves the computational efficiency and resource utilization of video processing. At the same time, the strategy of keeping the static region unchanged and only updating the motion region ensures the temporal continuity and spatial consistency of the processing results. (2) By integrating the pixel-by-pixel difference calculation and adaptive threshold judgment mechanism, the absolute difference algorithm is used to perform inter-frame pixel change quantization analysis and motion state recognition. Combined with the preset motion threshold, the moving pixels and static pixels are dynamically distinguished. The marking results are binarized according to the pixel difference, which improves the accuracy and robustness of motion region recognition. (3) By integrating spatial sorting algorithm and overlap measurement analysis mechanism, the spatial relationship evaluation and fusion judgment of bounding box are performed by calculating the overlap area ratio. Combined with preset overlap threshold, the bounding box merging strategy is dynamically controlled. The overlapping bounding boxes are enclosedly fused according to the coordinate extreme value calculation, which improves the calculation efficiency of motion area boundary positioning. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a residual frame video processing convolution method based on dynamic gating according to the present invention; Figure 2 This is a schematic diagram illustrating the processing effect of the Skip Convolution algorithm in a sequence of nine consecutive images according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 This invention provides a convolution method for residual frame video processing based on dynamic gating, comprising the following steps: The current frame image and the previous frame image are obtained, and grayscale conversion processing is performed on the current frame image and the previous frame image respectively to obtain the current frame grayscale image and the previous frame grayscale image; Gaussian blur filtering is applied to the current frame grayscale image and the previous frame grayscale image respectively to obtain the current frame filtered image and the previous frame filtered image; Calculate the absolute difference between the current frame filtered image and the previous frame filtered image to obtain the inter-frame difference image; The inter-frame difference image is subjected to threshold binarization to obtain a binary mask image of the motion region; A morphological dilation operation is performed on the binary mask image of the motion region to obtain a dilated mask image; Contour detection is performed on the inflated mask image to extract the contours of the moving regions and calculate the minimum bounding rectangle of each contour to obtain an initial bounding box set. The overlapping bounding boxes in the initial bounding box set are merged to obtain a merged bounding box set; The composite background frame is updated based on the merged bounding box set. The pixel regions in the current frame image located within the merged bounding box set are copied to the corresponding positions in the composite background frame, while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, thus obtaining the updated composite background frame. Output the merged bounding box set and the updated composite background frame. Specifically, this embodiment integrates inter-frame difference analysis with a dynamic gating mechanism, utilizes Gaussian filtering and threshold binarization algorithms for accurate motion region identification, dynamically determines the update region range using contour detection and bounding box merging techniques, and selectively updates the background frame based on the motion region distribution. The residual frame processing mechanism solves the problems of redundancy and resource waste in full-frame computation in traditional video processing, improving computational efficiency and resource utilization. Simultaneously, by maintaining static regions unchanged and updating only motion regions, it ensures the temporal continuity and spatial consistency of the processing results.

[0019] The step of obtaining the current frame image and the previous frame image, and performing grayscale conversion processing on the current frame image and the previous frame image respectively to obtain the current frame grayscale image and the previous frame grayscale image, includes: The color image data of the current frame image and the previous frame image are acquired respectively. The color image data of the current frame image is subjected to color space conversion processing to obtain the grayscale conversion parameters of the current frame.

[0020] In one specific embodiment, the color space conversion process includes: The red, green, and blue channel data of the color image data of the current frame image are extracted respectively. The red, green, and blue channel data are weighted and summed based on preset weight coefficients to obtain the grayscale conversion parameters of the current frame. The preset weight coefficients are 0.299 for red, 0.587 for green, and 0.114 for blue.

[0021] Based on the grayscale conversion parameters of the current frame, grayscale mapping processing is performed on the color image data of the current frame image to obtain the grayscale image of the current frame. Color space conversion processing is performed on the color image data of the previous frame image to obtain the grayscale conversion parameters of the previous frame. Based on the grayscale conversion parameters of the previous frame, grayscale mapping processing is performed on the color image data of the previous frame image to obtain the grayscale image of the previous frame.

[0022] In one specific embodiment, the grayscale mapping process includes: The red, green, and blue component values ​​of each pixel in the color image data of the current frame are multiplied by the corresponding weight coefficients in the grayscale conversion parameters of the current frame, and the multiplication results are accumulated to obtain the grayscale value of the pixel. The grayscale values ​​of all pixels in the current frame are combined to form the grayscale image of the current frame. The same processing method is used to process the color image data of the previous frame to obtain the grayscale image of the previous frame.

[0023] Specifically, this embodiment integrates a color space conversion algorithm with a weighted summation mechanism. It uses preset weight coefficients to perform linear combination and grayscale value calculation of the RGB three-channel data. A standardized weight allocation strategy dynamically balances the contribution of the red, green, and blue channels to grayscale conversion. Furthermore, it performs unified grayscale mapping processing on the color image data based on the characteristics of human visual perception. This intelligent color space conversion and data dimensionality reduction mechanism solves the problems of redundancy and excessive computational complexity in traditional video processing, improving image data processing efficiency and algorithm speed.

[0024] The step of performing Gaussian blur filtering on the current frame grayscale image and the previous frame grayscale image respectively to obtain the current frame filtered image and the previous frame filtered image includes: A Gaussian filter kernel matrix is ​​generated according to a preset filter kernel size. A convolution operation is performed on each pixel in the current frame grayscale image to obtain the current frame convolution result data. The current frame convolution result data is then normalized to obtain the current frame filtered image.

[0025] In one specific embodiment, generating the Gaussian filter kernel matrix according to the preset filter kernel size includes: The preset filter kernel size is set to 5×5. The weight value of each position in the 5×5 matrix is ​​calculated according to the two-dimensional Gaussian function calculation formula. The standard deviation parameter of the two-dimensional Gaussian function is set to 1.0. The calculated weight values ​​are filled into the corresponding positions to obtain the Gaussian filter kernel matrix. The Gaussian filter kernel matrix is ​​normalized so that the sum of all elements in the matrix is ​​equal to 1.

[0026] A Gaussian filter kernel matrix is ​​generated according to the preset filter kernel size. A convolution operation is performed on each pixel in the previous grayscale image to obtain the convolution result data of the previous frame. The convolution result data of the previous frame is then normalized to obtain the previous frame filtered image.

[0027] In one specific embodiment, the convolution operation includes: Taking each pixel in the current frame grayscale image as the first center pixel, a 5×5 pixel region corresponding to the size of the Gaussian filter kernel matrix is ​​extracted. Each pixel value in the 5×5 pixel region is multiplied by the weight value at the corresponding position in the Gaussian filter kernel matrix. The multiplication results are accumulated and summed to obtain the convolution output value of the first center pixel. All pixels in the current frame grayscale image are traversed to obtain the convolution result data of the current frame.

[0028] Specifically, this embodiment integrates Gaussian filtering and convolution operations, using a two-dimensional Gaussian function calculation formula to design a 5×5 filter kernel weight distribution and noise suppression process. It dynamically controls the filtering intensity and smoothing effect using the standard deviation parameter, and performs numerical standardization and weight balancing on the convolution results according to a normalization strategy. This intelligent image smoothing and noise filtering mechanism solves the problems of severe noise interference and unstable edge details in the original grayscale image during traditional inter-frame difference calculation, improving the smoothness of image quality. Simultaneously, a unified filtering strategy ensures the consistency and comparability of image data between the current and previous frames.

[0029] The step of calculating the absolute difference between the current frame filtered image and the previous frame filtered image to obtain an inter-frame difference image includes: The absolute difference between the pixel values ​​at corresponding positions in the current frame filtered image and the previous frame filtered image is calculated pixel by pixel to obtain the original difference matrix; For each pixel position in the original difference matrix, a threshold judgment is performed. When the pixel difference is greater than a preset motion threshold, the corresponding pixel position is marked as a moving pixel. When the pixel difference is less than or equal to the preset motion threshold, the corresponding pixel position is marked as a static pixel. The labeling results of the moving pixels and the static pixels are combined to obtain an inter-frame difference image.

[0030] The pixel-by-pixel calculation includes: Obtain the grayscale pixel values ​​at the same coordinate positions in the current frame filtered image and the previous frame filtered image, and record them as the current pixel value and the previous pixel value, respectively; Calculate the absolute value of the difference between the current pixel value and the previous pixel value to obtain the pixel difference at the current pixel position. Set the preset motion threshold to 2. When the pixel difference is greater than 2, set the pixel value at that position to 255 and mark it as a moving pixel. When the pixel difference is less than or equal to 2, the pixel value at that location is set to 0 and marked as a static pixel; Iterate through all pixel positions in the image until the inter-frame difference image is generated.

[0031] Specifically, this embodiment integrates pixel-by-pixel difference calculation with an adaptive threshold judgment mechanism. It utilizes an absolute difference algorithm for quantification analysis of inter-frame pixel changes and motion state recognition, dynamically distinguishes between moving and static pixels using a preset motion threshold, and binarizes the marking results based on the pixel difference magnitude. This precise pixel-level motion detection mechanism solves the problems of noise interference and inaccurate motion detection in traditional inter-frame difference analysis, improving the accuracy and robustness of motion region recognition.

[0032] The step of thresholding and binarizing the inter-frame difference image to obtain a binary mask image of the motion region includes: A binarization threshold is determined for the pixel value of each pixel in the inter-frame difference image. When the pixel value is greater than the preset binarization threshold, the corresponding pixel is set as the foreground pixel value. When the pixel value is less than or equal to the preset binarization threshold, the corresponding pixel is set as the background pixel value, thus obtaining the initial binarized image.

[0033] In one specific embodiment, the binarization threshold determination includes: The preset binarization threshold is set to 2. The coordinates of each pixel in the inter-frame difference image are traversed, and the pixel gray value at the coordinates of the pixel is extracted. When the pixel gray value is greater than 2, the corresponding pixel is determined to be a moving pixel and marked as a foreground pixel value 1. When the pixel gray value is less than or equal to 2, the corresponding pixel is determined to be a static pixel and marked as a background pixel value 0. This process continues until the binarization classification of all pixels is completed.

[0034] The initial binary image is subjected to pixel value normalization processing, with the foreground pixel values ​​being uniformly normalized to 255 and the background pixel values ​​being uniformly normalized to 0, to obtain a binary mask image of the motion region.

[0035] In one specific embodiment, the pixel value normalization process includes: Traverse all pixels in the initial binarized image, detect the current pixel value of each pixel. When a pixel value of 1 is detected as a foreground pixel value, reassign the corresponding pixel value to 255 to represent the white foreground of the moving area. When a pixel value of 0 is detected as a background pixel value, keep the corresponding pixel value of 0 to represent the black background of the static area. Combine all the processed pixel values ​​to obtain the binary mask image of the moving area.

[0036] Specifically, this embodiment integrates binarization threshold judgment and pixel value standardization processing mechanisms. It utilizes a preset binarization threshold algorithm to accurately classify and identify moving and static pixels, and combines this with a pixel value standardization strategy to dynamically unify the numerical representation of foreground and background pixels. Furthermore, it standardizes pixel values ​​for white foreground and black background based on motion region characteristics. This intelligent binarization classification and standardized output mechanism solves the problems of inconsistent pixel value representation and blurred motion region boundaries in traditional inter-frame difference analysis, improving the clarity and visual contrast of motion region recognition.

[0037] The step of performing a morphological dilation operation on the binary mask image of the motion region to obtain a dilated mask image includes: A dilation kernel matrix of a preset size is generated. The dilation kernel matrix is ​​then used to perform a dilation kernel convolution operation on each foreground pixel in the binary mask image of the motion region to obtain the dilation result image.

[0038] In one specific embodiment, generating an expansion kernel matrix of a preset size includes: A 3×3 dilation kernel matrix is ​​constructed, and all 9 elements in the 3×3 dilation kernel matrix are set to 1 to obtain an all-1 matrix structure. The foreground pixel with a pixel value of 255 in the binary mask image of the motion region is taken as the second center pixel. A 3×3 neighborhood pixel region is extracted. The 3×3 neighborhood pixel region is multiplied element-wise with the dilation kernel matrix. The product results are summed to obtain the dilation output value of the second center pixel.

[0039] The resulting image of the dilation operation is post-processed with pixel values. The positions where the pixel value is greater than 0 after the dilation operation are set to 255, and the positions where the pixel value is equal to 0 after the dilation operation are kept as 0, thus obtaining the dilated mask image.

[0040] In one specific embodiment, the pixel value post-processing includes: The process iterates through the coordinates of each pixel in the resulting image of the dilation operation, and obtains the output value of the dilation operation for each pixel. When the output value of the dilation operation is greater than 0, it is determined that the corresponding pixel position belongs to the expanded motion region, and the pixel value of the pixel position is set to 255 to represent the foreground region. When the output value of the dilation operation is equal to 0, it is determined that the corresponding pixel position belongs to the static background region, and the pixel value of the pixel position is kept at 0 to represent the background region. This process continues until all pixel positions have been post-processed, resulting in the dilated mask image.

[0041] Specifically, this embodiment integrates morphological dilation algorithms with convolutional operations. It utilizes a 3×3 all-1 dilation kernel matrix to expand the neighborhood of the moving region and enhance spatial connectivity. The dilation output value is dynamically calculated using element-wise multiplication, and the dilation result is binarized based on a pixel value threshold. This intelligent region dilation and connectivity optimization mechanism solves the problems of region fragmentation and boundary discontinuities in traditional moving region detection, improving the spatial integrity and detection stability of the moving region.

[0042] The process involves contour detection of the expanded mask image, extraction of motion region contours, and calculation of the minimum bounding rectangle for each contour to obtain an initial bounding box set, including: Contour detection is performed on the inflated mask image to extract all external contours and calculate the bounding area of ​​each external contour. External contours with bounding areas greater than a preset minimum area threshold are marked as valid contours, thus obtaining a set of valid contours.

[0043] In one specific embodiment, the contour detection includes: The expanded mask image is scanned using a contour detection algorithm to extract all closed boundary contours. The area of ​​each extracted contour is calculated, and the preset minimum area threshold is set to 300 pixels. When the contour area is greater than 300 pixels, the contour is determined to be a valid motion region contour and added to the valid contour set. When the contour area is less than or equal to 300 pixels, the valid motion region contour is determined to be a noise contour and filtered out.

[0044] Traverse each valid contour in the set of valid contours, calculate the coordinates of the four vertices of the minimum bounding rectangle of the valid contour, mark the coordinates of the top left corner and the bottom right corner respectively, construct a bounding box format rectangle based on the coordinates of the top left corner and the bottom right corner, and obtain the initial bounding box set.

[0045] In one specific embodiment, calculating the coordinates of the four vertices of the minimum bounding rectangle of the effective contour includes: Analyze the coordinates of all boundary points of each valid contour in the set of valid contours, find the minimum x-coordinate of all boundary points as x1, the minimum y-coordinate of all boundary points as y1, the maximum x-coordinate of all boundary points as x2, and the maximum y-coordinate of all boundary points as y2. Combine the coordinates (x1, y1, x2, y2) to obtain the bounding box of the contour. Collect the bounding boxes of all contours to obtain the initial set of bounding boxes.

[0046] Specifically, this embodiment integrates contour detection algorithms with area threshold filtering mechanisms. It utilizes a closed boundary scanning algorithm for motion region contour extraction and geometric shape analysis, dynamically filtering noisy contours and small-area interference using a preset minimum area threshold. Furthermore, it constructs the minimum bounding rectangle for effective contours based on the extreme values ​​of boundary point coordinates. This intelligent contour filtering and bounding box generation mechanism solves the problems of severe noise interference and inaccurate boundary positioning in traditional motion detection, significantly improving the accuracy and noise resistance of motion region boundary recognition. Simultaneously, a unified rectangle format ensures geometric consistency and processing stability for bounding box merging and background update operations.

[0047] The step of merging overlapping bounding boxes in the initial bounding box set to obtain a merged bounding box set includes: The bounding boxes in the initial bounding box set are sorted in ascending order according to the horizontal coordinate value of the top left corner to obtain a sorted bounding box sequence. Traverse the sorted bounding box sequence and calculate the ratio of the overlapping area of ​​the two bounding boxes to the smaller area of ​​the two bounding boxes to obtain the overlap ratio; When the overlap ratio is greater than the preset overlap threshold, the two bounding boxes are merged to obtain a merged bounding box. When the overlap ratio is less than or equal to the preset overlap threshold, the two bounding boxes are kept independently until the merging judgment of all adjacent bounding boxes is completed, and then a merged bounding box set is obtained.

[0048] The fusion process includes: Extract the top-left and bottom-right coordinates of the first bounding box, and extract the top-left and bottom-right coordinates of the second bounding box. Set the preset overlap threshold to 0.2. When the overlap ratio is greater than the preset overlap threshold, determine the top-left coordinate of the merged bounding box by calculating the minimum x-coordinate and minimum y-coordinate values ​​of all coordinate points of the two bounding boxes, and determine the bottom-right coordinate of the merged bounding box by calculating the maximum x-coordinate and maximum y-coordinate values ​​of all coordinate points of the two bounding boxes. Generate a merged bounding box that can completely surround the two original bounding boxes, and remove the two original bounding boxes from the sorted bounding box sequence.

[0049] Specifically, this embodiment integrates spatial sorting algorithms and overlap measurement analysis mechanisms. It uses the overlap area ratio calculation to evaluate and merge the spatial relationships of bounding boxes, dynamically controls the bounding box merging strategy based on a preset overlap threshold, and performs enclosing fusion processing on overlapping bounding boxes based on coordinate extreme values. This intelligent bounding box optimization mechanism solves the problems of redundant and repetitive bounding boxes and unreasonable spatial division in traditional motion region detection, improving the computational efficiency of motion region boundary localization.

[0050] The step of updating the composite background frame based on the merged bounding box set involves copying the pixel regions located within the merged bounding box set in the current frame image to their corresponding positions in the composite background frame, while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, to obtain the updated composite background frame, including: Traverse each bounding box in the merged bounding box set, extract pixel region data within the bounding box range from the current frame image, map the extracted pixel region data to the corresponding region position of the synthesized background frame according to the coordinate position, perform pixel value replacement operation, and keep the pixel values ​​of static regions outside the range of the merged bounding box set in the synthesized background frame unchanged, to obtain the updated synthesized background frame, which includes motion region update information and static region history information.

[0051] The pixel value replacement operation includes: Obtain the coordinates of the top-left and bottom-right corners of the bounding box, determine the range of pixel coordinates covered by the bounding box, traverse each pixel position within the range of pixel coordinates, read the RGB pixel value of each pixel position from the current frame image, and directly assign the read RGB pixel value to the same pixel coordinate position in the synthesized background frame, until the update operation of all pixel positions within the bounding box area is completed, and obtain the hybrid background frame structure. The hybrid background frame structure includes using the current frame pixels in the dynamic motion area and retaining historical pixels in the static background area.

[0052] Specifically, this embodiment integrates selective region update and coordinate mapping mechanisms, utilizes bounding box traversal algorithms for motion region pixel extraction and spatial localization analysis, and dynamically updates the motion region in the background frame using a direct RGB pixel value replacement strategy. Furthermore, it differentiates between static and dynamic regions based on the bounding box range. This hybrid background frame construction mechanism solves the problems of redundant full-frame update calculations and background information loss in traditional video background modeling, improving the computational efficiency of video processing and the accuracy of background modeling. Simultaneously, by maintaining the historical pixels of static regions unchanged, it ensures the temporal stability and spatial continuity of the background frame.

[0053] The output of the merged bounding box set and the updated composite background frame includes: The merged bounding box set is subjected to structured data encapsulation processing to generate bounding box output data, resulting in structured bounding box data. The bounding box output data includes bounding box coordinates, identification information, and timing information. The bounding box coordinates include the coordinates of the upper left corner and the lower right corner. The identification information includes a unique identifier for the bounding box. The timing information includes the current frame number.

[0054] In one specific embodiment, the structured data encapsulation process includes: Traverse each bounding box in the merged bounding box set, extract the coordinates of the top left corner and the bottom right corner of the bounding box, organize the coordinate values ​​according to a preset data structure format, assign a unique integer identifier to each bounding box as the bounding box identifier, record the frame number of the current processing frame as a timing mark, package the coordinate values, bounding box identifier and timing mark to form a data tuple of a single bounding box, and collect all the bounding box data tuples to obtain a structured bounding box data output list.

[0055] The updated synthetic background frame is processed by data formatting, and the pixel data is encoded according to the number of image channels and pixel bit depth to generate synthetic background frame output data that conforms to the standard image format specification, thus obtaining formatted synthetic background frame data.

[0056] In one specific embodiment, the data formatting process includes: The image size and channel number information of the updated synthetic background frame are detected, the image pixel data is rearranged according to the RGB three-channel format to ensure that each pixel position contains the values ​​of the three color components: red, green and blue. The pixel values ​​are subjected to bit depth normalization processing to limit all pixel component values ​​to the integer range of 0 to 255. The pixel data is optimized for memory layout according to the standard image data storage format to generate formatted synthetic background frame data that can be directly used by subsequent image processing or display systems.

[0057] Specifically, this embodiment integrates structured data encapsulation and standardized format processing mechanisms. It utilizes data tuple organization algorithms for structured management of bounding box information and time-series tagging traceability. A unique identifier allocation strategy dynamically constructs a traceable output data structure, and pixel data undergoes channel rearrangement and bit-depth standardization according to standard image format specifications. This unified output interface mechanism solves the problems of non-standard data formats and system integration difficulties in traditional video processing results, improving the standardization and system compatibility of the processing results. Simultaneously, a complete data encapsulation and formatting strategy ensures the traceability of the output data and seamless integration with subsequent processing systems.

[0058] In one specific embodiment, please refer to Figure 2 This is a schematic diagram illustrating the processing effect of the Skip Convolution algorithm obtained from simulation on a sequence of 9 consecutive image frames, where: Frames 1-9 represent a continuous sequence of video frames, each 400×300 pixels in size. A red-yellow gradient block moves horizontally to the right at a rate of 20 pixels per frame; a blue-purple gradient block moves slowly downwards vertically at a rate of 1 pixel every 4 frames; a cyan-green gradient block appears starting from the 3rd frame and moves downwards vertically at a rate of 20 pixels per frame; colored bounding boxes represent the bounding boxes of moving areas detected by the system; and "New Object" labels indicate newly identified objects. The coordinate axes include an X-axis range of 0-400 and a Y-axis range of 0-300, used to display pixel coordinate positions.

[0059] Simulation results show that the target detection box of the red-yellow gradient block can maintain accurate selection while moving to the right. In the third frame (Frame 3), after the cyan-green gradient block enters the field of view, the system accurately identifies it as a newly appearing target (labeled "New Object") and continues to track it in subsequent frames. In frames 6 to 8, the overlapping area of ​​the detection boxes of the red-yellow and cyan-green gradient blocks exceeds the set threshold of 20%, and the system successfully triggers the box fusion mechanism to merge the overlapping detection boxes. Due to the slow movement of the blue-purple gradient block (moving 1 pixel every 4 frames), the system only identifies its displacement after every 4 frame intervals (when the cumulative movement exceeds the detection threshold) and performs pixel recalculation and box selection update for this area.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A convolutional method for residual frame video processing based on dynamic gating, characterized in that, Includes the following steps: The current frame image and the previous frame image are obtained, and grayscale conversion processing is performed on the current frame image and the previous frame image respectively to obtain the current frame grayscale image and the previous frame grayscale image; Gaussian blur filtering is applied to the current frame grayscale image and the previous frame grayscale image respectively to obtain the current frame filtered image and the previous frame filtered image; Calculate the absolute difference between the current frame filtered image and the previous frame filtered image to obtain the inter-frame difference image; The inter-frame difference image is subjected to threshold binarization to obtain a binary mask image of the motion region; A morphological dilation operation is performed on the binary mask image of the motion region to obtain a dilated mask image; Contour detection is performed on the inflated mask image to extract the contours of the moving regions and calculate the minimum bounding rectangle of each contour to obtain an initial bounding box set. The overlapping bounding boxes in the initial bounding box set are merged to obtain a merged bounding box set; The composite background frame is updated based on the merged bounding box set. The pixel regions in the current frame image located within the merged bounding box set are copied to the corresponding positions in the composite background frame, while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, thus obtaining the updated composite background frame. Output the merged bounding box set and the updated composite background frame.

2. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The step of calculating the absolute difference between the current frame filtered image and the previous frame filtered image to obtain an inter-frame difference image includes: The absolute difference between the pixel values ​​at corresponding positions in the current frame filtered image and the previous frame filtered image is calculated pixel by pixel to obtain the original difference matrix; For each pixel position in the original difference matrix, a threshold judgment is performed. When the pixel difference is greater than a preset motion threshold, the corresponding pixel position is marked as a moving pixel. When the pixel difference is less than or equal to the preset motion threshold, the corresponding pixel position is marked as a static pixel. The labeling results of the moving pixels and the static pixels are combined to obtain an inter-frame difference image.

3. The residual frame video processing convolution method based on dynamic gating as described in claim 2, characterized in that, The pixel-by-pixel calculation includes: Obtain the grayscale pixel values ​​at the same coordinate positions in the current frame filtered image and the previous frame filtered image, and record them as the current pixel value and the previous pixel value, respectively; Calculate the absolute value of the difference between the current pixel value and the previous pixel value to obtain the pixel difference at the current pixel position. Set the preset motion threshold to 2. When the pixel difference is greater than 2, set the pixel value at that position to 255 and mark it as a moving pixel. When the pixel difference is less than or equal to 2, the pixel value at that location is set to 0 and marked as a static pixel; Iterate through all pixel positions in the image until the inter-frame difference image is generated.

4. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The step of merging overlapping bounding boxes in the initial bounding box set to obtain a merged bounding box set includes: The bounding boxes in the initial bounding box set are sorted in ascending order according to the horizontal coordinate value of the top left corner to obtain a sorted bounding box sequence. Traverse the sorted bounding box sequence and calculate the ratio of the overlapping area of ​​the two bounding boxes to the smaller area of ​​the two bounding boxes to obtain the overlap ratio; When the overlap ratio is greater than the preset overlap threshold, the two bounding boxes are merged to obtain a merged bounding box. When the overlap ratio is less than or equal to the preset overlap threshold, the two bounding boxes are kept independently until the merging judgment of all adjacent bounding boxes is completed, and then a merged bounding box set is obtained.

5. The residual frame video processing convolution method based on dynamic gating as described in claim 4, characterized in that, The fusion process includes: Extract the top-left and bottom-right coordinates of the first bounding box, and extract the top-left and bottom-right coordinates of the second bounding box. When the overlap ratio is greater than a preset overlap threshold, determine the top-left coordinate of the fused bounding box by calculating the minimum x-coordinate and minimum y-coordinate values ​​of all coordinate points of the two bounding boxes, and determine the bottom-right coordinate of the fused bounding box by calculating the maximum x-coordinate and maximum y-coordinate values ​​of all coordinate points of the two bounding boxes. Generate the fused bounding box, and remove the original two bounding boxes from the sorted bounding box sequence.

6. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The step of updating the composite background frame based on the merged bounding box set involves copying the pixel regions located within the merged bounding box set in the current frame image to their corresponding positions in the composite background frame, while keeping the regions in the composite background frame not covered by the merged bounding boxes unchanged, to obtain the updated composite background frame, including: Traverse each bounding box in the merged bounding box set, extract pixel region data within the bounding box range from the current frame image, map the extracted pixel region data to the corresponding region position of the synthesized background frame according to the coordinate position, perform pixel value replacement operation, and keep the pixel values ​​of static regions outside the range of the merged bounding box set in the synthesized background frame unchanged, to obtain the updated synthesized background frame, which includes motion region update information and static region history information.

7. The residual frame video processing convolution method based on dynamic gating as described in claim 6, characterized in that, The pixel value replacement operation includes: Obtain the coordinates of the top-left and bottom-right corners of the bounding box, determine the range of pixel coordinates covered by the bounding box, traverse each pixel position within the range of pixel coordinates, read the RGB pixel value of each pixel position from the current frame image, and directly assign the read RGB pixel value to the same pixel coordinate position in the synthesized background frame, until the update operation of all pixel positions within the bounding box area is completed, and obtain the hybrid background frame structure. The hybrid background frame structure includes using the current frame pixels in the dynamic motion area and retaining historical pixels in the static background area.

8. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The step of thresholding and binarizing the inter-frame difference image to obtain a binary mask image of the motion region includes: A binarization threshold is determined for the pixel value of each pixel in the inter-frame difference image. When the pixel value is greater than the preset binarization threshold, the corresponding pixel is set as the foreground pixel value. When the pixel value is less than or equal to the preset binarization threshold, the corresponding pixel is set as the background pixel value, thus obtaining the initial binarized image. The initial binary image is subjected to pixel value normalization processing, with the foreground pixel values ​​being uniformly normalized to 255 and the background pixel values ​​being uniformly normalized to 0, to obtain a binary mask image of the motion region.

9. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The process involves contour detection of the expanded mask image, extraction of motion region contours, and calculation of the minimum bounding rectangle for each contour to obtain an initial bounding box set, including: Contour detection is performed on the inflated mask image to extract all external contours and calculate the bounding area of ​​each external contour. External contours with bounding areas greater than a preset minimum area threshold are marked as valid contours to obtain a set of valid contours. Traverse each valid contour in the set of valid contours, calculate the coordinates of the four vertices of the minimum bounding rectangle of the valid contour, mark the coordinates of the top left corner and the bottom right corner respectively, construct a bounding box format rectangle based on the coordinates of the top left corner and the bottom right corner, and obtain the initial bounding box set.

10. The residual frame video processing convolution method based on dynamic gating as described in claim 1, characterized in that, The output of the merged bounding box set and the updated composite background frame includes: The merged bounding box set is subjected to structured data encapsulation processing to generate bounding box output data, resulting in structured bounding box data. The bounding box output data includes bounding box coordinates, identification information, and timing information. The bounding box coordinates include the coordinates of the upper left corner and the lower right corner. The identification information includes a unique identifier for the bounding box. The timing information includes the current frame number. The updated synthetic background frame is formatted by encoding the pixel data according to the number of image channels and pixel bit depth to generate synthetic background frame output data, thus obtaining formatted synthetic background frame data.

Images