Video monitoring management method and system

Abstract

The invention relates to the technical field of video processing, and particularly discloses a video monitoring management method and system, and the method comprises the following steps: obtaining a video frame sequence collected by video monitoring equipment, and taking any two adjacent video frames in the video frame sequence as a pair of continuous frames; calculating a motion vector field of the pair of consecutive frames; extracting a high-frequency motion component from the motion vector field; deriving spatial transformation mapping according to the high-frequency motion component, wherein the spatial transformation mapping is used for compensating geometric distortion of a video frame caused by the atmospheric turbulence; and applying the spatial transformation mapping to each video frame of the video frame sequence, and generating an output video according to the video frames after the spatial transformation mapping is applied. The method can improve the identifiability and visual continuity of the image in a long-distance monitoring environment with the significant influence of atmospheric turbulence, and enhances the recognition and tracking capability of a monitoring system to a target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a video monitoring management method and system. BACKGROUND

[0002] In the intelligent construction site and other long-distance monitoring scenes in municipal construction, atmospheric turbulence is a key factor affecting the imaging quality. Due to the differences in temperature and humidity of different height air layers, turbulence groups with uneven density are formed in the atmosphere. When light passes through these turbulence areas, the propagation path of the light will be refracted and scattered randomly, causing the wavefront of the light to be distorted and the arrival time to fluctuate.

[0003] This effect is reflected in the camera imaging, which is manifested as "shaking" of the received light, causing the target image to produce continuous fluctuations, blurring or ghosting and other phenomena. Although the monitoring device itself has a long detection capability, the image degradation caused by atmospheric turbulence will significantly reduce the effective recognition distance and image clarity, thereby substantially affecting the accurate identification and continuous tracking of the target, becoming a technical challenge that cannot be ignored in practical applications. SUMMARY

[0004] The purpose of the present application is to provide a video monitoring management method and system to solve the above technical problems.

[0005] The purpose of the present application can be achieved by the following technical solutions: A video monitoring management method, comprising the following steps: Obtaining a video frame sequence collected by a video monitoring device, and taking any two adjacent video frames in the video frame sequence as a pair of continuous frames; Calculating a motion vector field of the pair of continuous frames, the motion vector field representing the motion displacement of pixels between the two adjacent video frames; Extracting a high-frequency motion component from the motion vector field, the high-frequency motion component being characterized by randomness and spatial non-uniformity, and being associated with image shaking caused by atmospheric turbulence; Deriving a spatial transformation mapping from the high-frequency motion component, the spatial transformation mapping being used to compensate for the geometric distortion of the video frames caused by the atmospheric turbulence; Applying the spatial transformation mapping to each video frame of the video frame sequence, and generating an output video according to the video frames after applying the spatial transformation mapping.

[0006] As a further scheme of the present application, the process of calculating the motion vector field is: Respectively taking the former video frame and the latter video frame in the pair of continuous frames as a reference video frame and a target video frame, and respectively dividing the reference video frame and the target video frame into a plurality of grids; The grid A in the reference video frame and the target video frame are denoted as grid A1 and grid A2, respectively. A rectangular area of ​​a predetermined size is set in the target video frame with grid A2 as the center, and the grids in the rectangular area are used as candidate grids. Calculate the average pixel value X1 of the pixels in grid A1, and calculate the average pixel value X2 of the pixels in the candidate grid. Calculate the pixel deviation X'=|X1-X2|, and take the candidate grid corresponding to the minimum value of the pixel deviation as the target grid A1' of grid A1. Obtain the center point coordinates Y1 of grid A1 and the center point coordinates Y2 of target grid A1'. Calculate the motion vector Y = Y2 - Y1 of grid A1. Refer to the motion vectors of all grids in the video frame to form the motion vector field of the corresponding pair of consecutive frames.

[0007] As a further aspect of the present invention: extracting high-frequency motion components from the motion vector field includes: For a single grid, extract the corresponding motion vector from each motion vector field, denoted as the target vector, calculate the variance of the target vector, denoted as the target variance, and if the target variance is greater than the preset target variance threshold, mark the corresponding grid as an abnormal grid. The abnormal meshes are grouped, and each abnormal mesh in a group has at least one edge with the other abnormal meshes in the same group. The abnormal meshes in each group are considered as a turbulence candidate region in a portion of the video frame. Extract all motion vectors belonging to the turbulence candidate region from the motion vector field, and use the set of these motion vectors as the high-frequency motion component; If the number of abnormal grids in a group is less than a preset threshold, then the group does not correspond to a turbulence candidate region.

[0008] As a further aspect of the present invention: deriving the spatial transformation mapping based on the high-frequency motion components includes: The average direction and average amplitude of all motion vectors in the high-frequency motion component are respectively used as the main correction direction and reference amplitude; A two-dimensional Gaussian function is generated based on the center position D of the turbulence candidate region in the video frame. The two-dimensional Gaussian function reaches its maximum value at the center position D and then smoothly decays towards the edge. The principal correction direction is used as the direction of the geometric correction vector for each grid within the turbulence candidate region; the function value of the two-dimensional Gaussian function at the grid is multiplied by the reference amplitude to obtain the amplitude of the geometric correction vector for the grid. A geometric correction vector that assigns a zero value to the grid outside the turbulence candidate region; The geometric correction vectors of all grids are spatially smoothed using a bilinear interpolation algorithm to generate a continuous vector field covering the entire video frame, which is the spatial transformation mapping.

[0009] As a further aspect of the present invention: the process of applying the spatial transformation mapping to each video frame of the video frame sequence is as follows: Extract the current video frame from the video frame sequence as the video frame to be processed, and create a blank frame with the same size as the video frame to be processed as the output video frame; For pixel f in the output video frame, read the corresponding geometric correction vector f1 in the spatial transformation mapping, and subtract the geometric correction vector f1 from the coordinates of pixel f in the output video frame to obtain the coordinates of the source pixel. When both the x-coordinate and y-coordinate of the source pixel are integers, the pixel value of pixel f in the video frame to be processed is directly read, and the obtained pixel value is filled into pixel f of the output video frame. When the x-coordinate and / or y-coordinate of the source pixel point coordinates are not integers, select the pixel values ​​of the four pixels in the video frame to be processed that are closest to the source pixel point coordinates, use the bilinear interpolation algorithm to calculate the pixel value at the source pixel point coordinates, and fill the obtained pixel value into the pixel point f of the output video frame; After filling in the pixel values ​​of all pixels in the output video frame, the corrected output video frame is obtained; All output video frames are arranged in timeline order to form the output video.

[0010] As a further aspect of the present invention, it also includes: The average motion amplitude and the main motion direction are obtained by analyzing the statistical distribution of all motion vectors in the motion vector field. Multiplying the average motion amplitude by a preset scaling factor yields the reference size of the rectangular region; Rotate the mainstream direction of motion by 90 degrees to obtain the vertical direction; The side length of the rectangular region in the main direction of motion is set as the reference size, and the side length of the rectangular region in the vertical direction is set as half of the reference size.

[0011] As a further aspect of the present invention: obtaining the average motion amplitude and the main motion direction includes: Obtain the amplitude of each motion vector in the motion vector field, and calculate the mean of the amplitudes as the average motion amplitude; Each motion vector in the motion vector field is converted into a unit vector, and the sum of all unit vectors is calculated to obtain the mainstream direction vector. The direction of the mainstream direction vector is taken as the mainstream motion direction.

[0012] A video surveillance management system, comprising: Acquisition module: Acquires the video frame sequence captured by the video surveillance equipment, and takes any two adjacent video frames in the video frame sequence as a pair of consecutive frames; Calculation module: Calculates the motion vector field of the pair of consecutive frames, wherein the motion vector field represents the motion displacement of a pixel between two adjacent video frames; Extraction module: Extracts high-frequency motion components from the motion vector field. The high-frequency motion components are characterized by randomness and spatial non-uniformity, and are associated with image jitter caused by atmospheric turbulence. Optimization module: Derives a spatial transformation map based on the high-frequency motion components. The spatial transformation map is used to compensate for the geometric distortion of video frames caused by atmospheric turbulence. Correction module: applies the spatial transformation mapping to each video frame of the video frame sequence, and generates an output video based on the video frames after applying the spatial transformation mapping.

[0013] The beneficial effects of this invention compared to the prior art are as follows: This invention analyzes the motion characteristics of continuous video frames to effectively identify and extract random, high-frequency disturbance components caused by atmospheric turbulence. Based on this disturbance information, a spatial transformation mapping is constructed to perform directional, gradual geometric correction on image regions affected by turbulence. Through the smooth expansion of a continuous vector field across the entire image, this invention adaptively compensates for local distortions in video frames, significantly suppressing image jitter in both time and space. By uniformly reconstructing the corrected video frames, this invention improves the stability and clarity of target contours while maintaining the integrity of the image structure, thereby mitigating blurring, distortion, and slight ghosting caused by turbulence. Overall, this invention can improve image recognizability and visual continuity in long-distance monitoring environments significantly affected by atmospheric turbulence, enhancing the monitoring system's ability to identify and track targets, and providing reliable support for stable imaging in complex environments. Attached Figure Description

[0014] The invention will now be further described with reference to the accompanying drawings.

[0015] Figure 1 This is a flowchart illustrating a video surveillance management method according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 As shown, this invention is a video surveillance management method, comprising the following steps: Obtain the video frame sequence captured by the video surveillance equipment, and take any two adjacent video frames in the video frame sequence as a pair of consecutive frames; Calculate the motion vector field of the pair of consecutive frames, where the motion vector field represents the motion displacement of a pixel between two adjacent video frames; In a preferred embodiment of the present invention, the process of calculating the motion vector field is as follows: The preceding and following video frames in a pair of consecutive frames are respectively used as the reference video frame and the target video frame, and the reference video frame and the target video frame are respectively divided into several grids; The grid A in the reference video frame and the target video frame are denoted as grid A1 and grid A2, respectively. A rectangular area of ​​a predetermined size is set in the target video frame with grid A2 as the center, and the grids in the rectangular area are used as candidate grids. Calculate the average pixel value X1 of the pixels in grid A1, and calculate the average pixel value X2 of the pixels in the candidate grid. Calculate the pixel deviation X'=|X1-X2|, and take the candidate grid corresponding to the minimum value of the pixel deviation as the target grid A1' of grid A1. Obtain the center point coordinates Y1 of grid A1 and the center point coordinates Y2 of target grid A1'. Calculate the motion vector Y = Y2 - Y1 of grid A1. Refer to the motion vectors of all grids in the video frame to form the motion vector field of the corresponding pair of consecutive frames.

[0018] It is important to note that in video frames continuously acquired from video surveillance equipment, any two adjacent frames are considered to be consecutive images with temporal correlation. This close temporal relationship allows the pixel position differences between adjacent frames to reflect the minute displacement changes caused by external disturbances during the imaging process.

[0019] In order to derive motion vectors representing the actual offset of pixels from these subtle changes, it is necessary to establish comparable local regions between two frames.

[0020] The previous frame is regarded as the reference image, and the next frame is regarded as the target image. Multiple grids with fixed positions are divided in the same way in both images, so that each grid represents a stable spatial block in the image, and the overall brightness level of the pixels inside the grid can be used as a region feature for matching.

[0021] When a region in the reference image shifts in the target image due to factors such as turbulence, its appearance will move to a new position. Therefore, using a grid in the reference image as a reference, a rectangular area is defined around its corresponding position in the target image to find the most likely landing point of that grid after its actual movement. By comparing the average pixel value difference between the reference grid and each grid in the candidate area, the slow change in local brightness of the image over a short period can be used to determine which candidate grid is most similar to the reference grid, thus inferring the direction and magnitude of the displacement of the reference grid during the actual imaging process. Once the target position most similar to the reference grid is found, the motion vector of that grid can be determined by the spatial difference between the corresponding center points in two frames. Since each grid is processed in the same way, the motion vectors of all grids together form a complete motion vector field. This vector field essentially records the local displacement distribution of the entire image between two consecutive frames, reflecting the overall disturbance pattern of the image.

[0022] High-frequency motion components are extracted from the motion vector field. These high-frequency motion components are characterized by randomness and spatial non-uniformity and are associated with image jitter caused by atmospheric turbulence. In another preferred embodiment of the present invention, extracting high-frequency motion components from the motion vector field includes: For a single grid, extract the corresponding motion vector from each motion vector field, denoted as the target vector, calculate the variance of the target vector, denoted as the target variance, and if the target variance is greater than the preset target variance threshold, mark the corresponding grid as an abnormal grid. The abnormal meshes are grouped, and each abnormal mesh in a group has at least one edge with the other abnormal meshes in the same group. The abnormal meshes in each group are considered as a turbulence candidate region in a portion of the video frame. Extract all motion vectors belonging to the turbulence candidate region from the motion vector field, and use the set of these motion vectors as the high-frequency motion component; If the number of abnormal grids in a group is less than a preset threshold, then the group does not correspond to a turbulence candidate region.

[0023] It is worth noting that in the motion vector field formed between consecutive frames, regions affected by atmospheric turbulence exhibit significant random fluctuations, while the motion changes in stable regions are usually relatively gentle. In order to distinguish the high-frequency disturbances caused by turbulence from the overall motion information, it is necessary to analyze the motion behavior of each grid by utilizing the characteristic of variance reflecting the amplitude of fluctuations.

[0024] If the vector values ​​of the same grid in different motion vector fields change around a certain stable trend, its dispersion is small and its variance is also low. When the grid is subjected to turbulence, its motion vector will shift irregularly in a short period of time, resulting in a significant increase in the vector difference between different frames and a corresponding increase in variance.

[0025] Based on this statistical characteristic, the variance of the motion vector corresponding to each grid can be calculated to determine whether there are abnormal fluctuations. When the variance exceeds a set threshold, the grid is identified as an abnormal grid.

[0026] Since turbulence often clusters in space, adjacent anomalous grids are more likely to originate from the same turbulent structure. Therefore, grouping anomalous grids with shared boundaries into the same group can identify the local region where turbulence actually occurs, while scattered anomalous grids that are insufficient to form an effective region are discarded. After identifying the turbulent region, extracting all motion vectors within that region from the motion vector field yields high-frequency motion components with randomness and spatial non-uniformity, which characterize the local jitter caused by turbulence and provide a basis for subsequent correction processing.

[0027] A spatial transformation map is derived based on the high-frequency motion components. The spatial transformation map is used to compensate for the geometric distortion of video frames caused by atmospheric turbulence. In another preferred embodiment of the present invention, deriving the spatial transformation mapping based on the high-frequency motion components includes: The average direction and average amplitude of all motion vectors in the high-frequency motion component are respectively used as the main correction direction and reference amplitude; A two-dimensional Gaussian function is generated based on the center position D of the turbulence candidate region in the video frame. The two-dimensional Gaussian function reaches its maximum value at the center position D and then smoothly decays towards the edge. The principal correction direction is used as the direction of the geometric correction vector for each grid within the turbulence candidate region; the function value of the two-dimensional Gaussian function at the grid is multiplied by the reference amplitude to obtain the amplitude of the geometric correction vector for the grid. A geometric correction vector that assigns a zero value to the grid outside the turbulence candidate region; The geometric correction vectors of all grids are spatially smoothed using a bilinear interpolation algorithm to generate a continuous vector field covering the entire video frame, which is the spatial transformation mapping.

[0028] Understandably, deriving the spatial transformation mapping from high-frequency motion components relies on the manifestation of turbulent disturbances in the image and the statistical characteristics of these disturbances. Local jitter caused by turbulence often exhibits a certain concentration trend in direction, while displaying a relatively consistent disturbance intensity in amplitude. Therefore, statistical processing of high-frequency motion components can extract the average direction describing the dominant distortion direction of turbulence from numerous fluctuation vectors, and the average vector length reflects the typical influence amplitude of turbulence in that region. The average direction reflects the main directional characteristics of geometric shift caused by turbulence, and the average amplitude reflects the overall energy level of local image shift caused by turbulence. These two statistical results together constitute the basic characteristics of the correction vector, enabling the correction process to remain consistent with the distortion direction and intensity caused by turbulence.

[0029] Distortions caused by turbulence are often concentrated in a specific area and gradually weaken outwards. Therefore, constructing a two-dimensional Gaussian function with the center of the turbulence candidate region as the peak value allows us to utilize its spatially decreasing distribution characteristic from the center outwards to characterize the range variation of turbulence influence. Since the Gaussian function can continuously and smoothly simulate the attenuation characteristic of disturbance intensity with distance, using its corresponding value as the amplitude factor of the correction vector allows for stronger correction in areas near the turbulence center and weaker correction in peripheral areas, thus better reflecting the actual influence pattern of turbulence. It is worth noting that the two-dimensional Gaussian function is a continuously distributed function with a peak at the center and a smooth decay outwards; its value gradually decreases with increasing distance from the center point, forming a structure similar to a mountain peak transitioning outwards, often used to describe the gradually weakening influence range in space. In this invention, atmospheric turbulence disturbances to images typically concentrate in a specific area and gradually weaken outwards. This influence pattern is highly consistent with the spatial attenuation characteristics of the two-dimensional Gaussian function; therefore, using the Gaussian function to characterize the variation of turbulence intensity within a region is very consistent with physical phenomena. By using the center of the turbulence candidate region as the peak point of the Gaussian function, a corresponding function value can be generated for each grid. The closer the value is to the center, the larger it is, and the farther away from the center, the smaller it is. Multiplying the function value by the reference amplitude yields the correction vector amplitude of the grid, making the correction intensity present a natural and gradual change. This ensures that the correction field is continuous and smooth in space, without producing abrupt boundaries, and is more consistent with the actual distribution law of turbulence.

[0030] After determining the direction and magnitude of the correction vector, the grid outside the turbulent region is not affected by high-frequency disturbances. Therefore, assigning a zero-value correction vector can maintain the original image structure. Since the actual image is a continuous pixel distribution, assigning discrete correction vectors only based on the grid center would result in obvious segmentation of the mapped field. To avoid this problem, bilinear interpolation is used to perform spatial transition processing on the correction vectors of all grids, making the correction changes between adjacent regions smoother. Bilinear interpolation uses the vector information of the surrounding grids to perform a continuous transition through a weighted method. This smoothing method can maintain the spatial continuity of the generated vector field, avoiding visual discomfort caused by abrupt changes, thus forming a continuous spatial transformation mapping covering the entire frame.

[0031] Under this mapping effect, the areas in the image affected by turbulence will be geometrically reversed along the main correction direction with a corresponding magnitude, while the less affected areas will receive a gentler correction, so that the distortion of the overall image is gradually reduced, and finally the geometric distortion caused by atmospheric turbulence is effectively corrected.

[0032] The spatial transformation mapping is applied to each video frame of the video frame sequence, and an output video is generated based on the video frames after the spatial transformation mapping is applied.

[0033] In a preferred embodiment of the present invention, the process of applying the spatial transformation mapping to each video frame of the video frame sequence is as follows: Extract the current video frame from the video frame sequence as the video frame to be processed, and create a blank frame with the same size as the video frame to be processed as the output video frame; For pixel f in the output video frame, read the corresponding geometric correction vector f1 in the spatial transformation mapping, and subtract the geometric correction vector f1 from the coordinates of pixel f in the output video frame to obtain the coordinates of the source pixel. When both the x-coordinate and y-coordinate of the source pixel are integers, the pixel value of pixel f in the video frame to be processed is directly read, and the obtained pixel value is filled into pixel f of the output video frame. When the x-coordinate and / or y-coordinate of the source pixel point coordinates are not integers, select the pixel values ​​of the four pixels in the video frame to be processed that are closest to the source pixel point coordinates, use the bilinear interpolation algorithm to calculate the pixel value at the source pixel point coordinates, and fill the obtained pixel value into the pixel point f of the output video frame; After filling in the pixel values ​​of all pixels in the output video frame, the corrected output video frame is obtained; All output video frames are arranged in timeline order to form the output video.

[0034] It should be noted that during the geometric correction of video frames, the correction vector recorded in the spatial transformation mapping needs to be applied point by point to the pixel coordinates of the current frame, so that each pixel can be accurately back-positioned to its true position during imaging based on the turbulence offset it experiences.

[0035] After the current frame is extracted as the image to be processed, a blank image of the same size is created as the corrected output image. Each pixel in the output image (f represents any pixel) will obtain a geometric correction vector based on its corresponding position in the spatial transformation mapping. This vector reflects the direction and magnitude of the offset caused by turbulence at that point. Therefore, by subtracting the direction and displacement contained in this vector from the coordinates of a pixel in the output image, we can obtain the position that the pixel should be in the original image when it is not distorted, which is the source pixel coordinate.

[0036] The source pixel coordinates may fall exactly at an integer position in the original image, meaning that the position completely coincides with a real pixel in the original image. In this case, correction can be completed by directly reading the pixel value of that point. More often, the source pixel coordinates will fall at a non-integer position, that is, in the continuous space between four adjacent pixels in the original image. The pixels form a local two-dimensional coordinate grid, and each grid point has a nearest neighbor in the vertical and horizontal directions. Therefore, the four pixels closest to the source coordinates refer to the four integer pixels adjacent to the source coordinates in the horizontal and vertical directions.

[0037] To calculate the expected pixel value at the source coordinates, a pixel value transition function that continuously varies with position in two-dimensional space needs to be constructed using a bilinear interpolation algorithm, based on the relative positions of the source coordinates among these four pixels. Bilinear interpolation can establish a linear relationship in two directions simultaneously, ensuring that the final pixel value maintains continuity with the original image content while avoiding unnatural jumps caused by coordinate misalignment.

[0038] Once all pixels in the output image have been filled with their corresponding values, a complete image with geometric distortion correction can be obtained. These corrected frames, arranged in the original time sequence, constitute the final output video, effectively smoothing and correcting the jitter caused by turbulence visually.

[0039] Another preferred embodiment of the present invention further includes: The average motion amplitude and the main motion direction are obtained by analyzing the statistical distribution of all motion vectors in the motion vector field. Multiplying the average motion amplitude by a preset scaling factor yields the reference size of the rectangular region; Rotate the mainstream direction of motion by 90 degrees to obtain the vertical direction; The side length of the rectangular region in the main direction of motion is set as the reference size, and the side length of the rectangular region in the vertical direction is set as half of the reference size.

[0040] It is important to note that obtaining the average range of motion and the main direction of motion includes: Obtain the amplitude of each motion vector in the motion vector field, and calculate the mean of the amplitudes as the average motion amplitude; Each motion vector in the motion vector field is converted into a unit vector, and the sum of all unit vectors is calculated to obtain the mainstream direction vector. The direction of the mainstream direction vector is taken as the mainstream motion direction.

[0041] Understandably, when analyzing the motion vector field to determine the size and orientation of the search rectangle, it is necessary to use the statistical characteristics of the overall motion in the real scene to construct a search structure that is more in line with physical laws, so that subsequent motion estimation can complete the matching operation within a more reasonable spatial range.

[0042] The motion vector field reflects the displacement of local areas in the scene between consecutive frames, and the amplitude of different vectors can reflect the strength of the overall scene motion. Therefore, averaging the amplitudes of all vectors can yield a representative value of the overall motion magnitude of the scene. When the average amplitude is large, the actual displacement of objects in the scene between adjacent frames will also increase accordingly. If a fixed small range is still used as the search area, it may be impossible to find a position that matches the reference block content in the target frame, causing real large displacements to be mistaken for abnormal fluctuations, thus affecting the accuracy of turbulence detection.

[0043] By multiplying the average amplitude by a preset scaling factor to obtain the baseline size, the search area can be automatically expanded or shrunk with the overall movement speed, so that it always maintains a reasonable scale that can cover the actual range of movement.

[0044] In addition to amplitude, motion vectors also contain directional information. The overall motion of large-scale objects in a scene usually has a significant consistency in direction. Therefore, normalizing all vectors into unit vectors of the same length and accumulating these unit vectors can make the truly dominant motion direction form the result with the largest directional quantity after accumulation, thus obtaining the mainstream direction that can represent the overall motion trend.

[0045] Based on this directional relationship, the shape of the rectangular region is adjusted to have a longer side length in the mainstream direction and a shorter side length in the perpendicular direction. This leverages the generally greater uncertainty of actual motion along the trajectory direction, improving the fault tolerance of matching in the mainstream direction and reducing the risk of mismatches due to noise in the perpendicular direction. When the design of the rectangular region more closely matches the actual motion distribution, the estimation accuracy of the motion vector field will improve. A high-quality motion field can avoid miscalculating the motion of real objects into the turbulence component, making the subsequent extraction of high-frequency turbulence components more reliable, thus providing a more stable foundation for the entire turbulence correction algorithm.

[0046] A video surveillance management system, comprising: Acquisition module: Acquires the video frame sequence captured by the video surveillance equipment, and takes any two adjacent video frames in the video frame sequence as a pair of consecutive frames; Calculation module: Calculates the motion vector field of the pair of consecutive frames, wherein the motion vector field represents the motion displacement of a pixel between two adjacent video frames; Extraction module: Extracts high-frequency motion components from the motion vector field. The high-frequency motion components are characterized by randomness and spatial non-uniformity, and are associated with image jitter caused by atmospheric turbulence. Optimization module: Derives a spatial transformation map based on the high-frequency motion components. The spatial transformation map is used to compensate for the geometric distortion of video frames caused by atmospheric turbulence. Correction module: applies the spatial transformation mapping to each video frame of the video frame sequence, and generates an output video based on the video frames after applying the spatial transformation mapping.

[0047] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A management method of video surveillance, characterized by, The method comprises the following steps: obtaining a video frame sequence collected by a video monitoring device, taking any two adjacent video frames in the video frame sequence as a pair of continuous frames; calculating a motion vector field of the pair of continuous frames, the motion vector field representing the motion displacement of pixels between the two adjacent video frames; extracting a high-frequency motion component from the motion vector field, the high-frequency motion component being characterized by randomness and spatial non-uniformity and being related to image jitter caused by atmospheric turbulence; deriving a spatial transformation mapping from the high-frequency motion component, the spatial transformation mapping being used to compensate for geometric distortion of the video frames caused by atmospheric turbulence; applying the spatial transformation mapping to each video frame of the video frame sequence and generating an output video according to the video frames after the spatial transformation mapping is applied.

2. The method of claim 1, wherein, The process of calculating the motion vector field comprises: respectively taking the former video frame and the latter video frame in the pair of continuous frames as a reference video frame and a target video frame, and respectively dividing the reference video frame and the target video frame into a plurality of grids; denoting the grid A in the reference video frame and the target video frame as grid A1 and grid A2 respectively, setting a rectangular region of a predetermined size in the target video frame with the position of grid A2 as the center, and taking the grids in the rectangular region as candidate grids; calculating the average pixel value X1 of the pixel points in grid A1, calculating the average pixel value X2 of the pixel points in the candidate grids, calculating the pixel deviation X'=|X1-X2|, and taking the candidate grid corresponding to the minimum value in the pixel deviation as the target grid A1' of grid A1; obtaining the center point coordinate Y1 of grid A1 and the center point coordinate Y2 of the target grid A1', calculating the motion vector Y=Y2-Y1 of grid A1, and constructing the motion vector field of the corresponding pair of continuous frames by using the motion vectors corresponding to all the grids in the reference video frame.

3. The method of claim 2, wherein, The process of extracting the high-frequency motion component from the motion vector field comprises: for a single grid, extracting the corresponding motion vector from each motion vector field, denoting it as a target vector, calculating the variance of the target vector, denoting it as a target variance, and marking the corresponding grid as an abnormal grid if the target variance is greater than a preset target variance threshold; grouping the abnormal grids, the abnormal grids in a group having at least one common edge with the remaining abnormal grids in the same group, and taking the part of the video frame in which the abnormal grids in each group are located as a turbulence candidate region; extracting all the motion vectors belonging to the turbulence candidate region from the motion vector field, and taking the set of these motion vectors as the high-frequency motion component; wherein if the number of abnormal grids in a group is less than a preset number threshold, the group does not correspond to a turbulence candidate region.

4. The method of claim 1, wherein, The process of deriving the spatial transformation mapping from the high-frequency motion component comprises: respectively taking the average direction and average amplitude of all the motion vectors in the high-frequency motion component as a main correction direction and a reference amplitude; generating a two-dimensional Gaussian function based on the center position D of the turbulence candidate region in the video frame, the two-dimensional Gaussian function taking the maximum value at the center position D and smoothly decaying towards the edge. The main correction direction is taken as the direction of the geometric correction vector of each grid in the turbulence candidate region; the function value of the two-dimensional Gaussian function at the grid is multiplied by the reference amplitude to obtain the amplitude of the geometric correction vector of the grid; A grid outside the turbulence candidate region is given a geometric correction vector with a value of zero; The geometric correction vectors of all grids are spatially smoothed using a bilinear interpolation algorithm to generate a continuous vector field covering the entire video frame, which is the spatial transformation mapping.

5. The method of claim 1, wherein, The process of applying the spatial transformation mapping to each video frame of the video frame sequence is as follows: A current video frame is taken out from the video frame sequence as a video frame to be processed, and a blank picture with the same size as the video frame to be processed is created as an output video frame; For a pixel point f in the output video frame, the corresponding geometric correction vector f1 in the spatial transformation mapping is read, and the coordinates of the pixel point f in the output video frame are subtracted from the geometric correction vector f1 to obtain the coordinates of a source pixel point; When the horizontal and vertical coordinates of the source pixel point coordinates are both integers, the pixel value of the pixel point f in the video frame to be processed is directly read, and the obtained pixel value is filled into the pixel point f of the output video frame; When the horizontal and / or vertical coordinates of the source pixel point coordinates are non-integers, the pixel values of the four nearest pixel points of the source pixel point coordinates in the video frame to be processed are selected, the pixel value at the source pixel point coordinates is calculated using a bilinear interpolation algorithm, and the obtained pixel value is filled into the pixel point f of the output video frame; After the pixel value filling of all pixel points in the output video frame is completed, a corrected output video frame is obtained; All output video frames are arranged in time sequence to form an output video.

6. The method of claim 3, wherein, Further comprising: An average motion amplitude and a main flow direction are obtained by analyzing the statistical distribution of all motion vectors in the motion vector field; A reference size of a rectangular region is obtained by multiplying the average motion amplitude by a preset proportion coefficient; The main flow direction is rotated by ninety degrees to obtain a vertical direction; The length of the rectangular region in the main flow direction is set to the reference size, and the length of the rectangular region in the vertical direction is set to half of the reference size.

7. The method of claim 6, wherein, Obtaining the average motion amplitude and the main flow direction comprises: The average motion amplitude is calculated by obtaining the amplitude of each motion vector in the motion vector field; The main flow direction is obtained by converting each motion vector in the motion vector field into a unit vector, calculating the vector sum of all unit vectors, and obtaining a main flow direction vector.

8. A management system for video surveillance, characterized by Comprising: A collection module: obtaining a video frame sequence collected by a video monitoring device, and taking any two adjacent video frames in the video frame sequence as a pair of consecutive frames; A calculation module: calculating a motion vector field of the pair of consecutive frames, the motion vector field representing the motion displacement of pixels between two adjacent video frames; An extraction module: extracting a high-frequency motion component from the motion vector field, the high-frequency motion component being characterized by randomness and spatial non-uniformity and being related to image jitter caused by atmospheric turbulence; an optimization module configured to derive a spatial transformation map from the high frequency motion components, the spatial transformation map being used to compensate for geometric distortion of the video frames caused by the atmospheric turbulence; a correction module configured to apply the spatial transformation map to each video frame of the sequence of video frames and generate an output video from the video frames after the spatial transformation map is applied.

Images