A surveillance video compression and restoration method

Abstract

The present invention relates to the field of data processing technology, and more particularly to a surveillance video compression and restoration method. The method comprises video compression processing, including extracting reference frames and reducing image quality, and video restoration processing, including image replacement and super-resolution algorithm restoration. This method, designed based on the inherent characteristics of surveillance video, reduces storage space and network transmission bandwidth, and achieves high-definition video after restoration.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a monitoring video compression and restoration method. Background Art

[0002] In accordance with the national requirements for the development and construction of safe cities, the focus is on protecting the safety of the people, property, public places, key warehouses, and all kinds of materials. The original large-scale investment in manpower and the traditional way of maintaining social order have been upgraded to the popularization of monitoring IPC cameras. The integrated safe city assessment system based on the video big data established by the widespread use of IPC cameras has become the largest achievable platform at present.

[0003] At present, the key material safety issues of the power system have received further attention, but there are still problems such as insufficient storage and insufficient bandwidth. We can no longer use the original method of continuously adding equipment to ensure normal information operation. Summary of the Invention

[0004] In order to overcome the deficiencies in the prior art, the present invention provides a surveillance video compression and restoration method designed based on the inherent characteristics of the surveillance video, which has high definition after restoration.

[0005] To achieve the above-mentioned purpose, the present invention adopts the following technical solutions: A monitoring video compression and restoration method, characterized in that:

[0006] Video compression processing:

[0007] A1. Input original video data;

[0008] A2. Extract reference frames. A high-resolution color reference frame is extracted every few frames, and the position of the reference frame corresponding to the original video frame is recorded to form a reference frame set.

[0009] A3. Compress the original video data to reduce its resolution and remove color information to obtain low-resolution grayscale degraded video data;

[0010] Video restoration processing:

[0011] B1. Obtain a reference frame set and degraded video data;

[0012] B2. The degraded video data is segmented by frame position to obtain a plurality of degraded frame sets, each of which has a corresponding frame at a frame position of a reference frame, and the frame serves as a reference degraded frame;

[0013] B3. In a degraded frame set:

[0014] B31. Increase the resolution of the degraded frame to match the resolution of the reference frame;

[0015] B32. Perform image partitioning on the degraded frame, the reference frame, and the reference degraded frame, and compare the degraded frame to be processed with the reference degraded frame to determine whether the partition remains unchanged or changes.

[0016] B33. If the partition is unchanged, the partition image corresponding to the reference frame is assigned to the corresponding position of the degraded frame to be processed for replacement;

[0017] B34. If the segmented area is a changed area, perform subject recognition: compare and match the subject within the segmented area of ​​the degraded frame to be processed with the subject within the segmented area of ​​the reference degraded frame. Calculate a deformation path based on the transformation from the reference degraded frame to the degraded frame to be processed. Apply this deformation path to the corresponding segment of the reference frame to obtain a deformed reference frame segment image.

[0018] B35. Extracting detailed features of the deformed reference frame image partition, reconstructing a super-resolution image of the changed partition of the degraded frame to be processed based on the detailed features and assigning colors to obtain a processed image of the changed partition, and assigning the processed image of the changed partition to the corresponding position of the degraded frame to be processed for replacement;

[0019] B36. Obtain a restored frame image with enhanced resolution and color;

[0020] B4. Perform the restoration operation of B3 above on all degraded frame sets of the degraded video data, and combine them to obtain restored video data.

[0021] Furthermore, in B32, the image is divided equally in width and length directions to obtain several partitions. The partitions of the degraded frame image are compared with the partitions of the reference frame image through a similarity algorithm. If the similarity between the two is greater than or equal to 95%, they are judged as unchanged partitions. If the similarity between the two is less than 95%, they are judged as changed partitions.

[0022] Furthermore, in B34, the changed partitions at similar positions are combined, and the adjacent unchanged partitions are expanded to obtain large changed partitions, and the subject is identified through the large changed partitions.

[0023] As can be seen from the above description of the present invention, compared with the prior art, the surveillance video compression and restoration method provided by the present invention has the following advantages:

[0024] The video compression and restoration method designed based on the characteristics of surveillance video reduces storage space and network transmission bandwidth; the image restoration process is carried out by combining replacement and reconstruction. The unchanged picture adopts the local replacement method of the reference frame image, which has low computational complexity and high speed, and is very suitable for video processing such as surveillance video, which has mostly unchanged pictures. The changed image is restored by the super-resolution algorithm. The restoration algorithm based on the reference frame picture enables the reconstruction of details in the picture to have a high degree of restoration, and the restored video has high clarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of a surveillance video compression and restoration method of the present invention.

[0026] Figure 2 FIG. 1 is a schematic diagram of a degraded frame according to an embodiment of the present invention.

[0027] Figure 3 FIG. 1 is a schematic diagram of a reference degraded frame according to an embodiment of the present invention.

[0028] Figure 4 For the present invention Figure 2 Enlarged view of the middle (P1, Q1) partition.

[0029] Figure 5 For the present invention Figure 3 Enlarged view of the middle (P1, Q1) partition.

[0030] Figure 6 For the present invention Figure 2 Enlarged view of the middle (P13, Q3) partition.

[0031] Figure 7 For the present invention Figure 3 Enlarged view of the middle (P13, Q3) partition.

[0032] Figure 8 For the present invention Figure 2 Degraded frame picture changes in large partitions.

[0033] Figure 9 For the present invention Figure 3 Refer to the large partition of the degraded frame picture change. DETAILED DESCRIPTION

[0034] The present invention is further described below through specific embodiments.

[0035] Reference Figures 1 to 9 As shown, a monitoring video compression and restoration method includes:

[0036] The surveillance camera shoots and obtains surveillance video data. For example, the original video data has a resolution of 1080P and a quality of 20 frames per second.

[0037] Video compression processing:

[0038] A1. Input the original video data.

[0039] A2. Extract reference frames. A high-resolution color reference frame is extracted every several frames, and the frame number position of the reference frame corresponding to the original video is recorded to form a reference frame set. In this embodiment, two frames are extracted per second, specifically the 5th and 15th frames of each second.

[0040] A3. Compress the original video data to reduce its resolution and remove color information, obtaining low-resolution grayscale degraded video data. Specifically, the original 1080P resolution video is reduced to 720P resolution and grayscaled to remove color information.

[0041] Video restoration processing:

[0042] B1. Obtain a reference frame set and degraded video data;

[0043] B2. The degraded video data is segmented by frame position to obtain several degraded frame sets, each of which contains a frame corresponding to the frame position of a reference frame, which serves as a reference degraded frame. Specifically, every 0.5 seconds, or 10 frames, is considered a frame set, and the fifth frame in the sorted position in the frame set serves as the reference degraded frame.

[0044] B3. In a degraded frame set:

[0045] B31. Increase the resolution of the degraded frame to match the reference frame resolution. Specifically, the 720P degraded frame is restored to 1080P resolution by scaling up the canvas resolution. Although the resolution is increased, there is a loss of detail and blurred edges.

[0046] B32. Perform image partitioning on the degraded frame, the reference frame, and the reference degraded frame, compare the degraded frame to be processed with the reference degraded frame, and determine whether the partition remains unchanged or changes; specifically: reference Figure 2 Figure 3 As shown, the width and length directions are divided into 15 equal parts, dividing the image into 225 parts. Figure 4 Figure 5 As shown, the (P1, Q1) partition of the degraded frame is compared with the (P1, Q1) partition of the reference degraded frame. The similarity between the two is calculated by the similarity algorithm to be greater than 95%, so the P1 partition is judged to be the unchanged partition; the reference Figure 6 Figure 7As shown, the P(P13, Q3) partition of the degraded frame is compared with the (P13, Q3) partition of the reference degraded frame. The similarity between the two is calculated by the similarity algorithm to be less than 95%, so the P5 partition is determined to be a changed partition.

[0047] B33. If the partition is an unchanged partition, assign the partition image corresponding to the reference frame picture to the corresponding position of the degraded frame picture to be processed for replacement; specifically: replace the (P1, Q1) partition of the reference frame picture with the (P1, Q1) partition of the degraded frame picture to be processed.

[0048] B34. If the segmented area is a changed area, perform subject recognition: compare and match the subject within the segmented area of ​​the degraded frame to be processed with the subject within the segmented area of ​​the reference degraded frame. Calculate a deformation path based on the transformation from the reference degraded frame to the degraded frame to be processed. Apply this deformation path to the corresponding segment of the reference frame to obtain a deformed reference frame segment image.

[0049] Specifically: In order to ensure the accuracy of subject recognition, the change partitions are combined and expanded, referring to Figure 8 Figure 9 As shown in the figure, P(P13, Q3), P(P13, Q4), P(P12, Q5) and so on are change partitions, and these partitions are in close positions. The adjacent unchanged partitions are expanded to obtain the large change partitions. Figure 7 Figure 8 As shown, the edges P(P12, Q3), (P14, Q3), (P12, Q14), and (P14, Q14) are the four corner positions of the large change partition. The large change partition of the reference degraded frame and the large change partition of the degraded frame to be processed are input for calculation to obtain a deformation path from the subject deformation in the large change partition of the reference degraded frame to the subject state of the large change partition of the degraded frame to be processed, such as zooming in and out, twisting, stretching, and offsetting. This deformation path is applied to the large change partition of the reference frame to obtain a deformed reference frame partition image with the subject position state deformed to a state close to that of the reference degraded frame.

[0050] B35. Extract the detail features of the deformed reference frame partition image, perform super-resolution image reconstruction on the changed partition of the degraded frame to be processed based on the detail features and assign colors to obtain a processed image of the changed partition, and assign the processed image of the changed partition to the corresponding position of the degraded frame to be processed to replace it; specifically: adopt the SRCNN super-resolution image algorithm, input the deformed reference frame partition image as a reference, reconstruct the changed large partition of the degraded frame to be processed, and by using the detail features of the reference image, the processed image has more pixels and richer details, and more high-frequency components are supplemented, thereby obtaining a clear and high-resolution changed large partition image, and replacing the changed large partition image position of the degraded frame to be processed with this image.

[0051] B36. Obtain a restored frame image with enhanced resolution and color;

[0052] B4. Perform the restoration operation of B3 above on all degraded frame sets of the degraded video data, and combine them to obtain restored video data.

[0053] Surveillance video typically features fixed camera angles, resulting in a high percentage of unchanging image segments within a short period. Therefore, replacing the reference frame can be used to quickly obtain a large portion of the image with increased resolution and color. For changing segments, since five frames correspond to 0.25 seconds, the image changes within that five-frame period are minimal. Therefore, after identifying the subject, a small amount of deformation can be used to transform the subject's state in the reference frame to the corresponding state in the degraded frame to be processed. Based on this image, the SRCNN algorithm uses machine learning to identify details, enabling faster and more accurate image sharpening through image fusion.

[0054] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A surveillance video compression and restoration method, characterized by: include Video compression processing: A1. Input original video data; A2. Extract reference frames. A high-resolution color reference frame is extracted every few frames, and the position of the reference frame corresponding to the original video frame is recorded to form a reference frame set. A3. Compress the original video data to reduce its resolution and remove color information, obtaining low-resolution grayscale degraded video data; Video restoration processing: B1. Obtain a reference frame set and degraded video data; B2. The degraded video data is segmented by frame position to obtain a plurality of degraded frame sets, each of which has a corresponding frame at a frame position of a reference frame, and the frame serves as a reference degraded frame; B3. In a degraded frame set: B31. Increase the resolution of the degraded frame to match the resolution of the reference frame; B32. Perform image partitioning on the degraded frame, the reference frame, and the reference degraded frame, and compare the degraded frame to be processed with the reference degraded frame to determine whether the partition remains unchanged or changes. B33. If the partition is unchanged, the partition image corresponding to the reference frame is assigned to the corresponding position of the degraded frame to be processed for replacement; B34. If the segmented area is a changed area, perform subject recognition: compare and match the subject within the segmented area of ​​the degraded frame to be processed with the subject within the segmented area of ​​the reference degraded frame. Calculate a deformation path based on the transformation from the reference degraded frame to the degraded frame to be processed. Apply this deformation path to the corresponding segment of the reference frame to obtain a deformed reference frame segment image. B35. Extracting detailed features of the deformed reference frame image partition, reconstructing a super-resolution image of the changed partition of the degraded frame to be processed based on the detailed features and assigning colors to obtain a processed image of the changed partition, and assigning the processed image of the changed partition to the corresponding position of the degraded frame to be processed for replacement; B36. Obtain a restored frame image with enhanced resolution and color; B4. Perform the restoration operation of B3 above on all degraded frame sets of the degraded video data, and combine them to obtain restored video data.

2. The surveillance video compression and restoration method according to claim 1, wherein: In B32, the image is divided equally in width and length to obtain several partitions. The partitions of the degraded frame are compared with the partitions of the reference frame using a similarity algorithm. If the similarity between the two is greater than or equal to 95%, they are judged as unchanged partitions. If the similarity between the two is less than 95%, they are judged as changed partitions.

3. The surveillance video compression and restoration method according to claim 1, wherein: In B34, the change partitions in similar positions are combined, and the adjacent unchanged partitions are expanded to obtain large change partitions, and the subject is identified through the large change partitions.

Images