A monitoring video compression processing method and device

Abstract

The present invention relates to the technical field of video compression processing. A monitoring video compression processing method and device are provided, the method comprising: using a video feature extraction model to extract video features of an image to be processed in a monitoring video of a target scene area, and obtaining feature vectors of multiple video features in different image areas; performing the following classifications on the image to be processed according to the feature vectors of multiple video features in different image areas, and obtaining classification results: image area classification, scene type classification, and importance level classification; determining the initial compression ratio of different image areas in the image to be processed according to the classification results; using an image decomposition method based on wavelet transform to decompose the image to be processed into sub-images of different scales, and determining the optimal number of decomposition layers according to the image quality; determining the final compression ratio and encoding mode of different image areas according to the scale and content characteristics of the sub-images of the image to be processed. The present invention greatly improves the compression ratio without significantly reducing the image quality.

Description

Technical Field

[0001] The invention belongs to the technical field of video processing and provides a monitoring video compression processing method and device. Background Art

[0002] Although traditional video compression algorithms can reduce storage space and transmission bandwidth requirements to a certain extent, the compression ratio is limited. In commercial and enterprise video surveillance management, large amounts of high-definition video data need to be stored for a long time and transmitted frequently. Existing technologies cannot meet this demand well, increasing storage costs and network burdens.

[0003] The image quality loss is large. Traditional algorithms often cause a significant decrease in image quality during the compression process, which may affect the effect and reliability of video surveillance. Especially in large-scale video surveillance systems, the loss of image quality may have serious consequences.

[0004] Therefore, it is necessary to provide a new monitoring video compression processing method to solve the above problems. Summary of the invention

[0005] The present invention provides a monitoring video compression processing method and device to solve the technical problems of image quality loss in the image compression process in the prior art, how to reduce the impact of image quality loss on the effect of monitoring video, how to greatly improve the compression ratio without significantly reducing the image quality, reduce the storage space and transmission bandwidth requirements, and meet the actual needs of commercial and enterprise video surveillance management for the storage and transmission of large amounts of high-definition video data. The technical problems to be solved by the present invention are achieved through the following technical solutions.

[0006] The first aspect of the present invention provides a monitoring video compression processing method, which comprises: using a video feature extraction model to extract video features of an image to be processed in a monitoring video of a target scene area, and obtaining feature vectors of multiple video features in different image areas; classifying the image to be processed according to the feature vectors of multiple video features in different image areas to obtain classification results: image area classification, scene type classification, and importance level classification; determining the initial compression ratio of different image areas in the image to be processed according to the classification results; using an image decomposition method based on wavelet transform to decompose the image to be processed into sub-images of different scales, and determining the optimal decomposition layer number according to image quality; determining the final compression ratio and encoding mode of different image areas according to the scale and content characteristics of the sub-images of the image to be processed.

[0007] The second aspect of the present invention proposes a surveillance video compression processing device, which is the surveillance video compression processing method described in the first aspect of the present invention, and the surveillance video compression processing device includes: an extraction module, which is used to use a video feature extraction model to extract the video features of the image to be processed in the surveillance video of the target scene area, and obtain feature vectors of multiple video features in different image areas; a classification module, which classifies the image to be processed as follows according to the feature vectors of the multiple video features in different image areas to obtain classification results: image area classification, scene type classification, and importance level classification; a first determination module, which determines the initial compression ratio of different image areas in the image to be processed according to the classification results; a second determination module, which uses an image decomposition method based on wavelet transform to decompose the image to be processed into sub-images of different scales, and determines the optimal decomposition layer number according to the image quality; a third determination module, which determines the final compression ratio and encoding mode of different image areas according to the scale and content characteristics of the sub-images of the image to be processed.

[0008] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the surveillance video compression processing method described in the first aspect of the present invention.

[0009] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the surveillance video compression processing method described in the first aspect of the present invention.

[0010] The embodiments of the present invention include the following advantages:

[0011] Compared with the prior art, the present invention uses adaptive compression and multi-scale compression technology based on image features to adaptively adjust compression parameters according to the characteristics of video content, and decomposes video images into sub-images of different scales for independent compression. Without significantly reducing image quality, the compression ratio is greatly improved, and the storage space and transmission bandwidth requirements are reduced, effectively meeting the actual needs of commercial and enterprise video surveillance management for the storage and transmission of large amounts of high-definition video data. Specifically, an image feature extraction method based on deep learning is adopted to accurately extract image features, combine multiple feature information to classify image regions, and adjust compression parameters according to different region types. At the same time, in the process of sub-image compression and image reconstruction, an optimized coding mode and inverse wavelet transform algorithm are adopted, combined with error compensation technology, which effectively improves the reconstruction quality of the image, reduces image quality loss, and effectively ensures the effect and reliability of video surveillance. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1It is a flowchart of the steps of the monitoring video compression processing method of the present invention;

[0013] Figure 2 It is a partial flow chart representing the image quality index in the monitoring video compression processing method of the present invention;

[0014] Figure 3 It is a structural block diagram of the monitoring video compression processing device of the present invention;

[0015] Figure 4 is a schematic structural diagram of an electronic device embodiment according to the present invention;

[0016] Figure 5 It is a schematic diagram of the structure of a computer-readable medium embodiment according to the present invention. DETAILED DESCRIPTION

[0017] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] In view of the above problems, the present invention proposes a monitoring video compression processing method. This method uses adaptive compression and multi-scale compression technology based on image features to adaptively adjust compression parameters according to the characteristics of the video content, and decomposes the video image into sub-images of different scales for independent compression. Without significantly reducing the image quality, the compression ratio is greatly improved, and the storage space and transmission bandwidth requirements are reduced, effectively meeting the actual needs of commercial and enterprise video surveillance management for the storage and transmission of large amounts of high-definition video data. Specifically, an image feature extraction method based on deep learning is adopted to accurately extract image features, combine multiple feature information to classify image regions, and adjust compression parameters according to different region types. At the same time, in the process of sub-image compression and image reconstruction, an optimized coding mode and inverse wavelet transform algorithm are adopted, combined with error compensation technology, which effectively improves the reconstruction quality of the image, reduces the image quality loss, and effectively ensures the effect and reliability of video surveillance.

[0019] Example 1

[0020] Refer to the following Figure 1 , Figure 2 , the contents of the present invention will be described in detail.

[0021] Figure 1 It is a flow chart of the steps of the monitoring video compression processing method of the present invention.

[0022] like Figure 1 As shown, in step S101, a video feature extraction model is used to extract video features of the image to be processed in the surveillance video of the target scene area, and feature vectors of multiple video features in different image areas are obtained.

[0023] It should be noted that the present invention is mainly aimed at efficiently compressing surveillance videos of target scene areas such as commercial and corporate places such as shopping malls, factories, office buildings, etc. The videos of these target scene areas contain dynamic images and static images, which are specifically divided into areas with complex textures (such as shopping malls and factory workshops) and areas with simple textures (such as corridors and parking lots).

[0024] A convolutional neural network (CNN) based on deep learning is used to build a video feature extraction model to extract video features for classifying the processed images.

[0025] Specifically, public video data is collected from different target scene areas such as shopping malls, factory workshops, and office building corridors, and the video images are annotated with different target scene area labels to establish a training data set.

[0026] The video feature extraction model includes an input layer, a convolution layer, a pooling layer and a classification layer.

[0027] The video image (i.e., the image to be processed) annotated with different target scene area labels is input into the input layer of the video feature extraction model. The input feature map size is (in and are the spatial dimensions of the feature map, representing height and width respectively, is the number of input channels). After a series of convolutional layers, pooling layers, and activation functions, the depthwise separable convolution and residual connection techniques are used to reduce the computational complexity of video features in the image to be processed and improve training efficiency.

[0028] Specifically, in the convolution layer, the depth convolution part uses indivual The convolution kernel ( and Represents the side length of the convolution kernel in two different spatial dimensions) and performs convolution operation on each input channel respectively, and the output feature map size is , the calculation formula is:

[0029] ;

[0030] in, Represents the value of the feature map at the image position (i, j); It is The convolution kernels are at positions The weight at (m is the number of the input channel, and t represent the position coordinates of the convolution kernel in the spatial dimension respectively). is the first input feature map Channels at position The value at and They are the position coordinates on the output feature map, specifically the coordinate values ​​on the horizontal axis and the coordinate values ​​on the vertical axis. and t represent the position coordinates of the convolution kernel in the spatial dimension respectively.

[0031] Next, perform point-by-point convolution using indivual The convolution kernel performs channel fusion on the output of the depth convolution, and the output feature map size is .

[0032] Specifically, the output contains the image locations The comprehensive performance of multiple low-level features and high-level features such as color, texture, shape, etc., and the value of the feature map output after the point-by-point convolution operation are calculated using the following expression:

[0033] ;

[0034] in, Indicates that the output contains image locations The comprehensive performance of the three low-level features of color, texture, and shape and the high-level features, and the value of the feature map output after the point-by-point convolution operation; It is The point-by-point convolution kernels are connected to the The weights of the input channels ( Indicates the sequence number or quantity identifier of the point-by-point convolution kernel, Indicates the serial number of the input channel), is the deep convolution output feature map Channels at image locations The value at (i represents the horizontal axis value, j represents the vertical axis value).

[0035] In a neural network, when a network layer has input When , the output is obtained after the specific transformation of this layer After the residual connection is introduced, its output is , the final output is the superposition of the transformed result and the original input.

[0036] In the reverse transmission stage, the error is transmitted from the output layer to the input layer. This process requires calculating the gradient of the loss function with respect to the parameters of each layer in order to update the parameters. However, as the number of network layers continues to increase, according to the chain rule, in the reverse propagation, the continuous multiplication of multiple gradient values ​​less than 1 will cause the gradient value to continue to decrease. Once the gradient is small and approaches zero, the update speed of the parameters will be extremely slow or even stagnant. However, the use of residual connections during model training can effectively address the problem of gradient vanishing during deep network training. Because the residual connection provides an additional path for the gradient, the gradient not only depends on the propagation path after multiple layers of transformation, but can also be directly transmitted through the network. The path transmission avoids the problem of excessive gradient attenuation caused by increasing depth and ensures that the parameters can be continuously and effectively updated, thereby improving the training effect and stability of the network.

[0037] During the model training process, the video features of the images to be processed in the surveillance video of the target scene area are extracted to obtain feature vectors of various video features in different image areas. These feature vectors are a digital representation of the video features in different image areas. For areas with complex textures (such as shopping malls and factory workshops), the feature vectors may contain rich feature information such as edges, texture details, and object shape changes. For example, for shopping malls, feature vectors also include the contour features of different commodities, the movement trajectory features of customers, and the arrangement features of shelves.

[0038] Assume that the feature vector for the shopping area of ​​a mall is ,in

[0039] A quantitative value representing the contour characteristics of a product, which can be expressed by the number of edge pixels obtained by an edge detection algorithm or the intensity value of a specific edge feature; Represents the characteristics of customer movement trajectory, which can be expressed by recording the customer's location changes over a period of time and then using statistical methods to obtain characteristic values ​​such as movement direction and speed; It can represent the characteristics of shelf arrangement and can be expressed by characteristic values ​​such as the linear equation parameters of the shelves and the angles between the shelves.

[0040] For simple texture areas (such as corridors and parking lots), feature vectors focus more on simple geometric shape features, color distribution features, and a small number of dynamic change features. For example, the feature vector of a corridor may mainly contain straight shape features, a single color distribution feature, and dynamic features of people walking.

[0041] Specifically, after being processed by the convolution layer and the pooling layer, the feature vectors of various video features in the image to be processed are output.

[0042] In one embodiment, the feature vector for the corridor is ,in, Representing the shape characteristics of the straight line, for example, using at least one of the following characteristic values: the length and width of the corridor and the slope of the straight line; Represents a single color distribution feature, such as using the main color component in the color histogram; Represents the dynamic features of people's movement, such as the frequency of people's appearance, movement speed, etc.

[0043] It should be noted that, in this example, the specific form of the feature vector is a multi-dimensional numerical array, wherein each dimension represents a specific feature attribute. The specific form of the feature vector can also be a matrix of multiple rows and columns. The above is only described as an optional example and cannot be understood as a limitation to the present invention.

[0044] Next, in step S102, the image to be processed is classified according to the obtained feature vector to obtain classification results, including image region classification, scene type classification, and importance level classification.

[0045] For image region classification, for example, the image to be processed can be divided into different types according to the obtained feature vectors, such as complex texture areas and simple texture areas. In terms of classification, the K-Means clustering algorithm first pre-processes the extracted feature vectors, such as normalization to unify the dimensions.

[0046] Next, k initial cluster centers are determined by random selection. The Euclidean distance from each feature vector to each cluster center is calculated. ,in and are the eigenvectors and cluster center vectors. elements, and assign each feature vector to the cluster to which the nearest cluster center belongs. Then recalculate the mean of each cluster as the new cluster center. Repeat the steps of assigning and updating the cluster center until the cluster center no longer changes or reaches the preset number of iterations (for example, 500 to 1000 iterations), and then stop the clustering calculation process. In this way, in image classification, the image to be processed is divided into different types such as complex texture areas and simple texture areas according to the clustering results of the feature vectors.

[0047] In a specific implementation, a support vector machine algorithm (SVM) is used to classify the image to be processed according to the obtained feature vector, such as into different scene types or importance levels. In the scene type classification, a scene classification model is established according to the feature vector characteristics of different scenes (such as shopping malls, factories, office buildings, etc.). For shopping mall scenes, there are a large number of features such as people and commodity shelves; factory scenes have large machinery and equipment, specific production process-related features, etc. By extracting feature vectors from a large number of images with labeled scene types for training, the SVM algorithm will find the optimal hyperplane for dividing different scene types in the feature space according to the feature vectors of the input images to be processed or classified.

[0048] Combined with function methods such as linear kernel function, polynomial kernel function, Gaussian kernel function, etc., the original feature space is mapped to a higher dimensional space to better handle nonlinear separable situations and improve classification accuracy. In terms of importance level classification, the importance level standard is defined. For example, images of frequent human activities or key areas (such as cash registers, factory valuable equipment storage areas, etc.) in surveillance videos are defined as multiple importance levels, such as high importance level, medium importance level, and low importance level. Extract the feature vectors of the images in these areas, including color, texture, shape, etc., and annotate the importance level as training data. In other words, use the images to be processed (i.e., annotated data) annotated with multiple importance levels for training. During the classification process, classification is performed based on the degree of match between the feature vector and the learned feature patterns of different importance levels. For example, a distance-based classification method is used. If the average distance between the feature vector of the image to be processed or classified and the feature vector of the image with a high importance level is less than a predetermined threshold, the image to be processed or classified is classified as a high importance level. At the same time, a multi-feature fusion method is used, such as further classifying the importance level of the feature vector after fusion of color features, texture features, shape features, etc., to obtain a classification result. The classification result obtained by the above method can effectively improve the classification accuracy and reliability.

[0049] Taking the Gaussian kernel function as an example, its expression is: ,in, and y is the eigenvector, is a parameter, is the Euclidean distance. After selecting the Gaussian kernel function, the parameters are determined by cross-validation. The value of . The above labeled data is trained using a classifier with a Gaussian kernel function to learn the characteristic patterns of images with different importance levels in a high-dimensional space.

[0050] In the classification process, the fused feature vector is extracted from the processed image. The distance between it and the feature patterns of different importance levels is calculated, such as the average distance to the feature vector of the high importance level image. If the distance is less than a predetermined threshold, it is classified as a high importance level; if the distance is greater than or equal to the predetermined threshold, it is classified as a medium importance level or a low importance level, and is further classified as a medium or low importance level according to the distance range.

[0051] It should be noted that the Gaussian kernel function has many advantages. First, it can be mapped to an infinite-dimensional space, effectively handle nonlinear separability, and improve the classification accuracy of the importance level of images in surveillance videos. Second, it has locality, with high weights for nearby points and rapidly reduced weights for distant points, adapting to local features of the image. In key area images, it can better focus on key local features. Finally, it is also effective for images that fuse feature vectors such as color, texture, and shape, better handle complex features in high-dimensional space, and improve classification accuracy and reliability.

[0052] By combining methods such as the Gaussian kernel function to classify image importance levels, the accuracy and reliability of classification can be effectively improved, meeting the needs of image classification in scenarios such as surveillance videos.

[0053] Next, in step S103, the initial compression ratios of different image regions in the image to be processed are determined according to the classification result.

[0054] For different types of regions, a variety of feature fusion methods are used. Taking the weighted fusion of texture, edge and color features as an example, first, the eigenvalues ​​are calculated. When calculating the texture eigenvalue, the image is converted to a grayscale image. For a given offset , count the grayscale values ​​in the image to be processed The pixel point is offset in the horizontal direction and vertical offset The gray value is The frequency of simultaneous occurrence of pixels, thus obtaining the gray-level co-occurrence matrix , then, from the gray-level co-occurrence matrix Feature statistics such as contrast correlation are extracted to characterize texture feature values.

[0055] Specifically, the following formula is used to calculate the frequency of the corresponding gray value combination in the gray level co-occurrence matrix:

[0056] S= , ;

[0057] Where S represents the gray level co-occurrence matrix The frequency of the combination of pixels with gray value i and pixels with gray value j appearing at the same time, It represents the probability of the combination of gray values ​​i and j in the gray-level co-occurrence matrix, where i is a positive integer and j is a positive integer.

[0058] The contrast between pixels is calculated using the following expression:

[0059] D= ;

[0060] Where D represents the contrast between the pixel with gray value i and the pixel with gray value j. It represents the probability of the combination of gray values ​​i and j in the gray-level co-occurrence matrix, where i is a positive integer and j is a positive integer.

[0061] When calculating the edge feature value of the feature to be processed, edge detection operators such as Sobel operator and Canny operator are used for pixel points. , its horizontal gradient approximation and the vertical gradient approximation The calculation design of convolution operation, then, through the formula The approximate value of the horizontal gradient of a pixel is calculated by convolution operations and other methods. The approximate value of the vertical gradient of the pixel point is also obtained by convolution operation and other methods) Calculate the gradient amplitude, and represent the edge feature value by counting the number of pixels whose gradient amplitude is greater than a specific threshold or calculating the average gradient amplitude. When calculating the color feature value, the image to be processed is converted to the HSV color space. For the hue, it is divided into several intervals, and the number of pixels in each interval is counted as the feature value. For saturation and brightness, the mean, variance and other statistics are calculated as the feature value.

[0062] Next, perform fusion calculation, assuming that the texture feature value is , edge eigenvalue , the color feature value is , and the texture feature value has been determined to be , edge eigenvalue , the color feature value is The corresponding weights are , , (satisfy , Greater than , Greater than ), the fused eigenvalues The formula Calculated, and normalized during the calculation process to map each eigenvalue to interval.

[0063] Preferably, to determine the optimal weight combination, firstly, different weight combinations are selected through experiments, for example, a grid search method is used to set the step size of the weights (such as Starting from 0.1 and increasing in steps of 0.1 to 0.9, and For each weight combination, multiple groups of sample images (i.e., multiple groups of images to be processed) are fused and classified, and evaluated using indicators such as accuracy and recall. By comparing the evaluation indicator results under different weight combinations, the weight combination with the highest accuracy or the best comprehensive evaluation indicator is selected as the optimal weight combination.

[0064] The following expression is used to calculate the accuracy: ;

[0065] in, Indicates the accuracy under a certain weight combination; For a real example, For a true counterexample, For a false positive example, A false counterexample.

[0066] The recall is calculated using the following expression: in, Represents the recall rate under a certain weight combination; For a real example, For a true counterexample, For a false positive example, A false counterexample.

[0067] The initial compression parameters of different regions are determined according to the classification results. The initial compression ratio of the complex texture region is relatively low. Specifically, the initial compression ratio of the complex texture region is in the range of 1:1 to 1:5. For example, for some image regions with extremely high image quality requirements, rich texture details and need to be presented clearly, a compression ratio close to 1:1 (i.e. no compression) or a compression ratio in the range of 1:1 to 1:3 can be used to retain more detail information. The initial compression ratio of the simple texture region is relatively high. Specifically, the initial compression ratio of the simple texture region is in the range of 1:5 to 1:20. For example, some large areas with a single color and little texture change, such as empty parking lots, white corridor walls, etc., can be compressed with a higher compression ratio, thereby saving storage space and transmission bandwidth.

[0068] Next, in step S104, for commercial and enterprise video surveillance, a wavelet transform-based image decomposition method is used to decompose the video image, i.e., the image to be processed, into sub-images of different scales. Wavelet transform can divide the image to be processed into a low-frequency part and a high-frequency part, the low-frequency part contains the main contour information and structural information, and the high-frequency part is the detail information and texture information.

[0069] It is very important to choose the appropriate wavelet basis function and decomposition level. Common wavelet basis functions such as Haar wavelet and Daubechies wavelet can be used, or custom wavelet basis functions can be designed according to the characteristics of the image.

[0070] Assume that the image to be processed has obvious directional texture, such as the texture of mechanical parts arranged regularly in a factory is mainly horizontal. Design a directional wavelet basis function. Suppose the two-dimensional wavelet basis function is , which can be obtained by two one-dimensional functions and The product of , and in order to reflect the directionality, one of the functions is stretched or compressed to obtain the transformed wavelet change function.

[0071] ;

[0072] in, Represents two one-dimensional functions and The transformation value obtained by stretching or compressing any function in , x represents the variable in the horizontal direction, y represents the variable in the vertical direction, and the vertical direction is the direction perpendicular to the horizontal direction; is a tuning parameter. When , the horizontal sensitivity can be enhanced; when , the horizontal sensitivity can be reduced. It is a compactly supported, oscillatory function used to construct the wavelet basis function :

[0073] ;

[0074] in, Indicates the one-dimensional function of the two-dimensional wavelet basis function constructed by the user; x indicates The value in the horizontal direction; Indicates the control oscillation frequency, represents the width of the influence function, Used to determine the support space of a function.

[0075] It should be noted that for Similar design principles can be used, and adjustments can also be made based on the image's vertical characteristics. If the image changes relatively little in the vertical direction, Relatively smooth, for example, a Gaussian function is selected. The above is only described as an optional example and cannot be understood as a limitation of the present invention.

[0076] The number of decomposition layers needs to be adjusted according to the image resolution and compression ratio. Specifically, the optimal number of decomposition layers can be determined by conducting multiple tests on surveillance videos of shopping malls, factories, etc. with different resolutions and observing the compression ratio and image quality indicators. For example, an initial number of decomposition layers can be determined first. For medium-resolution images, the test can be started from 2 or 3 layers. Then, the compression ratio and image quality indicators at different numbers of decomposition layers are calculated. The compression ratio can be calculated by the ratio of the original image data size to the compressed image data size of the image to be processed. The image quality indicators can be peak signal-to-noise ratio (PSNR) and / or structural similarity index (SSIM), etc. See Figure 2 The calculation formula for peak signal-to-noise ratio (PSNR) is:

[0077] ;

[0078] in, represents the peak signal-to-noise ratio of the image to be processed, so as to characterize the image quality index of the image to be processed; is the maximum pixel value of the image to be processed (for example, for an 8-bit grayscale image, ), is the mean square error of the image to be processed, and the calculation formula is:

[0079] ;

[0080] in, Represents the mean square error of the image to be processed; and (i, j) are the original image and compressed image of the image to be processed, i represents the i-th row, and j represents the j-th column; and are the number of rows and columns of the image to be processed respectively. The calculation formula of the structural similarity index (SSIM) of the image to be processed is:

[0081] ;

[0082] in, Represents the structural similarity index of the image to be processed; and The images to be processed are and The mean of x represents the original image to be processed, and y represents the compressed image of the image to be processed. and The images to be processed are and The variance of Is the image to be processed and The covariance of and A constant set to avoid the denominator being zero. Based on the experimental results, the relationship between the number of decomposition layers, compression ratio and image quality is analyzed, and the compression ratio is adjusted according to the following principles: if the compression ratio increases significantly with the increase in the number of decomposition layers, but the image quality does not decrease significantly, then the number of decomposition layers can be considered to be increased; if the image quality decreases too quickly, the number of decomposition layers should be reduced. Through multiple tests and adjustments, the optimal number of decomposition layers with relatively good image quality while meeting the compression ratio requirements is determined.

[0083] Next, the image to be processed is decomposed according to the determined optimal decomposition layer number to obtain multiple sub-images of different scales.

[0084] Next, in step S105, according to the scale and content characteristics of the sub-images of the image to be processed, different compression parameters and encoding modes are determined for the multiple sub-images obtained in the above steps according to the scale and content characteristics of each sub-image.

[0085] For the multiple sub-images obtained in the above steps, different compression parameters and encoding modes are determined according to the scale and content characteristics of each sub-image.

[0086] It should be noted that for each sub-image, the large-scale sub-image (it is recommended to increase the scale range) has less information, and a low resolution and simple encoding mode are used to improve the compression ratio; the small-scale sub-image (it is recommended to increase the scale range) has more information, and a high resolution and fine encoding mode are used to ensure image quality.

[0087] Specifically, for example, according to video coding standards such as H.264 / AVC and H.265 / HEVC, an advanced coding mode is used for small-scale sub-images with complex textures to improve image quality, and a baseline coding mode is used for large-scale sub-images with simple textures to improve compression ratio.

[0088] During the compression process, the image quality is monitored in real time and evaluated by calculating indicators such as the peak signal-to-noise ratio (PSNR) or the structural similarity index (SSIM). If the quality is lower than the preset threshold, the compression parameters are adjusted.

[0089] For example, parallel computing and hardware acceleration technologies are used to improve compression efficiency. For example, GPU acceleration can be used in large shopping mall video surveillance systems to shorten compression time. Inverse wavelet transform and other methods are used to reconstruct images during decoding. Optimized inverse wavelet transform algorithms such as biorthogonal wavelet transform and lifting wavelet transform are used to improve image reconstruction quality. Error compensation technology is used to reduce reconstruction errors.

[0090] In commercial and enterprise video surveillance systems, storage media can be selected according to actual needs. Hard disks are suitable for large-scale long-term storage, such as factory surveillance videos; optical disks are easy to carry and can be used to back up important data; network storage can achieve distributed storage and is suitable for surveillance systems in multiple locations. For large-scale systems, distributed storage technology is used to ensure data security and reliability through redundant backup and data recovery mechanisms, and load balancing is achieved to improve performance and efficiency.

[0091] The transmission method can be selected according to actual needs. Ethernet is suitable for fixed-location monitoring systems, with fast speed, high reliability and low cost; Wi-Fi is convenient and fast, suitable for temporary or less mobile scenarios; 4G / 5G is suitable for mobile video monitoring. Data compression, error correction coding and streaming media technology are used to improve transmission efficiency and reliability, and achieve high-quality real-time monitoring under limited network bandwidth.

[0092] In commercial and enterprise video surveillance, the video is collected and processed using the surveillance video compression processing method of the present invention. The compression parameters are adaptively adjusted according to different scenes and content characteristics to achieve the best compression effect and image quality. The compressed code stream can be stored in a hard disk, optical disk or network storage, or sent to a client via network transmission or wireless transmission for real-time playback or later viewing.

[0093] In terms of algorithm optimization and improvement, we continue to collect more commercial and enterprise surveillance video data, train and optimize the algorithms, and continuously improve compression efficiency and image quality to further improve accuracy and robustness.

[0094] Compared with the prior art, the present invention uses adaptive compression and multi-scale compression technology based on image features to adaptively adjust compression parameters according to the characteristics of video content, and decomposes video images into sub-images of different scales for independent compression. Without significantly reducing image quality, the compression ratio is greatly improved, and the storage space and transmission bandwidth requirements are reduced, effectively meeting the actual needs of commercial and enterprise video surveillance management for the storage and transmission of large amounts of high-definition video data. Specifically, an image feature extraction method based on deep learning is adopted to accurately extract image features, combine multiple feature information to classify image regions, and adjust compression parameters according to different region types. At the same time, in the process of sub-image compression and image reconstruction, an optimized coding mode and inverse wavelet transform algorithm are adopted, combined with error compensation technology, which effectively improves the reconstruction quality of the image, reduces image quality loss, and effectively ensures the effect and reliability of video surveillance.

[0095] Example 2

[0096] The following are embodiments of the device of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.

[0097] Figure 3 1 is a schematic diagram of an example of a monitoring video compression processing device according to the present invention. Figure 3 , a monitoring video compression processing device is described. The monitoring video compression processing device is used to execute the monitoring video compression processing method described in the first aspect of the present invention.

[0098] like Figure 3 As shown, the monitoring video compression processing device 300 includes an extraction module 310 , a classification module 420 , a first determination module 330 , a second determination module 340 and a third determination module 350 .

[0099] In a specific embodiment, the extraction module 310 is used to extract the video features of the image to be processed in the surveillance video of the target scene area using a video feature extraction model, and obtain the feature vectors of multiple video features in different image areas. The classification module 320 classifies the image to be processed as follows based on the feature vectors of multiple video features in different image areas to obtain classification results: image area classification, scene type classification, and importance level classification. The first determination module 330 determines the initial compression ratio of different image areas in the image to be processed based on the classification results. The second determination module 340 uses an image decomposition method based on wavelet transform to decompose the image to be processed into sub-images of different scales, and determines the optimal decomposition layer number based on the image quality. The third determination module 350 determines the final compression ratio and encoding mode of different image areas based on the scale and content characteristics of the sub-images of the image to be processed.

[0100] According to an optional embodiment, the optimal number of decomposition levels is determined according to the image quality.

[0101] Specifically, the compression ratio and image quality index at different decomposition levels are calculated, and the average value of the peak signal-to-noise ratio and the structural similarity index is adopted.

[0102] Calculate the compression ratio and image quality index under different decomposition levels. The image quality index is expressed by peak signal-to-noise ratio. The following expression is used to calculate the peak signal-to-noise ratio:

[0103] ;

[0104] in, represents the peak signal-to-noise ratio of the image to be processed, so as to characterize the image quality index of the image to be processed; is the maximum pixel value of the image to be processed, is the mean square error of the image to be processed.

[0105] The compression ratio and image quality index under different decomposition levels are calculated. The image quality index is represented by the structural similarity index. The following expression is used to calculate the structural similarity index of the image to be processed:

[0106] ;

[0107] in, Represents the structural similarity index of the image to be processed; and The images to be processed are and The mean of x represents the original image to be processed, and y represents the compressed image of the image to be processed. and The images to be processed are and The variance of Is the image to be processed and The covariance of and A constant used to avoid zero denominator.

[0108] A distance-based classification method is adopted. If the average distance between the feature vector of the image to be processed and the feature vector of the image with a high importance level is less than a predetermined threshold, the image to be processed or the image to be classified is classified as a high importance level. At the same time, a multi-feature fusion method is used to further classify the importance level of the feature vector calculated by fusing the color features, texture features, and shape features to obtain the classification result.

[0109] When the texture feature value is , edge eigenvalue , the color feature value is , and the texture feature value has been determined to be , edge eigenvalue , the color feature value is The corresponding first weight, second weight, and third weight are , , When , the eigenvalues ​​after fusion calculation The formula Calculated, and normalized during the calculation process to map each eigenvalue to Interval, where the first weight , second weight , the third weight Satisfies the following expression:

[0110] ;

[0111] in, Greater than , Greater than .

[0112] The initial compression parameters of different regions are determined according to the classification results, wherein the initial compression ratio of the region with complex texture is in the range of 1:1 to 1:5, and the initial compression ratio of the region with simple texture is in the range of 1:5 to 1:20.

[0113] According to an optional implementation manner, the image decomposition method based on wavelet transform is used to decompose the image to be processed into sub-images of different scales, including:

[0114] Through two one-dimensional functions and And in order to reflect the directionality, one of the functions is stretched or compressed to obtain the transformed wavelet change function:

[0115] ;

[0116] in, Represents two one-dimensional functions and The transformation value obtained by stretching or compressing any function in , x represents the variable in the horizontal direction, y represents the variable in the vertical direction, and the vertical direction is the direction perpendicular to the horizontal direction; is a tuning parameter. When , the horizontal sensitivity can be enhanced; when When the horizontal sensitivity is reduced, It is a compactly supported, oscillatory function used to construct the wavelet basis function :

[0117] ;

[0118] in, Indicates the one-dimensional function of the two-dimensional wavelet basis function constructed by the user; x indicates The value in the horizontal direction; Indicates the control oscillation frequency, represents the width of the influence function, Used to determine the support space of a function.

[0119] It should be noted that due to Figure 3 Monitoring video compression processing method and monitoring video compression processing method performed by monitoring video compression processing device Figure 1 The surveillance video compression processing methods in the examples are roughly the same, so the description of the same parts is omitted.

[0120] Compared with the prior art, the present invention uses adaptive compression and multi-scale compression technology based on image features to adaptively adjust compression parameters according to the characteristics of video content, and decomposes video images into sub-images of different scales for independent compression. Without significantly reducing image quality, the compression ratio is greatly improved, and the storage space and transmission bandwidth requirements are reduced, effectively meeting the actual needs of commercial and enterprise video surveillance management for the storage and transmission of large amounts of high-definition video data. Specifically, an image feature extraction method based on deep learning is adopted to accurately extract image features, combine multiple feature information to classify image regions, and adjust compression parameters according to different region types. At the same time, in the process of sub-image compression and image reconstruction, an optimized coding mode and inverse wavelet transform algorithm are adopted, combined with error compensation technology, which effectively improves the reconstruction quality of the image, reduces image quality loss, and effectively ensures the effect and reliability of video surveillance.

[0121] Figure 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0122] like Figure 4 As shown, the electronic device is presented in the form of a general computing device. The processor may be one or more and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0123] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0124] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0125] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0126] It should be understood that Figure 4 The electronic device shown is only an example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0127] Through the above description of the implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by combining software with necessary hardware. Figure 5 As shown, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of commands to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present invention.

[0128] The software product may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0129] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0130] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0131] The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the computer-readable medium implements the data interaction method of the present disclosure.

[0132] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0133] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the exemplary embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several commands to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

[0134] It should be noted that the above detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present application belongs.

[0135] In the above detailed description, reference is made to the accompanying drawings, which form a part of this document. In the accompanying drawings, similar symbols typically identify similar components unless the context indicates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A monitoring video compression processing method, characterized in that: The monitoring video compression processing method comprises: The video feature extraction model is used to extract the video features of the image to be processed in the surveillance video of the target scene area, and the feature vectors of multiple video features in different image areas are obtained; According to the obtained feature vectors of multiple video features in different image regions, the image to be processed is classified as follows to obtain classification results: image region classification, scene type classification, and importance level classification; Determining initial compression ratios of different image regions in the image to be processed according to the classification result; Using an image decomposition method based on wavelet transform, the image to be processed is decomposed into sub-images of different scales, and the optimal number of decomposition layers is determined according to the image quality; According to the scale and content characteristics of the sub-images of the image to be processed, the final compression ratio and encoding mode of different image regions are determined; wherein, Through two one-dimensional functions and And in order to reflect the directionality, one of the functions is stretched or compressed to obtain the transformed wavelet change function: ； in, Represents two one-dimensional functions and The transformation value obtained by stretching or compressing any function in , x represents the variable in the horizontal direction, y represents the variable in the vertical direction, and the vertical direction is the direction perpendicular to the horizontal direction; is a tuning parameter. When , the horizontal sensitivity can be enhanced; when When the horizontal sensitivity is reduced, It is a compactly supported, oscillatory function used to construct the wavelet basis function : ； in, Indicates the one-dimensional function of the two-dimensional wavelet basis function constructed by the user; x indicates The value in the horizontal direction; Indicates the control oscillation frequency, represents the width of the influence function, Used to determine the support space of a function.

2. The monitoring video compression processing method according to claim 1, characterized in that: Determining the optimal number of decomposition layers according to the image quality includes: The compression ratio and image quality index at different decomposition levels are calculated, and the average value of peak signal-to-noise ratio and structural similarity index is adopted.

3. The monitoring video compression processing method according to claim 1, characterized in that: Determining the optimal number of decomposition layers according to the image quality includes: Calculate the compression ratio and image quality index under different decomposition levels. The image quality index is expressed by peak signal-to-noise ratio. The following expression is used to calculate the peak signal-to-noise ratio: ； in, represents the peak signal-to-noise ratio of the image to be processed, so as to characterize the image quality index of the image to be processed; is the maximum pixel value of the image to be processed, is the mean square error of the image to be processed.

4. The monitoring video compression processing method according to claim 1, characterized in that: Determining the optimal number of decomposition layers according to the image quality includes: The compression ratio and image quality index under different decomposition levels are calculated. The image quality index is represented by the structural similarity index. The following expression is used to calculate the structural similarity index of the image to be processed: ； in, Represents the structural similarity index of the image to be processed; and The images to be processed are and The mean of x represents the original image to be processed, and y represents the compressed image of the image to be processed. and The images to be processed are and The variance of Is the image to be processed and The covariance of and A constant used to avoid zero denominator.

5. The monitoring video compression processing method according to claim 1, characterized in that: include: A distance-based classification method is adopted. If the average distance between the feature vector of the image to be processed and the feature vector of the image with a high importance level is less than a predetermined threshold, the image to be processed or the image to be classified is classified as a high importance level. At the same time, a multi-feature fusion method is used to further classify the importance level of the feature vector calculated by fusing the color features, texture features, and shape features to obtain the classification result.

6. The monitoring video compression processing method according to claim 3, characterized in that: Further including: When the texture feature value is , edge eigenvalue , the color feature value is , and the texture feature value has been determined to be , edge eigenvalue , the color feature value is The corresponding first weight, second weight, and third weight are , , When , the eigenvalues ​​after fusion calculation The formula Calculated, and normalized during the calculation process to map each eigenvalue to Interval, where the first weight , second weight , the third weight Satisfies the following expression: ； in, Greater than , Greater than .

7. The monitoring video compression processing method according to claim 1, characterized in that: include: The initial compression parameters of different regions are determined according to the classification results, wherein the initial compression ratio of the region with complex texture is in the range of 1:1 to 1:5, and the initial compression ratio of the region with simple texture is in the range of 1:5 to 1:

20.

8. A monitoring video compression processing device, characterized in that: The monitoring video compression processing method according to any one of claims 1 to 7 is implemented, and the monitoring video compression processing device comprises: An extraction module is used to extract video features of the image to be processed in the surveillance video of the target scene area by using a video feature extraction model, and obtain feature vectors of multiple video features in different image areas; A classification module, which classifies the image to be processed according to the feature vectors of various video features in different image regions to obtain classification results: image region classification, scene type classification, and importance level classification; A first determination module determines, according to the classification result, initial compression ratios of different image regions in the image to be processed; The second determination module uses an image decomposition method based on wavelet transform to decompose the image to be processed into sub-images of different scales, and determines the optimal decomposition layer number according to the image quality; The third determination module determines the final compression ratio and encoding mode of different image regions according to the scale and content characteristics of the sub-images of the image to be processed.

9. The monitoring video compression processing device according to claim 8, characterized in that: include: The compression ratio and image quality index under different decomposition levels are calculated, and the average value of the peak signal-to-noise ratio and the structural similarity index or the weighted average value of the peak signal-to-noise ratio and the structural similarity index is used to represent the image quality index.

Images