Rapid coding optimization method and system for monitoring video of power system

Abstract

The invention relates to the technical field of video coding, in particular to a power system monitoring video rapid coding optimization method and system, and the method comprises the steps: extracting a plurality of maximum coding units of a power system monitoring video frame; a quantization parameter is introduced, and a deep learning-based adaptive multi-scale space prediction network is used to predict the maximum coding units to obtain an optimal sub-coding block division mode of each maximum coding unit; dividing the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be coded; performing mode prediction on the sub-coding block by using a texture sensing network to obtain an optimal intra-frame coding mode of the sub-coding block; and coding the sub-coding blocks of the monitoring video of the power system according to the optimal intra-frame coding mode to obtain coded video stream data. According to the method, the intra-frame prediction process is optimized by combining the adaptive multi-scale space prediction network and the texture sensing network, the coding complexity is reduced, and the video coding speed is remarkably improved while the image quality is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of video encoding, and in particular to a method and system for optimizing fast encoding of power system monitoring videos. Background Art

[0002] As the infrastructure of modern society, the importance of safe and stable operation of power system is becoming increasingly prominent. Power system monitoring video is a key means to ensure the stability and safety of power system operation. The real-time transmission and image quality of power monitoring video are crucial for rapid response to abnormal conditions in the power grid, so that operation and maintenance personnel can ensure timely discovery and resolution of potential power grid abnormalities, such as equipment overheating, line aging or short circuit and other potential defects, and effectively prevent line accidents. Therefore, high image quality requirements are placed on power monitoring video. However, with the rapid increase in the amount of power system monitoring video data, the existing video compression technology can no longer meet the growing demand for high quality of power system monitoring video, especially in real-time encoding compression.

[0003] Versatile Video Coding (VVC) is the latest generation of video coding standard. Compared with High Efficiency Video Coding (HEVC), it can provide about 50% bit rate savings at the same video quality. However, Versatile Video Coding technology also brings higher coding complexity, especially in the intra-frame prediction part. Intra-frame prediction is a key step in video coding, which directly affects the coding efficiency and video quality. In Versatile Video Coding, intra-frame prediction introduces technologies such as Quadtree with Nested Multi-type Tree (QTMT), Multiple Reference Line (MRL) and Intra Sub-Partitions (ISP), which expands the number of intra-frame mode selections from 35 in High Efficiency Video Coding (HEVC) to 67.

[0004] Although the current multifunctional video coding technology has improved the compression performance of coding and can provide about 50% bit rate savings, it also brings about a higher coding calculation complexity, especially in the block division and mode selection process. This high complexity is not conducive to the real-time transmission of monitoring video, especially in power scenarios that require rapid response. Although the traditional fast algorithms based on statistical features and fast algorithms based on machine learning have achieved certain results in the block division and mode selection process, there are still many shortcomings. The algorithm based on statistical features relies on the characteristics and probability statistical information of the coding unit itself, but the accuracy of the prediction is limited by the rationality of the feature design, and the existing deep learning optimization method is not good in block division optimization. The QTMT partitioning structure of VVC adds a multi-tree partitioning mode, which results in the generation of coding blocks of different sizes during the partitioning process, increasing the difficulty of model prediction. In addition, in terms of mode selection optimization, the number of intra-frame mode selections in VVC has increased significantly, and the existing model only predicts based on the characteristics of the current coding block itself, ignoring the influence of adjacent coded blocks on the intra-frame mode of the current block, resulting in low prediction accuracy and reduced quality after video encoding, which cannot meet the requirements of current power monitoring videos. Therefore, how to effectively reduce the coding complexity of intra-frame prediction while ensuring the coding quality of power monitoring videos and ensure the safe and stable operation of the power system has become an important issue that needs to be urgently solved in the current power system monitoring field. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for optimizing fast encoding of monitoring videos of a power system.

[0006] In a first aspect, the present invention provides a method for optimizing fast encoding of power system monitoring video, the method comprising the following steps:

[0007] Preprocessing the received power system monitoring video frame to extract multiple maximum coding units of the power system monitoring video frame;

[0008] Introducing a quantization parameter, and using an adaptive multi-scale spatial prediction network based on deep learning to adaptively partition and predict the maximum coding unit, to obtain an optimal sub-coding block partitioning mode for each maximum coding unit;

[0009] Dividing the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded;

[0010] Using a texture perception network to perform mode prediction on the sub-coding block to obtain an optimal intra-frame coding mode of the sub-coding block;

[0011] The sub-coding blocks of the power system monitoring video are encoded according to the optimal intra-frame encoding mode to obtain encoded video stream data.

[0012] In a further embodiment, the step of introducing a quantization parameter, using an adaptive multi-scale spatial prediction network based on deep learning to adaptively partition and predict the maximum coding unit, and obtaining an optimal sub-coding block partitioning mode for each maximum coding unit includes:

[0013] The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information;

[0014] Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information;

[0015] Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result;

[0016] The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

[0017] In a further embodiment, the texture extraction module comprises a plurality of texture extraction units in series; the texture extraction unit comprises a first convolutional layer, a visual attention network module and a pooling layer;

[0018] The visual attention network module includes a second convolutional layer, an activation layer, a depthwise separable convolutional layer, a depthwise dilated convolutional layer, a third convolutional layer, a fourth convolutional layer and a feedforward neural network module connected in sequence; wherein the input of the visual attention network module forms a residual connection with the output of the fourth convolutional layer; and the output of the activation layer forms a residual connection with the output of the third convolutional layer.

[0019] In a further embodiment, the step of using a texture perception network to perform mode prediction on the sub-coding block to obtain the optimal intra-frame coding mode of the sub-coding block includes:

[0020] The sub-coding block is subjected to wavelet transform by a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block;

[0021] Using multiple cascaded lightweight self-attention modules to extract texture features from the shallow detail feature map of the sub-coding block to obtain sub-coding block texture features;

[0022] The sub-coding block quantization parameter information and the intra-frame prediction mode of the adjacent coded blocks of the sub-coding block are obtained, and the optimal intra-frame coding mode of the sub-coding block is predicted according to the texture feature of the sub-coding block, the quantization parameter information of the sub-coding block and the intra-frame prediction mode of the adjacent coded blocks.

[0023] In a further embodiment, the step of performing wavelet transform on the sub-coding block through a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block comprises:

[0024] Performing wavelet transform on the sub-coding block to decompose and obtain a sub-image set, wherein the sub-image set includes a low-frequency sub-image, a horizontal high-frequency sub-image, a vertical high-frequency sub-image, and a diagonal high-frequency sub-image;

[0025] The sub-image set is subjected to feature extraction using a deep separable convolutional network to obtain high- and low-frequency feature maps; the high- and low-frequency feature maps include horizontal high-frequency feature maps, vertical high-frequency feature maps, diagonal high-frequency feature maps, and low-frequency feature maps;

[0026] Add the horizontal high-frequency feature map, the vertical high-frequency feature map, and the diagonal high-frequency feature map to obtain a high-frequency feature sum image;

[0027] The low-frequency feature map and the high-frequency feature sum image are merged on the channel to obtain the sub-coding block feature map;

[0028] A deep separable convolutional network is used to extract shallow detail features from the sub-coding block feature map to obtain the shallow detail feature map of the sub-coding block.

[0029] In a further embodiment, the lightweight self-attention module includes a serial deep residual module, a hierarchical residual module, a self-attention module, and a deep separable convolutional network;

[0030] The deep residual module includes a serial deep residual block, a batch normalization layer and a global linear unit activation layer; the hierarchical residual module includes a batch normalization layer, a residual normalization module and a weighted batch normalization module connected in parallel.

[0031] In a further implementation manner, the intra-frame prediction mode of the adjacent coded block is an optimal intra-frame prediction mode adopted by the sub-coding block that is adjacent to the sub-coding block and has been coded.

[0032] In a second aspect, the present invention provides a system for optimizing fast encoding of monitoring videos of a power system, the system comprising:

[0033] A coding block extraction module is used to pre-process the received power system monitoring video frame and extract multiple maximum coding units of the power system monitoring video frame;

[0034] An optimal partition prediction module, used for introducing quantization parameters, using an adaptive multi-scale spatial prediction network based on deep learning to perform adaptive partition prediction on the maximum coding unit, and obtaining an optimal sub-coding block partition mode for each maximum coding unit;

[0035] A coding block division module, used to divide the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded;

[0036] An optimal coding prediction module, used to perform mode prediction on the sub-coding block using a texture perception network to obtain an optimal intra-frame coding mode of the sub-coding block;

[0037] The video encoding module is used to encode the sub-coding blocks of the power system monitoring video according to the optimal intra-frame encoding mode to obtain encoded video stream data.

[0038] In a further embodiment, the optimal partition prediction module is specifically used to:

[0039] The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information;

[0040] Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information;

[0041] Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result;

[0042] The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

[0043] In a further embodiment, the texture extraction module comprises a plurality of texture extraction units in series; the texture extraction unit comprises a first convolutional layer, a visual attention network module and a pooling layer;

[0044] The visual attention network module includes a second convolutional layer, an activation layer, a depthwise separable convolutional layer, a depthwise dilated convolutional layer, a third convolutional layer, a fourth convolutional layer and a feedforward neural network module connected in sequence; wherein the input of the visual attention network module forms a residual connection with the output of the fourth convolutional layer; and the output of the activation layer forms a residual connection with the output of the third convolutional layer.

[0045] The present invention provides a method and system for optimizing fast encoding of power system monitoring video. The method extracts multiple maximum coding units of the power system monitoring video frame by preprocessing the received power system monitoring video frame; introduces quantization parameters, uses an adaptive multi-scale spatial prediction network based on deep learning to adaptively divide and predict the maximum coding unit, and obtains the optimal sub-coding block division mode of each maximum coding unit; divides the maximum coding unit according to the optimal sub-coding block division mode to obtain the sub-coding block to be encoded; uses a texture perception network to perform mode prediction on the sub-coding block to obtain the optimal intra-frame coding mode of the sub-coding block; encodes the sub-coding block of the power system monitoring video according to the optimal intra-frame coding mode to obtain encoded video stream data. Compared with the prior art, the method saves a lot of encoding bit rate by combining an adaptive multi-scale spatial prediction network based on deep learning and a texture perception network to optimize the intra-frame prediction process, thereby reducing encoding complexity and improving compression efficiency, and significantly improving video encoding speed while ensuring image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the flow chart of the fast encoding optimization method for power system monitoring video provided by an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of an adaptive multi-scale spatial prediction network structure based on deep learning provided by an embodiment of the present invention;

[0048] Figure 3 is an example diagram of an adaptive partition prediction process provided by an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of a texture perception network structure provided by an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the structure of a wavelet transform preprocessing module provided by an embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of the structure of a lightweight self-attention module provided by an embodiment of the present invention;

[0052] Figure 7 The invention provides a block diagram of a fast encoding optimization system for monitoring video in a power system. DETAILED DESCRIPTION

[0053] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0054] refer to Figure 1 The embodiment of the present invention provides a method for optimizing fast encoding of monitoring video of a power system, such as Figure 1 As shown, the method comprises the following steps:

[0055] S1. Preprocess the received power system monitoring video frame and extract multiple maximum coding units of the power system monitoring video frame.

[0056] In order to improve the compression performance of coding, traditional video coding methods introduce technologies such as quadtree partitioning structures in the video coding process. Although these structures improve the coding efficiency, they also increase the coding time. In addition, since coding units (CU) of various sizes are generated, this increases the complexity in the construction of the data set and the prediction process of the model. For the time-consuming block partitioning process of the traditional method, this embodiment uses the texture information of the maximum coding unit to predict the partitioning mode of its sub-CU, thereby reducing the size diversity of CU in the data set. In addition, this embodiment also integrates quantization parameters (QP) to achieve adaptive QP partitioning prediction, which can terminate the partitioning process in advance and further improve the efficiency. Before predicting the partitioning mode, this embodiment uses sensor equipment or monitoring equipment to collect power system monitoring video frames in different environments in indoor and outdoor monitoring scenes of the power system, and extracts multiple maximum coding units (i.e., coding units of the largest size) from the power system monitoring video frames, so that these maximum coding units can be subsequently divided using an adaptive multi-scale spatial prediction network based on deep learning, and the partitioning mode of small coding unit blocks is predicted by the information of large CU (coding unit) to improve the coding speed and efficiency.

[0057] S2. Introduce quantization parameters, and use an adaptive multi-scale spatial prediction network based on deep learning to adaptively divide and predict the maximum coding unit to obtain the optimal sub-coding block division mode of each maximum coding unit.

[0058] In this embodiment, the step of introducing a quantization parameter, using an adaptive multi-scale spatial prediction network based on deep learning to adaptively partition and predict the maximum coding unit, and obtaining an optimal sub-coding block partitioning mode for each maximum coding unit includes:

[0059] The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information;

[0060] Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information;

[0061] Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result;

[0062] The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

[0063] For ease of understanding, this embodiment takes the example of dividing the maximum coding unit of 64×64 into sub-coding units of 16×16, 32×16, 16×32, etc. This embodiment selects 64×64 as the network input size because a block of this size contains the texture information of its sub-blocks, thereby allowing the texture information of the sub-block to be directly predicted by extracting the texture information of the parent block. In the adaptive multi-size partition network based on deep learning, In the AMSP-Net, the present embodiment uses the texture information of the maximum coding unit of 64×64 size to predict the division mode of its four smaller-sized (32×32, 16×16, 32×16, 16×32) sub-blocks. The method combines the division mode information extracted from the CU block of the previous layer and integrates the QP value information, which are jointly used to predict the optimal sub-coding block division mode of the current coding unit, thereby realizing the fast and accurate prediction of multi-sized block division in the monitoring video. Specifically, the present embodiment takes the maximum coding unit of 64×64 size as the target coding unit, extracts the brightness data of the target coding unit from the power system monitoring video frame, extracts the texture information according to the brightness data of the target coding unit through the texture extraction module in the AMSP-Net, obtains the coding block texture information, and then traverses multiple types of tree-like division methods, and determines the division probability of the target coding unit using each type of tree-like division method according to the coding block texture information. Traverse various types of tree partitioning methods (including quadtree QT, horizontal binary tree BTH, vertical binary tree BTV, etc.), and use the AMSP-Net model to predict the probability of the target coding unit using each type of tree partitioning method based on the extracted coding block texture information. According to these partitioning probabilities, determine the partitioning mode prediction result of the target coding unit, and then, according to the partitioning mode prediction result, divide the target coding unit into multiple sub-coding units (such as CUs of sizes such as 32x32 and 16x16), and use the multiple sub-coding units obtained by division as the target coding units of the next recursive partitioning stage. Repeat the above steps of feature extraction, partitioning probability prediction, partitioning mode determination and partitioning operation to predict the partitioning mode of coding units (CUs) of each size until the recursive end condition is met (such as the sub-coding unit reaches the minimum size or no longer needs to be further divided). Finally, generate the optimal sub-coding block partitioning mode, which will be used in the subsequent encoding process to improve encoding efficiency and video quality.

[0064] In this embodiment, if Figure 2As shown, the texture extraction module includes a plurality of texture extraction units in series; the texture extraction unit includes a first convolution layer Conv1*1, a visual attention network module VAN block and a pooling layer Pooling; the first convolution layers in the plurality of texture extraction units are all two-dimensional convolution layers, and the sizes of the convolution kernels thereof are 4x4 and 1x1 respectively; in this embodiment, the visual attention network module is composed of an attention module and a feedforward neural network module, and the attention module makes up for the deficiency that the ordinary convolution layer can only extract local information, and can effectively extract local and global texture features at the same time, thereby extracting rich global texture information in a 64×64 coding unit. Specifically, the visual attention network module includes a second convolution layer, an activation layer GELU, a depth-separable convolution layer DW-Conv3*3, a depth-expanded convolution layer DW-D-Conv3*3, a third convolution layer Conv1*1, a fourth convolution layer Conv1*1 and a feedforward neural network module FFN connected in sequence; wherein, the input of the visual attention network module forms a residual connection with the output of the fourth convolution layer; and the output of the activation layer forms a residual connection with the output of the third convolution layer.

[0065] It should be noted that in the recursive partition prediction stage, the partition prediction order follows the block partition order in the original encoder, that is, from large to small, and the size of the sub-block is determined by the partition mode of the parent block. For example, the optimal partition mode of the 64×64CU is determined, and its branch includes outputting two nodes: quadtree (QT) partition and no partition. If the prediction is no partition, the partition of the current block is terminated in advance; if the prediction is QT partition, the optimal partition mode of four 32×32CUs is predicted based on the extracted parent CU texture information, and the branch outputs 24 nodes (each 6 nodes represents a 32×32CU). For each 32×32CU, if the prediction is no division, the division is terminated in advance; if the prediction is QT division, the other three 32×32CUs are masked (that is, the extracted feature map is divided into four regions according to QT, and the values ​​of other regions are set to 0 except for the corresponding region currently required for prediction), and the optimal division mode is predicted for the four 16×16CUs after the current division at the same time, and the branch outputs 24 nodes (each 6 nodes represents the prediction probability of each division mode of a 16×16CU); if the prediction is a horizontal ternary tree (Horizontal If vertical binary tree (TTH) partitioning is predicted, mask operation is performed on the other three 32×32CUs, and optimal partitioning mode prediction is performed on the two 32×16CUs after the current partitioning at the same time, and the branch outputs 12 nodes (every 6 nodes represent the prediction probability of each partitioning mode of a 32×16CU); if vertical binary tree (Vertical Binary Tree, BTV) partitioning is predicted, mask operation is performed on the other three 32×32CUs, and optimal partitioning mode prediction is performed on the two 16×32CUs after the current partitioning at the same time, and the branch outputs 12 nodes (every 6 nodes represent the prediction probability of each partitioning mode of a 16×32CU); if other partitioning modes are predicted, the partitioning operation is performed according to the original encoding process. In the recursive partitioning prediction process of sub-coding units of each size, the present embodiment introduces quantization parameter (QP) information, so that the network can adaptively perform partitioning prediction based on QP, thereby being able to efficiently predict the partitioning mode of the coding block, thereby improving the efficiency and performance of video coding.

[0066] exist Figure 2 Medium, F i,j represents the partitioning mode, these partitioning modes F i,jThe subscripts i and j are defined as follows: 0 represents no partitioning, 1 represents quadtree (QT) partitioning, 2 represents horizontal binary tree (BTH) partitioning, 3 represents vertical binary tree (BTV) partitioning, 4 represents horizontal ternary tree (TTH) partitioning, and 5 represents vertical ternary tree (TTV) partitioning.

[0067] like Figure 3 As shown, in the video encoding process, this embodiment first starts encoding and processes the maximum coding unit (CU) of 64×64 size, and then uses the AMSP-Net model to predict the division of the current maximum coding unit (CU) of 64×64 size to obtain the division mode prediction result. If the division mode prediction result is no division, the further division of the current 64×64CU is directly skipped, and the next 64×64CU is turned to for encoding; if the division mode prediction result is QT division, the AMSP-Net model is used to simultaneously predict the optimal division mode of four 32×32 sub-CUs. For these four 32×32 sub-CUs, the encoder will judge their division conditions one by one. If it is judged that these four 32×32 sub-CUs are not divided, the sub-CU is skipped and the next 32×32 sub-CU is judged; if it is judged that these four 32×32 sub-CUs are QT divided, the AMSP-Net model is used to perform division on the four 16×16 sub-CUs after division. The optimal partitioning mode is predicted. If it is determined that the four 16×16 sub-CUs are divided into BTH, the AMSP-Net model is used to predict the optimal partitioning mode of the two 32×16 sub-CUs after the division; if it is determined that the four 16×16 sub-CUs are divided into BTV, the AMSP-Net model is used to predict the optimal partitioning mode of the two 16×32 sub-CUs after the division; if it is determined that the four 16×16 sub-CUs are divided into other parts, the partitioning process of the original encoder is followed to determine whether the partitioning prediction of the four 32×32 sub-CUs is completed. If completed, turn to the next 64×64CU for encoding; if not completed, continue to predict the partitioning of the next 32×32 sub-CU. Through the above process, compared with the original encoder, the current method significantly speeds up the partitioning process of 64×64, 32×32, 16×16, 32×16 and 16×32 coding blocks, thereby greatly reducing the encoding complexity.

[0068] S3. Divide the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded.

[0069] S4. Use a texture perception network to perform mode prediction on the sub-coding block to obtain an optimal intra-frame coding mode of the sub-coding block.

[0070] In this embodiment, the step of using a texture perception network to perform mode prediction on the sub-coding block to obtain the optimal intra-frame coding mode of the sub-coding block includes:

[0071] The sub-coding block is subjected to wavelet transform by a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block;

[0072] Using multiple cascaded lightweight self-attention modules to extract texture features from the shallow detail feature map of the sub-coding block to obtain sub-coding block texture features;

[0073] The sub-coding block quantization parameter information and the adjacent coded block intra-frame prediction mode of the sub-coding block are obtained, and the optimal intra-frame coding mode of the sub-coding block is predicted according to the texture feature of the sub-coding block, the sub-coding block quantization parameter information and the adjacent coded block intra-frame prediction mode. In this embodiment, the adjacent coded block intra-frame prediction mode is the optimal intra-frame prediction mode adopted by the sub-coding block that is adjacent to the sub-coding block and has been encoded.

[0074] Specifically, this embodiment encodes the power system monitoring video based on the texture-aware network to obtain the optimal intra-frame coding mode of each power system monitoring video under a specific QP. During the optimization process of the encoder, this embodiment uses the texture-aware network to accelerate the selection of intra-frame prediction mode. The texture-aware network is used to extract the texture direction characteristics of the coding block, and integrate the quantization parameter (QP) information and the prediction mode information of the adjacent blocks to jointly guide the model to make predictions. In traditional encoders, determining the intra-frame prediction mode of the current block requires reference to the most probable mode (MPM) list of the adjacent blocks. In order to improve the prediction accuracy and simplify the selection process, this embodiment directly introduces the prediction mode information of the adjacent blocks into the texture-aware network, which not only improves the accuracy of the model's prediction of the optimal intra-frame prediction mode, but also enables the encoding process to skip the tedious mode selection steps, thereby effectively reducing the time required for encoding.

[0075] In this embodiment, the step of performing wavelet transform on the sub-coding block through a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block includes:

[0076] Performing wavelet transform on the sub-coding block to decompose and obtain a sub-image set, wherein the sub-image set includes a low-frequency sub-image, a horizontal high-frequency sub-image, a vertical high-frequency sub-image, and a diagonal high-frequency sub-image;

[0077] The sub-image set is subjected to feature extraction using a deep separable convolutional network to obtain high- and low-frequency feature maps; the high- and low-frequency feature maps include horizontal high-frequency feature maps, vertical high-frequency feature maps, diagonal high-frequency feature maps, and low-frequency feature maps;

[0078] Add the horizontal high-frequency feature map, the vertical high-frequency feature map, and the diagonal high-frequency feature map to obtain a high-frequency feature sum image;

[0079] The low-frequency feature map and the high-frequency feature sum image are merged on the channel to obtain the sub-coding block feature map;

[0080] A deep separable convolutional network is used to extract shallow detail features from the sub-coding block feature map to obtain the shallow detail feature map of the sub-coding block.

[0081] Specifically, Figure 4 As shown in Figure 1, the texture perception network includes a wavelet transform preprocessing module, a lightweight self-attention module, and a pattern prediction module connected in sequence. In the input stage, the grayscale image of the sub-coding block and a row of pixels adjacent to it above and to the left are taken as network inputs. Figure 5 As shown in the figure, sub-coding blocks of different sizes are processed by the Wavelet Transform Preprocessing module. Module, WTPM), the wavelet transform preprocessing module uses Haar wavelet transform, through a low-pass filter and three high-pass filters, to decompose the input feature map into four sub-images: low-frequency sub-image fLL, horizontal high-frequency sub-image fLH, vertical high-frequency sub-image fHL and diagonal high-frequency sub-image fHH, the length and width of these sub-images are half of the original image, then, each sub-image is feature extracted by DSConv (depthwise separable convolution) to obtain high- and low-frequency feature maps including horizontal high-frequency feature map, vertical high-frequency feature map, diagonal high-frequency feature map and low-frequency feature map; the horizontal high-frequency feature map, the vertical high-frequency feature map and the diagonal high-frequency feature map are added to obtain the high-frequency feature sum image; the low-frequency feature map and the high-frequency feature sum image are merged on the channel to obtain the sub-coding block feature map, and the sub-coding block feature map is subjected to feature fusion of the deep separable convolution network to extract the shallow detail features of the image and obtain the shallow detail feature map of the sub-coding block, which effectively reduces the loss of image information.

[0082] Then, this embodiment uses multiple cascaded lightweight self-attention modules (LSAM) to extract features from the shallow detail feature map of the sub-coding block, such as Figure 6As shown, in this embodiment, the lightweight self-attention module includes a serial deep residual module (DW+BN+GELU), a hierarchical residual module, a self-attention module and a deep separable convolutional network; the deep residual module includes a serial deep residual block DW, a batch normalization layer BN and a global linear unit activation layer GELU; the hierarchical residual module includes a batch normalization layer BN, a residual normalization module (DW+BN) and a weighted batch normalization module (PW+BN) connected in parallel, PW indicates that a weighted operation is performed before batch normalization. In the process of feature extraction using the lightweight self-attention module, LSAM first performs multi-scale feature extraction on the shallow detail feature map of the sub-coding block output by the wavelet transform preprocessing module, and then inputs the feature map after convolution of different scales into the self-attention module (Self-Attention, SA). The self-attention module can dynamically capture the relationship between the elements in the image, enhance the model's ability to extract contextual information, and thus extract more key information that is helpful for classification. The number of LSAMs can be flexibly selected according to the size of the input coding block.

[0083] Finally, in the mode prediction module, this embodiment flattens the extracted sub-coding block texture features into a feature vector through global average pooling or similar operations, and concatenates this feature vector with the perceptual quantization parameter (QP) and the prediction modes of the coding blocks above and to the left of the coding block. By comprehensively considering the texture feature information of the coding block, the perceptual QP information, and the prediction mode information of the adjacent coding blocks, the optimal intra-frame coding mode of the sub-coding block is jointly predicted.

[0084] S5. Encode the sub-coding blocks of the power system monitoring video according to the optimal intra-frame coding mode to obtain encoded video stream data.

[0085] For surveillance video compression technology in complex scenarios, this embodiment targets the time-consuming coding unit (CU) division process in video encoding, and reduces the time required for CU division by introducing an adaptive multi-scale spatial prediction network based on deep learning; at the same time, in terms of mode selection, this embodiment uses a texture-aware network, combined with the texture features of the coding block, quantization parameter (QP) information and prediction mode information of adjacent coding blocks to predict the optimal intra-frame coding mode. This method can skip redundant mode traversal steps and further speed up the video encoding process. This embodiment embeds the proposed adaptive multi-scale spatial prediction network based on deep learning and the texture-aware network into the original video encoder. This collaborative optimization method not only greatly accelerates the overall process of video encoding, but also ensures the quality of the compressed video, achieves significant optimization of the encoding process, and effectively reduces the encoding complexity.

[0086] The embodiment of the present invention provides a method for optimizing fast encoding of power system monitoring video, wherein the method extracts multiple maximum coding units of the power system monitoring video frame by preprocessing the received power system monitoring video frame; introduces quantization parameters, and uses an adaptive multi-scale spatial prediction network based on deep learning to adaptively divide and predict the maximum coding unit to obtain the optimal sub-coding block division mode of each maximum coding unit; divides the maximum coding unit according to the optimal sub-coding block division mode to obtain the sub-coding block to be encoded; uses a texture perception network to perform mode prediction on the sub-coding block to obtain the optimal intra-frame coding mode of the sub-coding block; and encodes the sub-coding block of the power system monitoring video according to the optimal intra-frame coding mode to obtain encoded video stream data. Compared with the prior art, the method saves a lot of encoding bit rate by combining an adaptive multi-scale spatial prediction network based on deep learning and a texture perception network to optimize the intra-frame prediction process, thereby reducing encoding complexity and improving compression efficiency, and significantly improving video encoding speed while ensuring image quality.

[0087] It should be noted that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0088] In one embodiment, Figure 7 As shown, an embodiment of the present invention provides a power system monitoring video fast encoding optimization system, the system comprising:

[0089] The coding block extraction module 101 is used to pre-process the received power system monitoring video frame and extract multiple maximum coding units of the power system monitoring video frame;

[0090] An optimal partition prediction module 102 is used to introduce a quantization parameter and use an adaptive multi-scale spatial prediction network based on deep learning to perform adaptive partition prediction on the maximum coding unit to obtain an optimal sub-coding block partition mode for each maximum coding unit;

[0091] A coding block division module 103, configured to divide the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded;

[0092] The optimal coding prediction module 104 is used to perform mode prediction on the sub-coding block by using a texture perception network to obtain an optimal intra-frame coding mode of the sub-coding block;

[0093] The video encoding module 105 is used to encode the sub-coding blocks of the power system monitoring video according to the optimal intra-frame coding mode to obtain encoded video stream data.

[0094] In this embodiment, the optimal partition prediction module 102 is specifically used for:

[0095] The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information;

[0096] Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information;

[0097] Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result;

[0098] The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

[0099] In this embodiment, the texture extraction module includes a plurality of texture extraction units in series; the texture extraction unit includes a first convolutional layer, a visual attention network module and a pooling layer;

[0100] The visual attention network module includes a second convolutional layer, an activation layer, a depthwise separable convolutional layer, a depthwise dilated convolutional layer, a third convolutional layer, a fourth convolutional layer and a feedforward neural network module connected in sequence; wherein the input of the visual attention network module forms a residual connection with the output of the fourth convolutional layer; and the output of the activation layer forms a residual connection with the output of the third convolutional layer.

[0101] For the specific definition of a power system monitoring video fast encoding optimization system, please refer to the above-mentioned definition of a power system monitoring video fast encoding optimization method, which will not be repeated here. A person of ordinary skill in the art can appreciate that the various modules and steps described in conjunction with the embodiments disclosed in this application can be implemented in hardware, software, or a combination of both. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] The embodiment of the present invention provides a fast coding optimization system for power system monitoring video. The system extracts multiple maximum coding units of the power system monitoring video frame by preprocessing the received power system monitoring video frame; introduces quantization parameters, uses an adaptive multi-scale spatial prediction network based on deep learning to adaptively divide and predict the maximum coding unit, and obtains the optimal sub-coding block division mode of each maximum coding unit; divides the maximum coding unit according to the optimal sub-coding block division mode to obtain the sub-coding block to be encoded; uses a texture perception network to perform mode prediction on the sub-coding block to obtain the optimal intra-frame coding mode of the sub-coding block; encodes the sub-coding block of the power system monitoring video according to the optimal intra-frame coding mode to obtain encoded video stream data. Compared with the prior art, the system saves a lot of encoding bit rate by combining the adaptive multi-scale spatial prediction network based on deep learning and the texture perception network to optimize the intra-frame prediction process, thereby reducing the encoding complexity and improving the compression efficiency, and significantly improving the video encoding speed while ensuring the image quality.

[0103] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A method for optimizing fast encoding of power system monitoring video, characterized in that: The following steps are involved: Preprocessing the received power system monitoring video frame to extract multiple maximum coding units of the power system monitoring video frame; Introducing a quantization parameter, and using an adaptive multi-scale spatial prediction network based on deep learning to adaptively partition and predict the maximum coding unit, to obtain an optimal sub-coding block partitioning mode for each maximum coding unit; Dividing the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded; Using a texture perception network to perform mode prediction on the sub-coding block to obtain an optimal intra-frame coding mode of the sub-coding block; The sub-coding blocks of the power system monitoring video are encoded according to the optimal intra-frame encoding mode to obtain encoded video stream data.

2. A method for optimizing fast encoding of power system monitoring video according to claim 1, characterized in that: The step of introducing a quantization parameter and using an adaptive multi-scale spatial prediction network based on deep learning to adaptively partition and predict the maximum coding unit to obtain an optimal sub-coding block partitioning mode for each maximum coding unit includes: The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information; Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information; Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result; The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

3. A method for optimizing fast encoding of power system monitoring video according to claim 2, characterized in that: The texture extraction module includes a plurality of texture extraction units in series; the texture extraction unit includes a first convolutional layer, a visual attention network module and a pooling layer; The visual attention network module includes a second convolutional layer, an activation layer, a depthwise separable convolutional layer, a depthwise dilated convolutional layer, a third convolutional layer, a fourth convolutional layer and a feedforward neural network module connected in sequence; wherein the input of the visual attention network module forms a residual connection with the output of the fourth convolutional layer; and the output of the activation layer forms a residual connection with the output of the third convolutional layer.

4. The method for optimizing fast encoding of power system monitoring video according to claim 1, characterized in that: The step of using a texture perception network to perform mode prediction on the sub-coding block to obtain an optimal intra-frame coding mode for the sub-coding block comprises: The sub-coding block is subjected to wavelet transform by a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block; Using multiple cascaded lightweight self-attention modules to extract texture features from the shallow detail feature map of the sub-coding block to obtain sub-coding block texture features; The sub-coding block quantization parameter information and the intra-frame prediction mode of the adjacent coded blocks of the sub-coding block are obtained, and the optimal intra-frame coding mode of the sub-coding block is predicted according to the texture feature of the sub-coding block, the quantization parameter information of the sub-coding block and the intra-frame prediction mode of the adjacent coded blocks.

5. A method for optimizing fast encoding of power system monitoring video according to claim 4, characterized in that: The step of performing wavelet transform on the sub-coding block through a wavelet transform preprocessing module to obtain a shallow detail feature map of the sub-coding block comprises: Performing wavelet transform on the sub-coding block to decompose and obtain a sub-image set, wherein the sub-image set includes a low-frequency sub-image, a horizontal high-frequency sub-image, a vertical high-frequency sub-image, and a diagonal high-frequency sub-image; The sub-image set is subjected to feature extraction using a deep separable convolutional network to obtain high- and low-frequency feature maps; the high- and low-frequency feature maps include horizontal high-frequency feature maps, vertical high-frequency feature maps, diagonal high-frequency feature maps, and low-frequency feature maps; Add the horizontal high-frequency feature map, the vertical high-frequency feature map, and the diagonal high-frequency feature map to obtain a high-frequency feature sum image; The low-frequency feature map and the high-frequency feature sum image are merged on the channel to obtain the sub-coding block feature map; A deep separable convolutional network is used to extract shallow detail features from the sub-coding block feature map to obtain the shallow detail feature map of the sub-coding block.

6. A method for optimizing fast encoding of power system monitoring video according to claim 4, characterized in that: The lightweight self-attention module includes a serial deep residual module, a hierarchical residual module, a self-attention module and a deep separable convolutional network; The deep residual module includes a serial deep residual block, a batch normalization layer and a global linear unit activation layer; the hierarchical residual module includes a batch normalization layer, a residual normalization module and a weighted batch normalization module connected in parallel.

7. A method for optimizing fast encoding of power system monitoring video according to claim 4, characterized in that: The adjacent encoded block intra-frame prediction mode is an optimal intra-frame prediction mode adopted by the sub-coding blocks that are adjacent to the sub-coding blocks and have been encoded.

8. A fast encoding optimization system for power system monitoring video, characterized in that: The system comprises: A coding block extraction module is used to pre-process the received power system monitoring video frame and extract multiple maximum coding units of the power system monitoring video frame; An optimal partition prediction module, used for introducing quantization parameters, using an adaptive multi-scale spatial prediction network based on deep learning to perform adaptive partition prediction on the maximum coding unit, and obtaining an optimal sub-coding block partition mode for each maximum coding unit; A coding block division module, used to divide the maximum coding unit according to the optimal sub-coding block division mode to obtain sub-coding blocks to be encoded; An optimal coding prediction module, used to perform mode prediction on the sub-coding block using a texture perception network to obtain an optimal intra-frame coding mode of the sub-coding block; The video encoding module is used to encode the sub-coding blocks of the power system monitoring video according to the optimal intra-frame encoding mode to obtain encoded video stream data.

9. A power system monitoring video fast encoding optimization system as claimed in claim 8, characterized in that: The optimal partition prediction module is specifically used for: The maximum coding unit is taken as the target coding unit, the brightness data of the target coding unit is extracted from the power system monitoring video frame, and the texture extraction module is used to perform feature extraction on the brightness data of the target coding unit to obtain the coding block texture information; Traversing multiple types of tree partitioning methods, and determining the partitioning probability of the target coding unit using each type of tree partitioning method according to the coding block texture information; Determine a partitioning mode prediction result of a target coding unit according to the partitioning probability of each type of tree partitioning mode, and divide the target coding unit into a plurality of sub-coding units according to the partitioning mode prediction result; The multiple sub-coding units are used as target coding units in the next recursive division stage, and the above steps are repeated to recursively predict the target coding units according to the quantization parameters until the recursive end condition is reached, thereby generating an optimal sub-coding block division mode.

10. The power system monitoring video fast encoding optimization system according to claim 9, characterized in that: The texture extraction module includes a plurality of texture extraction units in series; the texture extraction unit includes a first convolutional layer, a visual attention network module and a pooling layer; The visual attention network module includes a second convolutional layer, an activation layer, a depthwise separable convolutional layer, a depthwise dilated convolutional layer, a third convolutional layer, a fourth convolutional layer and a feedforward neural network module connected in sequence; wherein the input of the visual attention network module forms a residual connection with the output of the fourth convolutional layer; and the output of the activation layer forms a residual connection with the output of the third convolutional layer.

Images