Power transmission channel video compression method and system based on depth feature extraction network

Abstract

The invention provides a power transmission channel video compression method and system based on a depth feature extraction network, and the method comprises the steps: obtaining a power transmission channel monitoring video stream, inputting the video stream into a pre-constructed end-to-end depth video compression frame, and achieving the video compression of a power transmission channel. Wherein the depth video compression framework comprises an optical flow network, an MV coding and decoding network, a motion compensation network, a residual error coding and decoding network and a rate distortion function network, and efficient compression and quality maintenance of a power transmission channel monitoring video are realized by constructing the end-to-end depth video compression framework; through a deep learning technology, video key information is ensured not to be lost while the compression ratio is improved, and the low-bandwidth transmission and efficient storage requirements of a power transmission channel monitoring system are met.

Description

Technical Field

[0001] This invention relates to the field of power equipment safety monitoring technology, and in particular to a method and system for video compression of power transmission channels based on deep feature extraction networks. Background Technology

[0002] A power transmission channel refers to a set of transmission lines connecting two central transformers across a region. It is a collection of transmission branches with generally consistent power flow direction and plays a crucial role in the power grid. In a power network, the main components of a power transmission channel include high-voltage transmission lines, transmission towers, and cables. These facilities have the capacity to provide large amounts of electricity to different regions, supporting various industrial activities in modern society. Furthermore, transmission channels have the ability to regulate power flow. In cases of power shortages, they ensure power supply through instantaneous overload; in cases of power surplus, they ensure system stability by downgrading transmission lines. However, during the transmission of electricity, power transmission channels are subject to many uncontrollable risks.

[0003] When a transmission line fails, the power grid's power delivery status changes rapidly. Failure to promptly resolve the fault can lead to unpredictable consequences. Therefore, transmission lines require precise and rapid monitoring. Furthermore, to achieve timely early warning, the real-time performance of the transmission line monitoring system's video monitoring is crucial. Transmission of transmission line monitoring video is far more complex than transmitting ordinary text, images, or videos, with a significantly larger data volume. During long-term operation, the edge devices of the transmission line monitoring system collect a massive amount of video and monitoring data, creating a huge burden on transmission and storage. Moreover, the amount of information recorded and data generated by high-definition and ultra-high-definition video acquisition is approximately 2 to 10 times that of ordinary video data, correspondingly increasing end-to-end data bandwidth pressure. This poses a significant challenge to the communication system of the transmission line monitoring system, and the massive data storage also places enormous pressure on the server. The transmission of large amounts of transmission line monitoring video image data by the communication system server inevitably consumes a large amount of communication bandwidth, placing a significant economic burden on the power grid company. Because high-voltage transmission lines are typically planned and constructed in mountainous or suburban areas far from cities, most edge-end cameras on these lines rely on 4G network signals for data transmission. Furthermore, the network signal strength of some edge-end cameras in mountainous areas is poor, limiting the efficiency of video image data transmission and leading to unstable data acquisition, poor image quality, and stuttering. Therefore, transmission line monitoring systems require more efficient video compression and encoding technologies. These technologies must achieve higher compression ratios while ensuring that critical information is not lost during transmission and that video quality remains within acceptable limits. Summary of the Invention

[0004] In view of the above problems, this invention proposes a video compression method and system for power transmission channels based on deep feature extraction networks, which solves the problems of high computational complexity and low compression efficiency of traditional video compression models in power transmission channel monitoring systems. This invention is applicable to edge devices in power transmission channel monitoring systems, reducing data transmission and storage pressure and improving the operating efficiency of the power transmission channel detection system.

[0005] This invention provides a method for video compression of power transmission channels based on a deep learning feature extraction network, comprising: S1: Acquire the video stream of the power transmission channel monitoring and input it into a pre-constructed end-to-end deep video compression framework; the deep video compression framework includes an optical flow network, an MV codec network, a motion compensation network, a residual codec network, and a rate distortion function network; S2: Utilize optical flow networks to extract inter-frame optical flow information from the video stream monitored by the power transmission channel; S3: The inter-frame optical flow information is compressed, encoded, decoded, and reconstructed using an MV encoding / decoding network to obtain reconstructed motion information; S4: Using the motion compensation network, the reconstructed frame corresponding to the previous moment image obtained by the depth video compression framework and the reconstructed motion information are used to perform motion information compensation to obtain the predicted frame, and the residual information between the predicted frame and the original frame corresponding to the current moment is calculated. S5: The residual information is used by the residual encoding / decoding network to recover the compressed residual details and obtain the reconstructed frame; S6: Utilize the rate-distortion function network to dynamically balance the bit rate and reconstruction quality of video compression.

[0006] Optionally, the optical flow network is constructed based on a spatial pyramid network with four levels, and step S2 uses the optical flow network to extract inter-frame optical flow information, including: S21: Input the original frame corresponding to the image at the current moment and the reconstructed frame corresponding to the image at the previous moment into the optical flow network, and downsample the original frame and the reconstructed frame three times respectively to obtain images of four sizes; S22: Initialize the optical flow matrix based on the smallest image among the four sizes; S23: Bilinear interpolation is used to warp the reconstructed frame to the viewpoint of the original frame, generating an optical flow warp feature matrix; S24: The original frame, the optical flow distortion feature matrix, and the initialized optical flow matrix are concatenated and input into a custom neural network for feature learning to capture multi-scale features; S25: Upsample the output of step S24 by a factor of 2; S26: After repeating steps S23 to S25 multiple times, optical flow information with the same resolution as the original frame is finally obtained.

[0007] Optionally, step S3 involves compressing, encoding, decoding, and reconstructing the inter-frame optical flow information using the MV codec network to obtain the reconstructed motion information, including: Optical flow information is input into the MV encoder, and the optical flow information is processed by multi-layer convolution operation and normalized GDN using the MV coding network to obtain the corresponding motion representation; The motion representation is quantized to obtain a quantized motion information representation, which is then input into the MV decoder in the MV codec network for decoding to obtain a reconstructed motion information representation.

[0008] Optionally, in the MV encoding / decoding network process, the quantized representation of motion information obtained by the quantization operation is also used as input for entropy coding; wherein, entropy coding is used to estimate the bit rate of the input data, which helps to optimize the network's loss function.

[0009] Optionally, the entropy coding process includes: Perform bit rate estimation, bit count = -log2(p(x)), where p(x) is the probability density of input x; and output an approximate cumulative distribution function (CDF) value using a sigmoid function. Number of bits y = log(CDF(upper) - CDF(lower)), where upper / lower are the quantization boundaries; By performing residual design, f(x) = x + tanh(x) × tanh(a), which can be regarded as an approximation of a higher-order Taylor expansion, it is possible to learn any smooth function.

[0010] Optionally, step S4, which uses the motion compensation network to perform motion information compensation on the reconstructed frame corresponding to the previous moment image obtained by the depth video compression framework and the reconstructed motion information to obtain the predicted frame, includes: The reconstructed frame and reconstructed motion information are used to generate a transform frame through bilinear interpolation; The transformed frame, reconstructed frame, and reconstructed motion information are simultaneously input into the motion compensation network to obtain the predicted frame.

[0011] Optionally, step S5, which uses the residual information to recover the compressed residual details and obtain the reconstructed frame through the residual codec network, includes: The original frame and residual information are input into the residual encoding / decoding network; The residual information is input into a series of convolutional layers and normalization layers for processing to obtain the potential high-dimensional feature information of the residual, and the high-dimensional feature information is input into the quantization layer for quantization processing to obtain the quantized representation of the residual information; The quantized representation of the residual information is input into a series of convolutional layers and an iterative grouped dilatation network for processing, and the reconstructed residual representation is output; wherein, the quantized representation of the residual information is also used as the input of the entropy coding network.

[0012] Optionally, the calculation mode of the quantization layer is as follows: During the model training phase, values ​​ranging from 10 are added to the optical flow information. -4 ~10 -5 The random quantization noise matrix reduces network overfitting; During the model validation phase, quantization operations will be performed by rounding up or down.

[0013] Optionally, in step S6: by introducing a rate-distortion function and network autonomous training adjustment, based on the negative correlation between the distortion function D and the number of bits R, the balance point with the minimum loss value is obtained, that is, the optimal balance point between compression ratio and video quality is obtained.

[0014] The present invention also provides a power transmission channel video compression system based on a deep learning feature extraction network, for performing the power transmission channel video compression method based on a deep learning feature extraction network as described above. The system includes: a pre-constructed end-to-end deep video compression framework; the deep video compression framework includes an optical flow network, an MV codec network, a motion compensation network, a residual codec network, and a rate-distortion function network. The optical flow network is used to extract inter-frame optical flow information from the video stream monitored by the power transmission channel; The MV encoding / decoding network is used to compress, encode, decode, and reconstruct the inter-frame optical flow information to obtain reconstructed motion information; The motion compensation network is used to perform motion information compensation on the reconstructed frame corresponding to the previous moment image obtained by the depth video compression framework and the reconstructed motion information to obtain a predicted frame, and to calculate the residual information between the predicted frame and the original frame corresponding to the current moment. The residual encoding / decoding network is used to recover compressed residual details using the residual information to obtain a reconstructed frame; The rate-distortion function network is used to dynamically balance the bit rate and reconstruction quality of video compression.

[0015] The present invention relates to a video compression method and system for power transmission channels based on deep feature extraction networks. By constructing an end-to-end deep video compression framework, it achieves efficient compression and quality maintenance of power transmission channel monitoring videos. Through deep learning technology, it improves the compression rate while ensuring that key video information is not lost, thus meeting the low-bandwidth transmission and high-efficiency storage requirements of power transmission channel monitoring systems.

[0016] Furthermore, the optical flow network and the MV codec network are jointly used to achieve high-precision motion information processing and improve the efficiency of inter-frame redundancy elimination; the residual codec network preserves key video details through nonlinear feature compression and dynamically balances compression ratio and reconstruction quality by incorporating a rate-distortion optimization function. The new framework significantly improves compression performance while reducing computational complexity. Experiments demonstrate that this technology has significant advantages in compression ratio, reconstruction quality, and edge-end adaptability for power transmission channel monitoring videos, and its robustness to complex scenes is enhanced through a sequential training strategy. Its efficient compression and reconstruction capabilities can be extended to other video monitoring scenarios in power systems, providing a new solution for the massive video data processing of smart grids.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic flowchart of a power transmission channel video compression method based on a deep feature extraction network according to an embodiment of the present invention is shown. Figure 2 A schematic diagram of the DVC model framework according to an embodiment of the present invention is shown; Figure 3 A spatial pyramid network structure diagram according to an embodiment of the present invention is shown; Figure 4 The diagram shows the MV encoding compression network structure according to an embodiment of the present invention; Figure 5 A diagram of the motion compensation network structure according to an embodiment of the present invention is shown; Figure 6 A residual encoding / decoding network structure diagram according to an embodiment of the present invention is shown; Figure 7 This diagram illustrates a comparison and evaluation of the peak signal-to-noise ratio of the compression model according to an embodiment of the present invention. Figure 8 A schematic diagram illustrating the multi-scale structural similarity comparison evaluation of the compression model according to an embodiment of the present invention is shown. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0021] This invention provides a method for video compression of power transmission channels based on deep feature extraction networks, such as... Figure 1 As shown, the power transmission channel video compression method based on deep feature extraction network in this embodiment of the invention may include the following steps S1 to S6.

[0022] S1: Acquire the video stream from the power transmission channel monitoring and input it into a pre-built end-to-end deep video compression framework; the deep video compression framework includes an optical flow network, an MV codec network, a motion compensation network, a residual codec network, and a rate distortion function network.

[0023] S2: Inter-frame optical flow information of the power transmission channel monitoring video stream is extracted using an optical flow network. In this embodiment, the power transmission channel monitoring video stream is input into the video compression framework, and the current input image in the video stream is recorded as the original frame. (i.e., the image corresponding to time t), for the input raw frame Perform optical flow motion and preliminary compression processing to transform the original frame and the reconstructed frame corresponding to the image at the previous moment (i.e., the reconstructed frame corresponding to the image at time t-1) is input into the optical flow network for optical flow estimation to obtain optical flow information. .

[0024] S3: The inter-frame optical flow information is compressed, encoded, decoded, and reconstructed using the MV codec network to obtain the reconstructed motion information.

[0025] S4: Using a motion compensation network, the reconstructed frame and reconstructed motion information of the previous moment obtained from the deep video compression framework are used to compensate for motion information to obtain the predicted frame, and the residual information between the predicted frame and the original frame corresponding to the current moment is calculated. In other words, the obtained reconstructed motion information... Predicted frames are obtained by performing motion information compensation through a motion compensation network. The motion information is about to be reconstructed. and the previous reconstructed frame Copy the corresponding pixel in the original frame In the middle, the predicted frame is obtained. The original frame With the predicted frame Subtraction is performed to obtain residual information. ,Right now .

[0026] S5: The reconstructed frame corresponding to the current moment is obtained by recovering the compressed residual details using a residual codec network. In this embodiment, the residual information... The input is fed into a highly nonlinear residual codec network for compression, quantization, and decompression operations; that is, the residual is processed by the residual coding network. Nonlinear mapping to Above, General Quantified as Then the residual is represented The reconstructed residual is obtained through the residual decoding network. Finally, the predicted frames... and residual representation Add to obtain the reconstructed frame ,Right now and will reconstruct frames The input is buffered and output to the decoding frame buffer, where the reconstructed frame... The original frame at time (t+1) will be used for subsequent motion estimation operations.

[0027] S6: Utilize the rate distortion function network to dynamically balance the bit rate and reconstruction quality of video compression.

[0028] This embodiment introduces a rate-distortion function and network autonomous training adjustment. Based on the negative correlation between the distortion function D and the number of bits R, it obtains the balance point with the minimum loss value, that is, the optimal balance point between compression ratio and video quality.

[0029] This invention provides a video compression method for power transmission channels based on a deep feature extraction network. By constructing an end-to-end deep video compression framework, it achieves efficient compression and quality preservation of power transmission channel monitoring videos. Through deep learning technology, it improves the compression rate while ensuring that key video information is not lost, meeting the low-bandwidth transmission and high-efficiency storage requirements of power transmission channel monitoring systems. The specific implementation process of each step is described in detail below.

[0030] Step S1: Acquire the power transmission channel monitoring video stream and input it into a pre-built end-to-end deep video compression framework (DVC). The deep video compression framework in this embodiment integrates multimodal deep learning modules, including an optical flow network, an MV codec network, a motion compensation network, a residual codec network, and a rate-distortion function network. For example... Figure 2 As shown, the end-to-end deep video compression framework (DVC) in this embodiment achieves efficient compression and quality preservation of power transmission channel monitoring videos, meeting the low-bandwidth transmission and high-efficiency storage requirements of the power transmission channel monitoring system.

[0031] Step S2: Use an optical flow network to extract inter-frame optical flow information from the power transmission channel monitoring video stream to provide key input for subsequent motion compensation and video compression.

[0032] The optical flow network in this embodiment employs a spatial pyramid architecture and differentiable interpolation technology to construct a multi-receptive field feature processing system, achieving sub-pixel-level optical flow estimation and optimizing robustness through a joint loss function. It can dynamically adjust its strategy based on spatiotemporal features, enhancing its ability to capture complex motions and mitigating the instability of traditional methods in low-texture and occluded scenes. This provides accurate motion information for motion compensation and residual compression of power transmission channel monitoring videos, improving the adaptability and efficiency of the compression framework.

[0033] In an optional embodiment of the present invention, step S2 may further include: S21: Input the original frame corresponding to the current time-phase image and the reconstructed frame corresponding to the previous time-phase image into the optical flow network, and downsample the original frame and the reconstructed frame three times respectively to obtain images of four sizes. In this embodiment, the original frames corresponding to two adjacent images are... and reconstructed frames As the input spatial pyramid structure network for the optical flow network, a neural network with multiple receptive fields was constructed. This neural network has a 4-level pyramid structure. For each of the two frames input to the optical flow network, the images were downsampled 3 times (2x downsampling), resulting in a total of 4 image sizes.

[0034] S22: Initialize the optical flow matrix based on the smallest image among the four sizes; S23: Bilinear interpolation is used to warp the reconstructed frame to the viewpoint of the original frame, generating an optical flow warp feature matrix; the reconstructed frame... warp back to the original frame The perspective is to align the reconstructed frame to the original frame, generating an optical flow warp feature matrix. This means reconstructing the frame... Align to original frame This generates an optical flow distortion feature matrix. The specific steps are as follows: first, the original frame... Normalization and reconstructed frames The results are summed, then bilinear interpolation and boundary filling are performed, and finally, a grid sampling operation is performed.

[0035] S24: The original frame, the optical flow distortion feature matrix, and the initialized optical flow matrix are concatenated and input into a custom neural network for feature learning to capture multi-scale features. In this embodiment, the custom neural network can be a network... G k It includes various convolution operations (3×3, 5×5, 7×7, 9×9 convolution).

[0036] S25: Upsample the output of step S24 by a factor of 2; S26: After repeating steps S23 to S25 multiple times, optical flow information with the same resolution as the original frame is finally obtained. .

[0037] This embodiment employs an optical flow network based on a spatial pyramid network architecture to achieve sub-pixel-level precision optical flow estimation, effectively capturing motion details in complex scenes. Through multi-scale feature fusion and joint loss function optimization, this network provides high-precision motion representation for subsequent motion compensation networks, thereby enhancing residual compression efficiency and ultimately improving the adaptability and compression performance of the entire video compression framework for specific power transmission channel scenarios. The spatial pyramid network is as follows: Figure 3 As shown.

[0038] Step S3: Compress, encode, decode, and reconstruct the inter-frame optical flow information using the MV codec network to obtain the reconstructed motion information. Figure 4 A diagram of the MV encoding compression network structure according to an embodiment of the present invention is shown.

[0039] Optionally, step S3 may further include: S31: The optical flow information is input into the MV encoder. The MV coding network performs multi-layer convolution operations and normalized GDN (Generalized Divisive Normalization) processing on the optical flow information to obtain the corresponding motion representation. In this embodiment, the optical flow information... The optical flow information is input into the MV encoder, where the MV encoding network processes it. Perform multi-layer convolution operations The normalized GDN processing involves the following steps: first, a downsampled 3×3 convolution is performed, followed by normalized GDN, then a regular 3×3 convolution (without downsampling), and finally, normalized GDN is applied again. This yields a motion representation of size M / 16×N / 16×128. .

[0040] S32: Quantize the motion representation to obtain the quantized motion information representation, and input the quantized motion information representation into the MV decoder in the MV codec network for decoding to obtain the reconstructed motion information representation.

[0041] This embodiment utilizes an MV codec network to process optical flow information. Compression, quantization, and decompression operations are performed; that is, the optical flow information is processed through the MV encoder network. Compression yields a motion representation and represent motion Quantization is performed to obtain motion information and its representation. Quantify motion information The input is fed into the MV decoder for decoding, i.e., the input... After multiple deconvolution operations The reconstructed motion information representation is obtained through iterative grouping dilatation network processing. Then, the optical flow information is processed using an MV codec network. Perform compression, quantization, and decompression operations.

[0042] Meanwhile, during the MV encoding / decoding network process, the motion information obtained from the quantization operation is quantized and represented. It is also used as input for entropy coding. Entropy coding is used to estimate the bit rate of the input data, aiding in the optimization of the network's loss function. The entropy coding process is as follows: First, perform bit rate estimation, bit number = -log2(p(x)), where p(x) is the probability density of input x, and the sigmoid function is used to output the approximate cumulative distribution function CDF value; Number of bits y = log(CDF(upper) - CDF(lower)), where upper / lower are the quantization boundaries; Then, a residual design is performed, f(x) = x + tanh(x) × tanh(a), which can be regarded as an approximation of a higher-order Taylor expansion and can learn any smooth function. The final bit is estimated using this formula, where a is the function expansion point.

[0043] Step S4: Use a motion compensation network to perform motion information compensation on the reconstructed frame and reconstructed motion information corresponding to the previous moment image obtained by the depth video compression framework to obtain the predicted frame, and calculate the residual information between the predicted frame and the original frame corresponding to the current moment. Figure 5 This is the structure diagram of the motion compensation network, which is responsible for obtaining more accurate prediction frames.

[0044] This embodiment employs a motion compensation network based on a CNN-based motion compensation method. The reconstructed frames obtained by the compression network and the original frames obtained by the MV coding network are reconstructed through the motion compensation network. The specific process is as follows: S41: Reconstruct the frame and reconstruct motion information Transformed frames are generated through bilinear interpolation; this process uses bilinear interpolation to... warp to The perspective involves aligning the reconstructed frame with the motion information to generate an optical flow distortion feature matrix (the specific steps are to first align the motion information...). Normalization and reconstructed frames The process involves adding the components, then performing bilinear interpolation and boundary filling, and finally performing grid sampling (i.e., transforming the frame).

[0045] S42: Input the transformed frame, reconstructed frame, and reconstructed motion information into the motion compensation network simultaneously to obtain the predicted frame.

[0046] Because artifacts exist in the transformed frame. To eliminate these artifacts, the transformed frame and the reconstructed frame are... and reconstruct motion information The inputs are fed into a CNN network, a motion compensation network, whose structure and computational flow are as follows: 1. 3×3 convolution; 2. ReLU activation function; 3. Residual module; 4. Average pooling (double downsampling); 5. Residual module; 6. Average pooling (double downsampling); 7. Residual module; 8. Residual module; 9. Bilinear interpolation (double upsampling); 10. Residual module; 11. Bilinear interpolation (double upsampling); 12. Residual module; 13. 3×3 convolution. Finally, accurate predicted frames are obtained after processing. .

[0047] In addition, the original frame With the predicted frame Subtraction is performed to obtain residual information. ,Right now .

[0048] Step S5: The reconstructed frame is obtained by recovering the compressed residual details using a residual codec network. This embodiment employs a residual codec network based on a highly nonlinear neural network to recover the compressed residual details and obtain the reconstructed frame. The residual codec network structure is as follows: Figure 6 As shown.

[0049] In an optional embodiment of the present invention, step S5 may include: S51: Transfer the original frame With residual information The input is fed into the residual encoding / decoding network. That is, residual encoding is first performed on the original frame... Predicted frames output by the motion compensation network residual information between The input is fed into the residual codec network.

[0050] S52: Transfer residual information The input is processed through a series of convolutional and normalized layers to obtain the latent high-dimensional feature information of the residuals. and high-dimensional feature information The residual information is quantized by quantization in the input quantization layer. The quantization layer is calculated as follows: during model training, a very small random quantization noise matrix with values ​​ranging from 10 is added to the optical flow information. -4 ~10 -5This reduces overfitting in the network; during the model validation phase, quantization is changed to rounding (rounding up or down) to avoid the impact of randomness on optical flow information.

[0051] S53: Next, perform residual decoding to quantize and represent the residual information. The input is processed by a series of convolutional layers and an iterative grouped dilatation network, and the output is a reconstructed residual representation. Among them, residual information is quantified. It also serves as the input to the entropy coding network. In this embodiment, the entropy coding network is a fixed, existing network used to obtain the reconstructed residual representation and the quantized representation of the residual information. The number of bits. And it is included as part of the rate-distortion function (loss).

[0052] Step S6: Utilize the rate-distortion function network to dynamically balance the bit rate of video compression and reconstruction quality, achieving globally optimal training of the end-to-end network.

[0053] This embodiment obtains the balance point with the minimum loss value by introducing a rate-distortion function and network autonomous training adjustment. The balance point is adjusted by the network autonomous training. Generally, there is a negative correlation between the distortion function D and the number of bits R, and there will exist a balance point with the minimum loss value. The rate-distortion function is:

[0054] in, These are the Lagrange coefficients of the rate-distortion function, used to balance video compression rate and distortion. It is a distortion function, representing the original frame. With reconstructed frames Distortion; It refers to the quantization representation used for encoding. and The sum of the number of bits.

[0055] During the model testing phase, the motion information is quantized and represented using an entropy coding network. The quantization representation of motion information obtained from residual information The data is encoded into bits and then sent to the decoder. During the training phase, a bit rate estimation network is used to obtain the bit rate cost. and The probability distribution of each symbol.

[0056] In an optional embodiment of the present invention, power transmission channel monitoring videos from five edge cameras corresponding to different scenes are collected as a dataset. Data collection for each scene is conducted during both daytime and nighttime periods. Specifically, in the compression model training section, 10 sets of power transmission channel monitoring videos were collected, each video lasting 20 seconds; in the compression model testing section, 10 sets of power transmission channel monitoring videos were collected, each video lasting 10 seconds. The 10 sets of video data in the testing section serve both as test data for the DVC model and as control data for the HEVC model output, as shown in Table 1.

[0057] Table 1

[0058] Preferably, the DVC model is trained using rate-distortion functions with different Lagrange coefficients (λ=256, 512, 1024, 2048), resulting in four models. The Adam optimizer is used, and the initial learning rate of the DVC video compression model network is set to 0.0001. Set to 0.9, Set to 0.999. When the LOSS function network in the DVC video compression model... When the system stabilizes, the initial learning rate is reduced by a factor of 10, and the mini-batch size is set to 4. This means that in a subsequent iteration, the video image data input to the DVC video compression model network is divided into 4 smaller batches for processing.

[0059] The DVC model was trained using a sequential training approach. First, the entire DVC video compression model network was trained using the Vimeo-90k dataset over a period of eight days, resulting in four preliminary DVC video compression models. Second, these four preliminary DVC video compression models were further trained using a power transmission channel monitoring video dataset.

[0060] Preferably, to objectively evaluate the compression performance of the implemented DVC model, a comparative experiment with the HEVC model was conducted. FFmpeg and OpenCV instructions and code were used to read the original acquired video data, obtaining four sets of HEVC model evaluation metrics. Furthermore, the test dataset was used to test the four trained DVC models, obtaining four sets of DVC model evaluation metrics.

[0061] Compressed video data from HEVC models with CRF=20, 23, 26, and 29 were obtained. Using FFmpeg and OpenCV instructions and code, the BPP (Bits Per Pixel), PSNR (Peak Signal-to-Noise Ratio), and MS-SSIM (Multi-Scale Structural Similarity) values ​​of the four sets of video data were read, as shown in Table 2.

[0062] Table 2

[0063] In the DVC model network, λ=2048, 1024, 512, and 256 were set respectively, and the corresponding DVC models (λ=2048, 1024, 512, and 256) were started in sequence. The corresponding test datasets (CRF=20, 23, 26, and 29) were obtained to test the models. Finally, four sets of DVC model evaluation index data were output, as shown in Table 3.

[0064] Table 3

[0065] Preferably, to objectively evaluate the compression performance of the HEVC and DVC models on power transmission channel monitoring videos, during the model testing phase, evaluation index data for both the HEVC and DVC models are obtained based on the collected and preprocessed power transmission channel monitoring videos. Specifically, the compression ratio of the HEVC and DVC models is compared by measuring the pixel depth (BPP). The quality of the compressed power transmission channel monitoring videos is compared by comparing the peak signal-to-noise ratio (PSNR), thus comparing the degree of quality loss caused by the HEVC and DVC models. Multi-scale structural similarity (MS-SSIM) is used to evaluate the similarity between the compressed and uncompressed power transmission channel monitoring videos in terms of global brightness, contrast, image edges, and texture, thus comparing the performance of the HEVC and DVC models. The size of the BPP is directly or indirectly positively correlated with the values ​​of PSNR and MS-SSIM. Therefore, Figure 7 and Figure 8 Scatter plots with BPP as the x-axis and PSNR and MS-SSIM as the y-axis, respectively. The magnitude of BPP is determined by the magnitude of the x-axis of each node, while the magnitudes of PSNR and MS-SSIM can be determined by the y-axis of each node and the trend of the curve.

[0066] Depend on Figure 7 and Figure 8Analysis can provide a direct evaluation of the merits of the HEVC and DVC models. (Summary) Figure 7 and Figure 8 It can be determined that, in terms of BPP, the DVC model outperforms the HEVC model in compression performance; in terms of MS-SSIM, the DVC model has similar or better compression performance than the HEVC model; and in terms of PSNR, the DVC model is slightly inferior to the HEVC model in compression performance.

[0067] Based on the same inventive concept, this invention also provides a power transmission channel video compression system based on a deep learning feature extraction network, used to execute the power transmission channel video compression method based on a deep learning feature extraction network described in the above embodiments. The system includes: a pre-constructed end-to-end deep video compression framework; the deep video compression framework includes an optical flow network, an MV codec network, a motion compensation network, a residual codec network, and a rate-distortion function network. The optical flow network is used to extract inter-frame optical flow information from the video stream monitored by the power transmission channel; The MV encoding / decoding network is used to compress, encode, decode, and reconstruct the inter-frame optical flow information to obtain reconstructed motion information; The motion compensation network is used to perform motion information compensation on the reconstructed frame corresponding to the previous moment image obtained by the depth video compression framework and the reconstructed motion information to obtain a predicted frame, and to calculate the residual information between the predicted frame and the original frame corresponding to the current moment. The residual encoding / decoding network is used to recover compressed residual details using the residual information to obtain a reconstructed frame; The rate-distortion function network is used to dynamically balance the bit rate and reconstruction quality of video compression.

[0068] In an optional embodiment of the present invention, the optical flow network is constructed based on a spatial pyramid network with four levels. The optical flow network is also used to input the original frame corresponding to the current time-phase image and the reconstructed frame corresponding to the previous time-phase image into the optical flow network, and downsample the original frame and the reconstructed frame three times respectively to obtain images of four sizes; initialize the optical flow matrix based on the smallest size of the four sizes; use bilinear interpolation to distort the reconstructed frame to the perspective of the original frame to generate an optical flow distortion feature matrix; concatenate the original frame, the optical flow distortion feature matrix and the initialized optical flow matrix, and input them into a custom neural network for feature learning to capture multi-scale features; upsample the output by 2 times; repeat the above steps multiple times to finally obtain optical flow information with the same resolution as the original frame.

[0069] In an optional embodiment of the present invention, the MV encoding and decoding network is further used to input optical flow information into the MV encoder, and to perform multi-layer convolution operation and normalized GDN processing on the optical flow information using the MV encoding network to obtain the corresponding motion representation; The motion representation is quantized to obtain a quantized motion information representation, which is then input into the MV decoder in the MV codec network for decoding to obtain a reconstructed motion information representation.

[0070] In an optional embodiment of the present invention, during the MV encoding and decoding network process, the quantized representation of motion information obtained by the quantization operation is also used as input for entropy coding; wherein, entropy coding is used to estimate the bit rate of the input data, which helps to optimize the loss function of the network.

[0071] In an optional embodiment of the present invention, the motion compensation network is further used to generate a transformed frame by bilinear interpolation of the reconstructed frame and the reconstructed motion information; the transformed frame, the reconstructed frame and the reconstructed motion information are simultaneously input into the motion compensation network to obtain a predicted frame.

[0072] In an optional embodiment of the present invention, the residual codec network is further used to input the original frame and residual information into the residual codec network; The residual information is input into a series of convolutional layers and normalization layers for processing to obtain the potential high-dimensional feature information of the residual, and the high-dimensional feature information is input into the quantization layer for quantization processing to obtain the quantized representation of the residual information; The quantized representation of the residual information is input into a series of convolutional layers and an iterative grouped dilatation network for processing, and the reconstructed residual representation is output; wherein, the quantized representation of the residual information is also used as the input of the entropy coding network.

[0073] For details on the specific structure and execution process of each network in the deep video compression framework of this embodiment, please refer to the above method embodiments.

[0074] This invention also provides a computer-readable storage medium for storing program code for executing the power transmission channel video compression method based on a deep learning feature extraction network described in the above embodiments.

[0075] This invention also provides a computing device, which includes a processor and a memory: the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the power transmission channel video compression method based on a deep learning feature extraction network as described in the above embodiments according to the instructions in the program code.

[0076] Those skilled in the art will clearly understand that the specific working process of the systems, devices, modules and units described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0077] Furthermore, the functional units in the various embodiments of the present invention can be physically independent of each other, or two or more functional units can be integrated together, or all functional units can be integrated into one processing unit. The integrated functional units described above can be implemented in hardware, or in software or firmware.

[0078] Those skilled in the art will understand that if the integrated functional unit is implemented in software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of the present invention when running the instructions. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as a computing device, personal computer, server, or network device) related to program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the computing device, the computing device executes all or part of the steps of the methods described in the various embodiments of the present invention.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to depart from the protection scope of the present invention.

Claims

1. A power transmission channel video compression method based on a deep learning feature extraction network, characterized in that, The method comprises the following steps: S1: acquiring a power transmission channel monitoring video stream and inputting the video stream into a pre-constructed end-to-end deep video compression framework; the deep video compression framework comprises an optical flow network, an MV coding and decoding network, a motion compensation network, a residual coding and decoding network and a rate-distortion function network; S2: extracting inter-frame optical flow information of the power transmission channel monitoring video stream by using the optical flow network; S3: obtaining reconstructed motion information by compressively coding and decoding the inter-frame optical flow information through the MV coding and decoding network; S4: obtaining a predicted frame by performing motion information compensation on the reconstructed frame corresponding to the image at the previous moment and the reconstructed motion information obtained by the deep video compression framework, and calculating residual information of the predicted frame and the original frame corresponding to the current moment; S5: obtaining a reconstructed frame by restoring compressed residual details by using the residual information through the residual coding and decoding network; S6: dynamically balancing the bit rate and the reconstruction quality of the video compression by using the rate-distortion function network.

2. The method of claim 1, wherein, The optical flow network is constructed based on a spatial pyramid network with four levels, and the step S2 of extracting the inter-frame optical flow information by using the optical flow network comprises the following steps: S21: inputting the original frame corresponding to the image at the current moment and the reconstructed frame corresponding to the image at the previous moment into the optical flow network, and down-sampling the original frame and the reconstructed frame three times to obtain four sizes of images; S22: initializing an optical flow matrix based on the image with the smallest size among the four sizes; S23: generating an optical flow distortion feature matrix by distorting the reconstructed frame to the visual angle of the original frame by using bilinear interpolation; S24: splicing the original frame, the optical flow distortion feature matrix and the initialized optical flow matrix, and inputting them into a self-defined neural network for feature learning to capture multi-scale features; S25: performing 2 times up-sampling on the output of the step S24; S26: after repeating the steps S23-S25 for multiple times, finally obtaining the optical flow information with the same resolution as the original frame.

3. The method of claim 2, wherein, The step S3 of obtaining the reconstructed motion information by compressively coding and decoding the inter-frame optical flow information through the MV coding and decoding network comprises the following steps: inputting the optical flow information into an MV encoder, performing multi-layer convolution operation and normalized GDN processing on the optical flow information by using the MV coding network to obtain corresponding motion representation; quantizing the motion representation to obtain a motion information quantized representation, and inputting the motion information quantized representation into an MV decoder in the MV coding and decoding network to obtain a reconstructed motion information representation.

4. The method of claim 3, wherein, In the process of the MV coding and decoding network, the motion information quantized representation obtained by the quantization operation is also used as the input of the entropy coding; wherein the entropy coding is used to estimate the bit rate of the input data, and provides help for the loss function optimization of the network.

5. The method of claim 4, wherein, The entropy coding process comprises the following steps: performing bit rate estimation, and the number of bits = -log2(p(x)), wherein p(x) is the probability density of the input x, and the sigmoid output approximates the cumulative distribution function CDF value; the number of bits y = log(CDF(upper) - CDF(lower)), wherein upper / lower is the quantization boundary; Residual design is performed, f(x) = x + tanh(x) * tanh(a), which is regarded as an approximation of high-order Taylor expansion, and can learn any smooth function, and a is a function expansion point.

6. The method of claim 3, wherein, Step S4 utilizes the motion compensation network to perform motion information compensation on the reconstructed frame corresponding to the image at the previous moment obtained by the deep video compression framework and the reconstructed motion information to obtain a predicted frame. The reconstructed frame and the reconstructed motion information are input into the motion compensation network to obtain a predicted frame. Step S5 utilizes the residual information to recover the compressed residual details by the residual coding and decoding network to obtain a reconstructed frame.

7. The method of claim 3, wherein, The original frame and the residual information are input into the residual coding and decoding network. The residual information is input into a series of convolution layers and normalization layers to obtain latent high-dimensional feature information of the residual, and the high-dimensional feature information is input into a quantization layer to obtain a residual information quantization representation. The residual information quantization representation is input into a series of convolution layers and iterative grouping dilation networks to output a reconstructed residual representation; wherein the residual information quantization representation is also input into an entropy coding network. The calculation mode of the quantization layer is:

8. The method of claim 7, wherein, In the model verification stage, the quantization operation becomes rounding up or down. In the model training stage, a random quantization noise matrix with a value range of 10 -4 ~10 -5 is added in the optical flow information to slow down the overfitting of the network. In step S6:

9. The method of claim 1, wherein, By introducing a rate-distortion function and network autonomous training adjustment, based on the negative correlation between the distortion function D and the bit number R, a balance point with the minimum loss value is obtained, that is, a balance point of the best compression ratio and video quality is obtained. The system for performing the power transmission channel video compression method based on the deep learning feature extraction network of any one of claims 1-9, the system comprising: a pre-constructed end-to-end deep video compression framework; the deep video compression framework comprising an optical flow network, an MV coding and decoding network, a motion compensation network, a residual coding and decoding network, and a rate-distortion function network; 10.A power transmission channel video compression system based on a deep learning feature extraction network, characterized in that, The optical flow network is used to extract inter-frame optical flow information of the power transmission channel monitoring video stream. The MV coding and decoding network is used to compress and decode the inter-frame optical flow information to obtain reconstructed motion information. The motion compensation network is used to perform motion information compensation on the reconstructed frame corresponding to the image at the previous moment obtained by the deep video compression framework and the reconstructed motion information to obtain a predicted frame, and calculate residual information of the predicted frame and the original frame corresponding to the current moment. The residual coding and decoding network is used to recover the compressed residual details by the residual information to obtain a reconstructed frame. The rate-distortion function network is used to dynamically balance the bit rate and reconstruction quality of video compression. ​

Images