Compression transmission method and device for electric power safety supervision video and electronic equipment

Abstract

The invention discloses a compression transmission method and device for an electric power safety supervision video and electronic equipment. The method comprises the steps that an electric power safety monitoring video needing to be transmitted is acquired, the electric power safety monitoring video is used for representing the electric power safety monitoring video, the electric power safety monitoring video at least comprises an image frame set and an audio, and the image frame set at least comprises a current image frame; identifying a dynamic area and a static area in the current image frame; code rate distribution is carried out on the current image frame based on the dynamic area and the static area, a code rate distribution result is obtained, and the code rate distributed to the dynamic area in the code rate distribution result is larger than the code rate distributed to the static area; compressing the image frame set based on the code rate distribution result to obtain a compressed image frame set; and constructing a compressed safety supervision video based on the compressed image frame set and the audio, and transmitting the compressed safety supervision video. According to the invention, the technical problem that the compression transmission efficiency of the electric power safety supervision video is low in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of video compression transmission, and in particular to a method, device and electronic equipment for compressing and transmitting electric power safety monitoring videos. Background Art

[0002] In recent years, we have explored and developed the deep integration and application of artificial intelligence in different business scenarios in the power sector, and formed new achievements represented by artificial intelligence and power Internet of Things and promoted their application. With the deepening of application frequency and application fields, infrastructure visualization devices are facing the problem of massive data set transmission. At present, video monitoring systems such as smart construction sites for safe remote intelligent control of infrastructure sites have been developed, and the monitoring center carries out video monitoring and other services based on systems such as infrastructure smart construction sites.

[0003] In order to ensure the quality of the captured video, most high-definition cameras are used, usually with the encoding format of H.264 and 720P resolution transmission. Each monitoring video occupies a bandwidth of about 2M, which brings about the contradiction between high-resolution video transmission and limited bandwidth, resulting in video screen distortion, freezes, delays, and interruptions. When the video compression transmission method in the relevant technology is used to compress and transmit the power safety monitoring video, there is a contradiction between the user's demand for video quality, the instability of the network transmission bandwidth, and the consumption of computing resources during video compression, resulting in low efficiency in the compression transmission of the power safety monitoring video.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method, device and electronic device for compressing and transmitting electric power safety monitoring videos, so as to at least solve the technical problem of low efficiency in compressing and transmitting electric power safety monitoring videos in related technologies.

[0006] According to one aspect of an embodiment of the present invention, a method for compressing and transmitting electric power safety and monitoring videos is provided, comprising: obtaining an electric power safety and monitoring video to be transmitted, wherein the electric power safety and monitoring video is used to represent an electric power safety monitoring video, the electric power safety and monitoring video includes at least an image frame set and audio, and the image frame set includes at least a current image frame; identifying a dynamic area and a static area in the current image frame; allocating a bit rate for the current image frame based on the dynamic area and the static area to obtain a bit rate allocation result, wherein in the bit rate allocation result, the bit rate allocated to the dynamic area is greater than the bit rate allocated to the static area; compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set; constructing a compressed safety and monitoring video based on the compressed image frame set and audio, and transmitting the compressed safety and monitoring video.

[0007] Optionally, identifying dynamic areas and static areas in the current image frame includes: comparing the pixel value probability distribution of the target area in the current image frame with a preset probability density threshold to obtain a comparison result, wherein the pixel value probability distribution is used to represent the probability density distribution of the pixel values ​​of the target area; when the comparison result is that the pixel value probability distribution is greater than the preset probability density threshold, determining that the target area is a static area; when the comparison result is that the probability distribution is less than or equal to the preset probability density threshold, determining that the target area is a dynamic area.

[0008] Optionally, the method also includes: obtaining a historical pixel value probability distribution of the target area, wherein the historical pixel value probability distribution is used to represent the probability density distribution of pixel values ​​of the target area in at least one historical image frame in the image frame set, and at least one historical past image frame is before the current image frame; and constructing a preset probability density threshold based on the historical pixel value probability distribution.

[0009] Optionally, the image frame set is compressed based on the bit rate allocation result to obtain a compressed image frame set, including: obtaining target key frames and non-target key frames in the image frame set, wherein the target key frames are used to represent image frames in which moving targets appear in two adjacent preset frame intervals in the image frame set, and the non-target key frames are used to represent image frames other than the target key frames in the image frame set; based on the bit rate allocation result, the real-time broadband status, the target key frames and the non-target key frames, the image frame set is compressed to obtain a compressed image frame set, wherein the real-time broadband status is used to characterize the real-time data transmission rate of the transmitted compressed safety monitoring video.

[0010] Optionally, based on the bit rate allocation result, the real-time broadband status, the target key frames and the non-target key frames, the image frame set is compressed to obtain a compressed image frame set, including: determining the number of discarded non-target key frames based on the real-time broadband status; compressing the image frame set by frame number based on the discarded number, the target key frames and the non-target key frames to obtain an initial compressed image frame set; and performing bit rate compression on the initial compressed image frame set based on the bit rate allocation result to obtain a compressed image frame set.

[0011] Optionally, a compressed safety and surveillance video is constructed based on a compressed image frame set and audio, and the compressed safety and surveillance video is transmitted, including: constructing an initial compressed safety and surveillance video based on a compressed image frame set and audio; performing quality enhancement processing on the initial compressed safety and surveillance video based on a compressed video enhancement model to obtain a compressed safety and surveillance video, wherein the compressed video enhancement model is used to represent a network model that improves the clarity of target identification objects under the same compression ratio.

[0012] According to another aspect of an embodiment of the present invention, a compression and transmission device for electric power safety and monitoring videos is also provided, including: an acquisition module, used to acquire electric power safety and monitoring videos to be transmitted, wherein the electric power safety and monitoring videos are used to represent electric power safety monitoring videos, and the electric power safety and monitoring videos include at least a set of image frames and audio, and the image frame set includes at least a current image frame; an identification module, used to identify dynamic areas and static areas in the current image frame; an allocation module, used to allocate bit rates for the current image frame based on the dynamic areas and the static areas, and obtain a bit rate allocation result, wherein the bit rate allocated to the dynamic area in the bit rate allocation result is greater than the bit rate allocated to the static area; a compression module, used to compress the image frame set based on the bit rate allocation result to obtain a compressed image frame set; a construction module, used to construct a compressed safety and monitoring video based on the compressed image frame set and audio, and transmit the compressed safety and monitoring video.

[0013] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0015] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0016] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0017] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0018] In an embodiment of the present invention, a method for compressing and transmitting an electric power safety and monitoring video is provided, comprising: obtaining an electric power safety and monitoring video to be transmitted, wherein the electric power safety and monitoring video is used to represent an electric power safety monitoring video, the electric power safety and monitoring video includes at least an image frame set and an audio, and the image frame set includes at least a current image frame; identifying a dynamic area and a static area in the current image frame; allocating a bit rate to the current image frame based on the dynamic area and the static area to obtain a bit rate allocation result, wherein the bit rate allocated to the dynamic area in the bit rate allocation result is greater than the bit rate allocated to the static area; compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set; constructing a compressed safety and monitoring video based on the compressed image frame set and the audio, and transmitting the compressed safety and monitoring video. It is easy to notice that the present application retains more detailed information for the dynamic area and compresses the static area to a higher degree, which can more effectively utilize bandwidth resources and improve video transmission efficiency. The detailed information of the dynamic area can be retained, the distortion phenomenon in the video transmission process can be reduced, and the visual quality of the video can be improved. Targeted compression processing for different areas can reduce the delay in the video transmission process, improve real-time performance and stability, effectively improve video transmission efficiency and quality, and provide better technical support for the application of power safety monitoring systems; this application also takes into account that users are much more sensitive to sound quality than to changes in image frames. Only compressing image frame data can retain high-quality audio data to improve user experience. It can effectively reduce the size of video data and improve transmission efficiency while ensuring audio quality, thereby optimizing user experience and saving network resources. This solves the technical problem of low efficiency in compressed transmission of power safety monitoring videos in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a flow chart of a method for compressing and transmitting electric power safety monitoring video according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of an optional power safety monitoring video according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of an optional compressed safety monitoring video according to an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of an optional optical flow comparison between a compressed area and an original area according to an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of an optional edge computing-based compressed transmission of power safety monitoring video according to an embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of a fault monitoring device for a power grid loop according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to an embodiment of the present invention, an embodiment of a method for compressing and transmitting electric power safety monitoring videos is provided. It can be explained that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1 is a flow chart of a method for compressing and transmitting electric power safety monitoring video according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0030] Step S102, obtaining the electric power safety monitoring video to be transmitted.

[0031] The power safety monitoring video is used to represent the power safety monitoring video. The power safety monitoring video includes at least an image frame set and audio. The image frame set includes at least the current image frame.

[0032] In an optional embodiment, first, the power safety monitoring video to be transmitted can be obtained, and the power safety monitoring video can include an image frame set and audio. The image frame set includes at least the current image frame. Specifically, the video signal captured by the power safety monitoring camera can be first collected, and the real-time monitoring screen, including image frames and audio, can be obtained through the camera. The collected video signal is encoded and processed, and the video signal is converted into a digital signal, which can reduce the amount of video data, improve transmission efficiency and save storage space. The collected audio signal can also be encoded and processed, and the audio signal is converted into a digital signal, which can effectively improve the transmission efficiency of the power safety monitoring video and save costs, and provide better support and guarantee for the operation of the power safety monitoring system.

[0033] Step S104, identifying the dynamic area and the static area in the current image frame.

[0034] In an optional embodiment, the dynamic area and the static area in the current image frame are identified, and the dynamic area that changes with time in the video and the static area that remains stable can be distinguished, so that different areas can be compressed in a targeted manner, thereby improving the video transmission efficiency and reducing the bandwidth occupancy. Specifically, by comparing the pixel differences between adjacent frames, the area where the video has moved, that is, the dynamic area, can be detected, and more detailed information needs to be retained. The motion detection algorithm may include the inter-frame difference method and the optical flow method. By modeling the static background, the area that has changed compared with the background can be detected, that is, the dynamic area. The background modeling algorithm includes the Gaussian mixture model and the adaptive background modeling. The above methods can also be used in combination to more accurately identify the dynamic area and the static area, and improve the accuracy and robustness of the recognition. The present application retains more detailed information for the dynamic area and compresses the static area to a higher degree, which can more effectively utilize bandwidth resources and improve the video transmission efficiency. The detailed information of the dynamic area can be retained, the distortion phenomenon in the video transmission process can be reduced, and the visual quality of the video can be improved. Targeted compression processing for different areas can reduce delays in video transmission, improve real-time performance and stability, effectively improve video transmission efficiency and quality, and provide better technical support for the application of power safety supervision systems.

[0035] Step S106, performing bit rate allocation on the current image frame based on the dynamic area and the static area to obtain a bit rate allocation result.

[0036] Among them, in the bit rate allocation result, the bit rate allocated to the dynamic area is greater than the bit rate allocated to the static area.

[0037] In an optional embodiment, the bit rate of the current image frame can be allocated based on the dynamic area and the static area. Specifically, the current image frame can be analyzed by a video analysis algorithm to identify the dynamic area and the static area therein. The dynamic area can refer to the area with large pixel value changes in the image, such as a moving object or a person, and the static area can refer to the area with small pixel value changes, such as a background or a fixed object. Since the dynamic area contains more details and changes, more bit rates need to be allocated during the transmission process to ensure the image quality and smoothness of the dynamic area. In the bit rate allocation result, the bit rate allocated to the dynamic area can be greater than the bit rate allocated to the static area. Through the bit rate allocation method based on the dynamic area and the static area, the video picture can be intelligently optimized, and the clarity and smoothness of the image can be improved. At the same time, since the dynamic area is allocated with a higher bit rate protection, the image distortion and blurring caused by the movement of dynamic objects can be effectively reduced, and the viewing experience and recognition accuracy of the video can be improved. This bit rate allocation method can be flexibly adjusted according to the characteristics of the image content to achieve better visual effects and transmission efficiency.

[0038] Step S108: compress the image frame set based on the bit rate allocation result to obtain a compressed image frame set.

[0039] In an optional embodiment, after obtaining the bit rate allocation result, the image frame set can be compressed according to the result. By adjusting the compression ratio of different areas, effective compression of the entire video frame can be achieved, thereby reducing the transmission bandwidth and storage space occupied. The present application can effectively improve the efficiency of video transmission by allocating bit rates to dynamic areas and static areas respectively. The high bit rate in the dynamic area can ensure the clarity of the moving part, and the low bit rate in the static area can reduce redundant data and reduce transmission costs. By performing differentiated bit rate allocation and compression processing on different areas, storage space can be effectively saved. The high bit rate in the dynamic area ensures the storage of key information, while the low bit rate in the static area reduces the storage requirements of redundant data, which can improve video transmission efficiency, image quality and storage space utilization, and provide more reliable and efficient support for the operation and management of security monitoring systems.

[0040] Step S110: construct a compressed safety monitoring video based on the compressed image frame set and the audio, and transmit the compressed safety monitoring video.

[0041] In an optional embodiment, the image frames in the video can be compressed, and the size of the image data can be reduced by adopting a compression coding algorithm while maintaining reasonable image quality, which can effectively reduce the size of the video data and reduce the demand for transmission bandwidth. The compressed image frame set and audio data can be merged to construct a compressed safety and surveillance video file. Specifically, the audio can be accelerated or doubled to match the compressed image frame set. Finally, the compressed safety and surveillance video file can be transmitted to the target device through the network for playback or storage. During the transmission process, the data transmission volume can be further reduced and the transmission efficiency can be improved by optimizing the network protocol and applying the data compression algorithm. The present application takes into account that the user's sensitivity to sound quality is much higher than the sensitivity to changes in image frames. High-quality audio data can be retained to enhance the user experience. On the premise of ensuring audio quality, the video data size can be effectively reduced and the transmission efficiency can be improved, thereby optimizing the user experience and saving network resources.

[0042] In an embodiment of the present invention, a method for compressing and transmitting an electric power safety and monitoring video is provided, comprising: obtaining an electric power safety and monitoring video to be transmitted, wherein the electric power safety and monitoring video is used to represent an electric power safety monitoring video, the electric power safety and monitoring video includes at least an image frame set and an audio, and the image frame set includes at least a current image frame; identifying a dynamic area and a static area in the current image frame; allocating a bit rate to the current image frame based on the dynamic area and the static area to obtain a bit rate allocation result, wherein the bit rate allocated to the dynamic area in the bit rate allocation result is greater than the bit rate allocated to the static area; compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set; constructing a compressed safety and monitoring video based on the compressed image frame set and the audio, and transmitting the compressed safety and monitoring video. It is easy to notice that the present application retains more detailed information for the dynamic area and compresses the static area to a higher degree, which can more effectively utilize bandwidth resources and improve video transmission efficiency. The detailed information of the dynamic area can be retained, the distortion phenomenon in the video transmission process can be reduced, and the visual quality of the video can be improved. Targeted compression processing for different areas can reduce the delay in the video transmission process, improve real-time performance and stability, effectively improve video transmission efficiency and quality, and provide better technical support for the application of power safety monitoring systems; this application also takes into account that users are much more sensitive to sound quality than to changes in image frames. Only compressing image frame data can retain high-quality audio data to improve user experience. It can effectively reduce the size of video data and improve transmission efficiency while ensuring audio quality, thereby optimizing user experience and saving network resources. This solves the technical problem of low efficiency in compressed transmission of power safety monitoring videos in related technologies.

[0043] Optionally, identifying dynamic areas and static areas in the current image frame includes: comparing the pixel value probability distribution of the target area in the current image frame with a preset probability density threshold to obtain a comparison result, wherein the pixel value probability distribution is used to represent the probability density distribution of the pixel values ​​of the target area; when the comparison result is that the pixel value probability distribution is greater than the preset probability density threshold, determining that the target area is a static area; when the comparison result is that the probability distribution is less than or equal to the preset probability density threshold, determining that the target area is a dynamic area.

[0044] In an optional embodiment, the dynamic area and the static area in the current image frame are identified, which can effectively improve the efficiency and quality of video transmission. Specifically, the current image frame can be analyzed first, and the pixel values ​​in the image frame are distributed and counted according to their probability density, so that the pixel value probability distribution can be obtained, which is used to represent the probability density distribution of the pixel values ​​of the target area in the image frame. Then, this pixel value probability distribution can be compared with a preset probability density threshold. The preset probability density threshold can be a set value used to determine whether the target area is a static area or a dynamic area. Then, the target area is classified according to the comparison result. If the pixel value probability distribution is greater than the preset probability density threshold, then the target area can be determined to be a static area; if the pixel value probability distribution is less than or equal to the preset probability density threshold, then the target area can be determined to be a dynamic area. Through the implementation of the above steps, the dynamic area and the static area in the current image frame can be effectively identified, so that corresponding optimization processing can be made in the process of video compression transmission. By identifying the static area and the dynamic area, different compression algorithms and parameter settings can be adopted for different areas, thereby improving the efficiency of compression transmission.

[0045] Optionally, the method also includes: obtaining a historical pixel value probability distribution of the target area, wherein the historical pixel value probability distribution is used to represent the probability density distribution of pixel values ​​of the target area in at least one historical image frame in the image frame set, and at least one historical past image frame is before the current image frame; and constructing a preset probability density threshold based on the historical pixel value probability distribution.

[0046] In an optional embodiment, the historical pixel value probability distribution of the target area can be obtained, and the historical pixel value probability distribution represents the probability density distribution of the pixel values ​​of the target area in at least one historical image frame in the image frame set, and the historical image frame is before the current image frame. Based on these historical pixel value probability distributions, a preset probability density threshold can be constructed for subsequent comparison and judgment of static areas and dynamic areas. It is convenient to perform statistics and analysis on the pixel values ​​of the target area, calculate the probability density distribution, and compare it with the preset threshold, so as to determine the static area and the dynamic area.

[0047] Optionally, the image frame set is compressed based on the bit rate allocation result to obtain a compressed image frame set, including: obtaining target key frames and non-target key frames in the image frame set, wherein the target key frames are used to represent image frames in which moving targets appear in two adjacent preset frame intervals in the image frame set, and the non-target key frames are used to represent image frames other than the target key frames in the image frame set; based on the bit rate allocation result, the real-time broadband status, the target key frames and the non-target key frames, the image frame set is compressed to obtain a compressed image frame set, wherein the real-time broadband status is used to characterize the real-time data transmission rate of the transmitted compressed safety monitoring video.

[0048] In an optional embodiment, the image frame set can be compressed based on the bit rate allocation result. Specifically, the target key frame and non-target key frame in the image frame set can be obtained first to distinguish the moving target from the static background, so as to perform compression processing in a targeted manner. The target key frame refers to the image frame in which the moving target appears in the interval between two adjacent preset frames, and the non-target key frame refers to the image frame other than the target key frame. After obtaining the target key frame and the non-target key frame, the compression processing can be performed in combination with the bit rate allocation result, the real-time broadband status, the target key frame, and the non-target key frame. The real-time broadband status refers to the real-time rate of the current data transmission. By monitoring the bandwidth in real time, the compression ratio can be dynamically adjusted according to different network conditions to ensure the smoothness and image quality of the video transmission. In the above steps, the target key frame and the non-target key frame can be classified and compressed, which can reduce unnecessary data transmission and improve the efficiency of video transmission. The compression ratio can be dynamically adjusted according to the real-time broadband status, which can improve the smoothness of video transmission while ensuring the image quality. By performing differential compression processing on image frame sets, bandwidth resources can be effectively saved, data transmission costs can be reduced, data transmission efficiency and image quality experience can be improved, and bandwidth resources can also be saved, which has a relatively significant application prospect.

[0049] Optionally, based on the bit rate allocation result, the real-time broadband status, the target key frames and the non-target key frames, the image frame set is compressed to obtain a compressed image frame set, including: determining the number of discarded non-target key frames based on the real-time broadband status; compressing the image frame set by frame number based on the discarded number, the target key frames and the non-target key frames to obtain an initial compressed image frame set; and performing bit rate compression on the initial compressed image frame set based on the bit rate allocation result to obtain a compressed image frame set.

[0050] In an optional embodiment, the number of discarded non-target key frames can be determined based on the real-time broadband state, so as to retain the target key frames as much as possible to ensure the video quality when the network bandwidth is limited. By determining the number of discarded non-target key frames, frame compression can be achieved without affecting the integrity of the video content, thereby improving the video transmission efficiency. Specifically, the current network transmission bandwidth can be determined according to the real-time broadband state, and the number of non-target key frames to be discarded can be determined according to the bandwidth. The image frame set can be compressed according to the discarded number, target key frames and non-target key frames, and the non-target key frames can be discarded to obtain an initial compressed image frame set. And the initial compressed image frame set can be compressed based on the bit rate allocation result to obtain a final compressed image frame set. Frame compression is to reduce the number of frames required to be transmitted in video transmission by discarding non-target key frames from a time perspective, thereby reducing the transmission time and occupied bandwidth, improving the video transmission efficiency, and at the same time, by retaining the target key frames, ensuring the continuity and integrity of the video content. From a spatial perspective, bitrate compression can reduce the resolution of each frame by compressing the bitrate of the initial compressed image frame set, reduce the bandwidth required during video transmission, and further improve the efficiency of video transmission. At the same time, by reasonably allocating bitrates, the image quality during video transmission can be guaranteed to ensure the viewing experience. Through the above steps, the amount of transmitted data can be effectively reduced, the transmission efficiency can be improved, and the video transmission requirements in different network environments can be adapted. At the same time, by discarding image frames and compressing bitrates, the network pressure can be reduced to a certain extent, the stability and reliability of video transmission can be improved, the video transmission process can be optimized, and the user experience can be improved.

[0051] Optionally, a compressed safety and surveillance video is constructed based on a compressed image frame set and audio, and the compressed safety and surveillance video is transmitted, including: constructing an initial compressed safety and surveillance video based on a compressed image frame set and audio; performing quality enhancement processing on the initial compressed safety and surveillance video based on a compressed video enhancement model to obtain a compressed safety and surveillance video, wherein the compressed video enhancement model is used to represent a network model that improves the clarity of target identification objects under the same compression ratio.

[0052] In an optional embodiment, an initial compressed safety and monitoring video can be constructed based on a set of compressed image frames and audio; and the initial compressed safety and monitoring video can be quality enhanced based on a compressed video enhancement model. The compressed video enhancement model can be trained by a deep learning method, and the model can improve the clarity of the target object under the same compression ratio, thereby improving the quality of the video. Finally, a compressed safety and monitoring video that has been subjected to compressed video enhancement processing can be obtained, which can be transmitted or stored for use. The present application can effectively improve the clarity of the video image through the compressed video enhancement model, making the target object more clearly visible, thereby improving the quality of the video and the viewing experience. The amount of data in the compressed safety and monitoring video is reduced, the transmission speed is faster, the video data can be transmitted faster, the efficiency and stability of video transmission are improved, the quality and transmission efficiency of the video can be effectively improved, and it can be applied to video monitoring and transmission applications in the field of electric power safety and monitoring.

[0053] The technical solution proposed in the present application is described below in combination with an optional embodiment. The present application provides an intelligent video adaptive compression method based on smart monitoring, including the following steps: based on dynamic area extraction and multi-target tracking technology, identifying dynamic areas and static areas in the monitoring video; using a mixed Gaussian model to distinguish between dynamic areas and static areas, and allocating bit rates to dynamic areas and static areas respectively based on the distinction results; defining key frames and non-key frames, detecting key frames using deep learning-based key frame detection technology, and selectively transmitting key frames and audio according to the user's available bandwidth; using a compressed video enhancement model to perform quality enhancement processing on the transmitted compressed video to improve the clarity of target identification objects under the same compression ratio; based on a pre-constructed video high-rate compression model, dynamically adjusting compression parameters in combination with real-time changes in monitoring video content and network conditions, and using the adjusted compression parameters to perform adaptive high-rate compression on real-time infrastructure video.

[0054] Among them, based on dynamic area extraction and multi-target tracking technology, dynamic areas and static areas in surveillance videos are identified, and the mixed Gaussian model is used to distinguish between dynamic areas and static areas. The following steps are included: using dynamic area extraction technology to extract the moving area in the surveillance picture, and retaining the key target detection results in the moving area; based on multi-target tracking technology, the static key targets that were mistakenly deleted are retrieved, and the key target detection process is skipped when the surveillance picture is completely still.

[0055] Specifically, as the basis of the video compression algorithm, the effect of target detection can directly affect the effects of various downstream attribute analysis algorithms. During the research process, this embodiment found that the simple target detection model is greatly affected by the background during use and is prone to false detection. In order to solve this problem, this embodiment observed that most of the key targets in the construction scene are usually in motion. Based on this observation, this embodiment uses dynamic area extraction technology to extract the moving area in the monitoring picture, and retains the key target detection results in the moving area. Simple processing based on the above process will cause the static key targets in the picture to be mistakenly deleted. This embodiment further uses multi-target tracking technology to retrieve the static key targets that were mistakenly deleted. In addition, by combining dynamic area extraction technology with multi-target tracking technology, the key target detection process can also be skipped when the monitoring picture is completely static, thereby further saving the amount of calculation and improving the real-time performance of the processing.

[0056] Among them, the use of a mixed Gaussian model to distinguish between dynamic areas and static areas, and the allocation of bit rates to dynamic areas and static areas respectively based on the distinction results includes the following steps: at a specific image position, historical pixel information at a preset time in the past is collected, and the mixed Gaussian model is used to estimate the probability density distribution of historical pixel values, and a probability density threshold is set based on the probability density distribution; based on the comparison result of the probability density of the pixel value of the current frame with the probability density threshold, the static area, dynamic area and dynamic area are determined, and the bit rate allocation to the dynamic area and static area is realized based on the determination result.

[0057] Specifically, the video dynamic area extraction technology can extract the moving area in the video screen according to the image information of each frame of the video given a video. From the perspective of digital image processing, if a certain area in the video moves, its pixel value will change. Therefore, in mathematical terms, dynamic area extraction is to extract the area where the pixel value changes in the video screen. To measure the change of pixel values ​​at a certain position in the video screen, the frame difference method can be used in the relevant technology, that is, the pixel value of each frame image in the video is compared with the pixel value of the same position of the previous frame image, and the position exceeding a certain threshold is considered to be a moving area. However, this method is greatly affected by noise and jitter, and the result of dynamic area extraction is unstable. In order to obtain stable dynamic area extraction results, longer historical information must be considered. The present application can use a mixed Gaussian model to extract the dynamic area of ​​the video. At time t, the algorithm first collects the historical pixel information of this position in the past T frames, and uses the algorithm to estimate the probability density distribution of the pixel value at this position. After estimation, the probability density distribution here is a unimodal Gaussian distribution, and the probability density threshold is set based on the probability density distribution. After that, the algorithm further calculates the probability density of the pixel value here at time t based on the estimate. If it is greater than the probability density threshold, it means that the pixel point here belongs to the same distribution as the pixel value at time T in the past. Therefore, this place belongs to the static area, thereby determining the moving area and moving target, allocating more bit rate to the dynamic area, and allocating less bit rate to the background static area, thereby realizing intelligent compression of surveillance video.

[0058] Among them, the definition of key frames and non-key frames can be performed by using key frame detection technology based on deep learning to detect key frames, and key frames and audio can be selectively transmitted according to the user's available bandwidth, including the following steps: the video frames in which moving targets appear in the interval between two adjacent key frames (I frames) in the picture group are defined as key frames, and the remaining frames are defined as non-key frames; the key frames in the video frames are detected by using key frame detection technology based on deep learning to obtain the corresponding key frames and non-key frames in the video frames; a preset number of non-key frames are discarded according to the user's available bandwidth, and the compressed non-key frames, key frames and audio that are not discarded are transmitted.

[0059] Specifically, considering that safety and surveillance video applications usually have a large number of similar frames, such as the edge of a construction site that has been unmanned for a long time. These frames have little impact on the user's acquisition of video information, and receiving these repeated and useless surveillance video frames will waste the user's time. Users often turn to receiving and watching frames with moving targets because these frames can provide users with more picture information. Transmitting useless frames puts pressure on the network but does not bring better results to users receiving information. When the network bandwidth is insufficient, this embodiment will reduce the transmission of useless frames to ensure that users receive more key video information.

[0060] For video frames in which a moving target appears in the interval between two adjacent I frames, this embodiment defines it as a key frame. Frames other than key frames are defined as non-key frames. Audio data usually has a higher transmission priority than non-key frames, and users are much more sensitive to sound degradation or freezes than to video resolution reduction or freezes. Therefore, when the network bandwidth is limited, this embodiment will discard a certain number of non-key frames and select uncompressed key frames and audio for transmission. The bandwidth required to transmit the video compressed by this embodiment is much smaller than the bandwidth required to transmit normal video, so when the network quality is poor, this method can provide users with better experience quality. A specific example is used to illustrate this process. A video is encoded using the H.264 video coding standard, Advanced Audio Coding (ACC) and a video file format.

[0061] Figure 2 is a schematic diagram of an optional power safety monitoring video according to an embodiment of the present invention, such as Figure 2 As shown, the data portion of the electric power safety monitoring video, that is, the video stream file, is composed of a large number of tags, each of which contains video data or audio data. Figure 2 Six video frames (label 1, label 2, label 4, label 5, label 6 and label 7) and three synchronous audio data (label 3, label 8 and label 9) are shown. It can be noticed that label 1, label 2, label 4, label 5 and label 6 are highly similar video frames, which are static pictures in the eyes of users. These frames are defined as non-key frames. These video frames are so similar that users can get information about all subsequent frames by only receiving the first frame. It is difficult to transmit these frames to users when the bandwidth is insufficient, which will squeeze the transmission bandwidth of the frames that users care about. When the bandwidth is insufficient, these frames can be removed to ensure the normal transmission of key frames. A white car (marked in the box) appears in the video frame of label 7. This car does not appear in label 6. It is a target that suddenly appears. It is a target worthy of concern to users. The frames containing the targets that users care about in label 7 cannot be deleted. When the bandwidth drops, the transmission of these frames can also be prioritized.

[0062] If the network condition at the user end is poor, the server extracts six tags: tag 1, tag 2, tag 4, tag 5, tag 6 and tag 7, and obtains six video frames. The six video frames are subjected to key frame detection based on deep learning. Through detection, tags 1, tag 2, tag 4 and tag 5 are non-key frames with high similarity. The server chooses to stop transmitting several non-key frames according to the user's available bandwidth, which can reduce the required bandwidth resources to ensure the real-time transmission of frames and audio data that the user cares about. How the edge server near the surveillance camera efficiently selects which frames are key frames and which frames are non-key frames is the key to achieving good performance in this embodiment. Video compression standards usually divide the encoded video frames into three categories: I frames, P frames and B frames. I frames are more difficult to compress and can be decompressed independently without other frames. They are generally called key frames. However, I frames are not key frames defined in this embodiment. I frames are similar to a background, and P frames and B frames are equivalent to making changes in the background. The existing method is to use the I frame as a reference benchmark, and use the target detection algorithm to obtain the target type and quantity of the frames before and after the I frame. When the target type and quantity in the frame are different from those in the I frame, the frame is defined as a key frame. In this embodiment, the video frame in which a moving target appears in the interval between two adjacent I frames is called a key frame. The change in definition can not only effectively avoid the misjudgment of the target type caused by the performance difference of the target detection algorithm, but also ensure that the frames between two adjacent I frames will not be discarded. This can ensure that the picture seen by the user is continuous and there will be no freeze problem.

[0063] Figure 3 is a schematic diagram of an optional compressed security monitoring video according to an embodiment of the present invention, such as Figure 3 As shown, the video file label transmitted after discarding all non-key frames is as follows Figure 3 As shown, it includes audio tag 3, image frame tag 6, image frame tag 7, audio tag 8 and audio tag 9.

[0064] Specifically, this embodiment explores the deep integration of transformer network and video compression algorithm based on the understanding of the essential characteristics of compressed video.

[0065] Figure 4 is a schematic diagram of an optional optical flow comparison between a compressed area and an original area according to an embodiment of the present invention, such as Figure 4 As shown, including (a) compressed area, (b) original area corresponding to optical flow and (c) compressed area corresponding to optical flow, Figure 4The optical flow comparison between the compressed area and the original area in the compressed video is shown. The video has two characteristics after compression: pixel loss and texture similarity. Grids of different colors are used to represent pixel blocks with different textures. By comparing the optical flow of the original area and the compressed area, it can be analyzed that the pixels in the original area have a natural transition and smooth changes. The compressed area has a strong segmentation, with obvious boundaries between different types of pixel blocks, and the same type of pixel blocks are more similar internally.

[0066] Therefore, based on the characteristics that the boundaries between different types of pixel blocks in the compressed area are obvious and the same type of pixel blocks are more similar, and taking into account the inapplicability of existing visual converter networks in video enhancement tasks, the algorithm in this embodiment believes that the process of indiscriminate global pixel relationship modeling for compressed videos contains a large amount of redundant calculations, and it is necessary to consider effectively combining the two, and propose the idea of ​​dynamic sampling modeling in the spatiotemporal domain of compressed videos, and then dynamically reconstructing them, so as to solve the inapplicability problem of existing visual converter networks in compressed video enhancement tasks due to the large amount of calculation.

[0067] This embodiment proposes a compressed video enhancement algorithm based on a transformer network (Transformer), which is called Compressed Videos Quality Enhancement Transformer (CeT for short). The entire model is built by a 3D convolutional neural network. The entire mechanism consists of three parts: a preprocessor module, a transformer network module, and a construction module. The transformer network module is composed of several self-attention modules. Each self-attention module includes a multi-head attention module and a feedforward module. The entire enhancement process first inputs 5 compressed frames into the preprocessor module to pre-extract features in the time domain and spatial domain. Then, the features are extracted and fused through 5 self-attention modules, and finally reconstructed in the construction module.

[0068] Since 3D convolutional neural networks have a large number of parameters and computational complexity, the goal of the feedforward module is to make the model as lightweight as possible while ensuring performance. First, this algorithm decomposes each ordinary 3D convolutional neural network layer using the idea of ​​P3D. The two decomposed convolutions act in the spatial domain and the temporal domain respectively. This structure greatly reduces the number of parameters and computational costs, while adopting a deeper structure to improve the nonlinear expression ability of the network. At the same time, the idea of ​​the applied bottleneck structure can effectively increase the network width, thereby further improving the performance of the network.

[0069] Video enhancement tasks use valid pixels to help restore invalid pixels. Therefore, it is important to utilize the contextual information of video frames in space and time. However, the convolution operation itself only captures local correlations, so many stacked layers can be used to capture the correlations between different parts of the entire image. How to more effectively capture long-distance dependencies between pixels is particularly critical for videos. 3D convolutional neural networks can effectively extract local features, while the converter network mechanism can extract global dependencies, and the two can complement each other.

[0070] The transformer network can enhance the entire video frame by directly feeding it into the network. However, due to the high-resolution characteristics of the model input, the existing transformer network module has a large amount of computation. Based on the characteristics that the boundaries between different types of pixel blocks in the compressed area are obvious and the same type of pixel blocks are more similar, this algorithm proposes a window association strategy. The network learns the mutual dependence between each window to form a sparse global feature matrix, and finally reconstructs it to the size of the input.

[0071] For super-resolution tasks, since a large amount of effective source information can be obtained with the increase in resolution, many super-resolution networks do not perform spatial downsampling in order to obtain better performance and avoid losing more information. Since the compressed video input in this embodiment loses a lot of source information, this algorithm uses the compressed video quality enhancement task as a special video super-resolution task (keeping the input and output resolutions the same) to eliminate this information gap. Specifically, the model does not use downsampling operations in the spatial domain, and all convolution steps are set to 1. This allows the network to maintain resolution without reducing features when learning features. This algorithm also conducts experiments to study the impact of spatial downsampling, which proves the negative effect of downsampling operations on network performance. The 3D convolutional neural network was chosen as a building block not only because it is a non-aligned network, but also because of its ability to express features in the time domain. This ability is why it has been widely used in pattern recognition in recent years.

[0072] Since the focus of compressed video enhancement is also on the effective use of inter-frame information, this embodiment is also considering how to make full use of the capabilities of 3D convolutional neural networks in the time domain when designing the network. In the initial and enhancement stages, the number of frame channels is maintained at 5, and the channels are not downsampled until the construction stage to ensure full fusion in the time domain. This embodiment compares the downsampling in the pre-extraction stage with the downsampling in the reconstruction stage in the time domain. Since the former loses too much information in the feature pre-extraction stage of the network, the performance of the network is greatly affected. This proves the effectiveness of downsampling in the time domain at a deep level.

[0073] Specifically, the research implements real-time high-rate compression technology for infrastructure monitoring videos, mainly to solve the problems of high-definition video quality, large network bandwidth occupation, and high wireless traffic charges when the power company performs fixed infrastructure monitoring and wireless backhaul of surveillance video. At the same time, the backhaul video is ensured to be clear enough and visually indistinguishable from the original video. The algorithm model is continuously upgraded in combination with the actual situation of the power company on site, and can be applied to the video content recognition work of the power company. Among them, the adaptive compression of video content is an important part of this embodiment.

[0074] According to the power company's infrastructure security video transmission scenario, the implementation process is as follows: conduct research on high-rate video compression algorithms, combine the basic principles of video transmission, use the asymmetric coding technology of images, and carry out research on high-rate video compression technology based on the attention mechanism encoding model, combined with the context extraction model and the quality enhancement model, and verify the feasibility and effect of the hybrid model. According to the requirements of the power company's infrastructure monitoring business, the model parameters are continuously adjusted to achieve a video compression ratio of more than 5 times; based on the existing algorithm model, targeted optimization is carried out, and research on a variety of hybrid artificial intelligence algorithm models and components, core parameter strategy algorithms is carried out to continuously improve the video quality level and compression degree, and develop an independent and autonomous video high-rate compression algorithm model suitable for the power company's safety monitoring video and on-site control ball monitoring video.

[0075] The R&D process is as follows: Model deployment and initialization: Set the neural network model to a convolutional neural network model, and the number of residual blocks is 64; Data preparation: The training set of the neural network model can select the inspection images of the power company as the training data set. The data set contains more than 3,800 uncompressed color images, including rich scenes and images of different sizes. During each training, a small image block of 256×256 is randomly intercepted for training. The test set uses more than 400 other images, and the safety and supervision recording video of the power company is used as the test set data. The parameter settings can be as follows: The selection of parameters N, n, and m is related to the bit rate: for high bit rate models, that is, λ∈{50,25,10,5}, this embodiment sets N to 256, m to 768, and n to 640; for low bit rate models, that is, λ∈{1000,500,200,100}, this embodiment sets N to 192, m to 640, and n to 512. In addition, in the densely connected residual block of the attention encoder-decoder network, c 0 Set to N,c 1 Set to 32. In the densely connected residual block of the quality enhancement network c 0 Set to 128, c 1 Set to 16.

[0076] Model training: The intra-frame compression algorithm with attention mechanism is used for model training, mainly to adjust the parameters of high bit rate model, low bit rate model and quality enhancement model. The optimizer selected in this experiment is the optimizer (Adam), the learning rate is 0.0001, the batch size is set to 8, and the number of iterations is set to 1000000.

[0077] Model testing: Use the prepared data test set for testing, mainly use the number of bits per pixel to measure the decoding rate after compression, use the Peak Signal to Noise Ratio (PSNR) to measure the degree of image distortion, and verify the effectiveness of attention mechanism coding, quality enhancement coding, and dense links. The calculation formula involved is as follows:

[0078] The calculation formula for the number of bits per pixel is:

[0079]

[0080] The peak signal-to-noise ratio (PSNR) is calculated as:

[0081]

[0082] In the formula, bit com It represents the number of bits of the compressed image; n represents the number of pixels; N represents the color depth of the image; MSE represents the mean square error; Represents the pixel value of the i-th point in the reconstructed image; x i Represents the pixel value of the i-th point in the real image.

[0083] Finally, network protocol conversion and development, and video encoding work are carried out for the wired network transmission environment and wireless network transmission environment of the power company. In order to ensure the real-time and image quality of the video network transmission of the monitoring and on-site surveillance in the power grid, the selection of the transport layer protocol is the key to the entire design and implementation. With sufficient bandwidth guarantee, the transport protocol (TCP) can also have a relatively good performance in meeting the real-time performance of video transmission. The video transmission performance of the transport protocol (TCP) and the transport protocol (RTP) based on the transport protocol (UDP) is not much different. However, when video data is transmitted in a wide area network, the transmission performance at this time depends greatly on the available bandwidth. Since the transport protocol (TCP) is a connection-oriented transport layer protocol, its retransmission mechanism and congestion control mechanism will further deteriorate the network condition, thereby causing catastrophic delays. Relatively speaking, the real-time and transmission quality of video data transmitted using the transport protocol (RTP) are much better. Based on the location of the video equipment of the power company and the network security requirements, the transport protocol (TCP) can be used for direct transmission when the network bandwidth is sufficient, and the transport protocol (RTP) can be used for transmission when the network bandwidth is general or poor.

[0084] Considering that the common format used by power companies for videos is H.264, and combined with the development trend of video, the final video output format is converted to H.264 and H.265 formats to facilitate access to other applications or platforms.

[0085] In addition, computing power orchestration and management technology during compression hardware acceleration: This research mainly solves the problem of limited size and power of equipment used and limited software computing power in infrastructure construction scenarios. It can ensure that when video compression is performed on low-computing power devices through artificial intelligence algorithms, high-quality compression tasks can be performed, and acceleration drive technology must be used during the compression encoding process.

[0086] In order to ensure smooth real-time transmission and timely response to video access in the video transmission scenario of the power company, the efficiency of hardware module usage in video encoding and transmission is improved, and the reasonable utilization rate of resources is improved. First, under the premise of being able to run real-time compression tasks normally, the reasonable computing power investment of processor (CPU) threads and the computing power investment when using the graphics processor unit (GPU) are studied, and the computing power is scheduled and arranged on the hardware. Under the same power consumption, the required computing power in the computing tasks is managed and allocated, on the one hand, to ensure the rapid completion of compression and decoding tasks, and on the other hand, to ensure the timeliness of data transmission. At the same time, maintenance management work that can be manually scheduled and configured by software is developed to effectively improve the stability and timeliness of the rapid completion and monitoring of video compression tasks in the safety and supervision operation monitoring scenario. Then, according to the actual work scenario application, problems and operation data are continuously collected to adjust the scheduling strategy algorithm. Summarize and iterate the optimal scheduling strategy for different scenarios and equipment. Ensure the rapid execution of compression tasks and stable and timely video transmission in the safety and supervision scenario of the power company. At the same time, this embodiment also includes building a video compression application for safety and surveillance scenarios based on previous research results, realizing application implementation, using the compression function of the edge computing prototype to achieve efficient and high-quality video return, and building a demonstration application.

[0087] Research on lightweight technology of video compression model to realize the operation of video compression model on low computing power devices; Convolutional neural network relies on tens of millions of network parameters in the neural network to participate in the calculation together, which has the disadvantages of complex network structure, large amount of calculation, slow speed, and is difficult to be transplanted to embedded devices. As the number of network model layers becomes deeper and deeper, and the number of parameters increases, it is crucial to reduce their size and computing loss. In order for the video compression algorithm model to run smoothly on the edge device, the algorithm needs to be lightweight. Here, the knowledge distillation method is used for lightweight processing. The main idea is to use a pre-trained network (usually with a complex structure and high accuracy) to guide the training of a small network and make the small network achieve an accuracy similar to that of a complex network. The big network is analogous to a teacher, and the small network is analogous to a student. After a long period of "training", the teacher has found a set of methods suitable for a certain task, so the method is refined into "knowledge" and taught to the student to help the student learn to handle similar tasks faster. The biggest difficulty in the whole idea is how to effectively express "knowledge" and effectively guide the training of the small network. Targeted improvements in training methods and parameter settings can improve the actual application effects of small models and enhance the applicability of large-scale compression algorithm models on low-computing power devices.

[0088] The prototype of edge video compression software and hardware is developed. For various transmission scenarios, the video compression all-in-one machine is connected to the original network video recorder, the safety monitoring video is obtained for compression, the compressed video is transmitted back to the NVR, and then transmitted back to the centralized monitoring center using the existing network facilities.

[0089] Develop integrated hardware and software equipment, and use compression edge terminals to provide compression services. In the wired transmission scenario, use hardware equipment with compression technology software installed, incorporate it into the transmission path of the front-end equipment in a simple and convenient way, directly provide compression algorithms, and transmit the compressed image video to the background. Carry out the development of hardware terminals equipped with compression algorithm functions. Develop and use domestic chips, memory, storage, and hardware terminals with peripheral interfaces to create a low-power, compact, lightweight, safe and stable hardware terminal near the front-end device side, which can access multiple video signals and perform compression processing, can achieve video enhancement, and can achieve video stream violation analysis. For example, when the power company's substation infrastructure construction video is transmitted to the centralized control station, the use of hardware terminals is more suitable for the intranet wired environment, independently provides computing power, does not interfere with the operation of other systems, is safer and more reliable, and has strong maintainability. Considering that the equipment can be equipped with artificial intelligence algorithm models, although it will be lightweight, it can still have stable and reliable computing power when accessing 16 surveillance videos; there are commonly used interfaces such as RJ45 network ports and USB to access the equipment, which is convenient for equipment operation and maintenance and debugging, and can also meet the deployment method of accessing hard disk recorders. In order to meet the subsequent expansion and development of operation and maintenance functions, there can be a data interface for platform docking.

[0090] To ensure compatibility with existing monitoring equipment, the device access requirements are as follows: Access protocol: support RTSP video stream access, GB / T28181, RS485; Decoding type: H265, H264; Access channel: support up to 16 channels of 1080p video streams.

[0091] The pilot application construction of video compression in typical infrastructure construction scenes aims to solve the problems of transmission jams, screen distortion, and inability to effectively support safety management in typical infrastructure construction videos. The technology of this embodiment is used to complete the deployment of edge video devices for construction sites, achieve efficient transmission of safety monitoring scene videos, support back-end intelligent recognition needs, realize safety situation awareness and judgment, and promptly warn of violations, ultimately reducing the occurrence of safety accidents.

[0092] According to another embodiment of the present application, there is provided an intelligent video adaptive compression system based on intelligent monitoring, including a target recognition module, a bit rate allocation module, a key frame detection module, a video quality enhancement module and an adaptive compression module; wherein the target recognition module is used to identify the dynamic area and the static area in the monitoring video based on the dynamic area extraction and multi-target tracking technology; the bit rate allocation module is used to distinguish the dynamic area from the static area by using the mixed Gaussian model, and allocate the bit rate to the dynamic area and the static area respectively based on the distinction result; the key frame detection module is used to define the key frame and the non-key frame, detect the key frame by using the key frame detection technology based on deep learning, and selectively transmit the key frame and the audio according to the available bandwidth of the user; the video quality enhancement module is used to perform quality enhancement processing on the transmitted compressed video by using the compressed video enhancement model, so as to improve the clarity of the target identification object under the same compression ratio; the adaptive compression module is used to dynamically adjust the compression parameters based on the pre-built video high-ratio compression model, combined with the real-time changes of the monitoring video content and the network conditions, and use the adjusted compression parameters to perform adaptive high-ratio compression on the real-time infrastructure video.

[0093] In summary, with the help of the above-mentioned technical solutions of the present application, the present application can effectively solve the problem of low video transmission efficiency through the application of dynamic area detection technology based on infrastructure monitoring, video key frame extraction and quality enhancement technology, and real-time adaptive high-rate compression technology of infrastructure monitoring video, improve the video transmission speed of infrastructure smart construction site scenes, reduce network bandwidth occupancy, shorten the defect identification cycle, reduce the impact of data transmission delays on background intelligent identification, and improve the technical level of safety management and control of infrastructure smart construction site scenes.

[0094] In addition, this application can be used at infrastructure construction sites. Based on the existing working mode, it provides new working methods, optimizes the existing working processes, uses real-time video compression to improve the efficiency of detecting violations, and promptly feeds back problems to relevant persons in charge, thereby improving the overall inspection efficiency. It can effectively alleviate the monitoring pressure of inspectors, improve the inspection utilization rate, and enhance the safety of the power grid. By combining video stream violation behavior and scene recognition technology, video real-time adaptive high-rate compression technology will significantly improve the efficiency of inspections, improve the accuracy of identifying faults and violations, and promptly discover hidden dangers and problems of substation equipment, providing a guarantee for the safe operation of the power grid.

[0095] In addition, through its widespread application in various business scenarios, this application can build a new intelligent and efficient video transmission model, which greatly improves the efficiency of the use of material resources, human resources and financial resources, and reduces the idleness and waste of resources. At the same time, the application and promotion of the system has improved the digitalization and intelligence of the grassroots, reduced the workload of front-line users, and improved the digital transformation of the power grid system.

[0096] Figure 5 is a schematic diagram of an optional edge computing-based compressed transmission of power safety monitoring video according to an embodiment of the present invention, such as Figure 5 As shown, it includes cloud and edge, wherein the cloud includes the network (Internet) and cloud servers, and the edge includes edge servers, user-side devices and smart devices, etc.

[0097] The real-time video stream processing architecture based on edge computing can effectively solve the problems related to cloud computing. Edge computing can transfer computing tasks from the cloud computing center located in the center of the network to the edge server or smart terminal device close to the location of the information sending device. When the cloud center loses its processing capacity due to special problems, the edge server can still perform data analysis and storage operations at the data sending or receiving end. In edge computing, the three levels of device, edge and cloud can calculate data according to task requirements.

[0098] At the device level, smart devices have certain computing power and can process video data at the video generation and reception ends; at the edge end, edge servers have higher computing power than smart devices and can process more and more complex tasks near the video generation and reception ends than smart devices; in the cloud, cloud servers only provide computing support and data fusion services when necessary. Since most of the video data has been processed on the device and the edge of the device, the upload bandwidth required for transmission will be greatly reduced. In addition, the edge server is close to the video generation end, and the time required for data packet transmission is greatly reduced compared to transmission to the cloud server. The video stream processing system using edge computing only needs to consider the transmission delay, and due to the reduction in computing power, the video stream processing system can support tasks such as augmented reality that are sensitive to delay.

[0099] This embodiment uses edge computing technology to process surveillance videos in real time. It detects bandwidth near users by relying on edge computing's low latency, reliable network, and strong privacy protection capabilities, and compresses videos near video sources by relying on its strong computing capabilities to reduce the pressure on bandwidth during transmission.

[0100] According to another aspect of an embodiment of the present invention, a compression and transmission device for electric power safety and monitoring videos is also provided. The device can execute the compression and transmission method for electric power safety and monitoring videos of the above embodiment. The specific implementation method and preferred application scenario are the same as those of the above embodiment and will not be repeated here.

[0101] Figure 6 is a schematic diagram of a compression transmission device for electric power safety monitoring video according to an embodiment of the present application, such as Figure 6 As shown, the apparatus includes the following: an acquisition module 602 , an identification module 604 , an allocation module 606 , a compression module 608 and a construction module 610 .

[0102] Among them, the acquisition module is used to acquire the electric power safety and supervision video that needs to be transmitted, wherein the electric power safety and supervision video is used to represent the electric power safety monitoring video, and the electric power safety and supervision video at least includes an image frame set and audio, and the image frame set includes at least the current image frame; the identification module is used to identify the dynamic area and the static area in the current image frame; the allocation module is used to allocate the bit rate to the current image frame based on the dynamic area and the static area, and obtain the bit rate allocation result, wherein the bit rate allocated to the dynamic area in the bit rate allocation result is greater than the bit rate allocated to the static area; the compression module is used to compress the image frame set based on the bit rate allocation result to obtain a compressed image frame set; the construction module is used to construct a compressed safety and supervision video based on the compressed image frame set and audio, and transmit the compressed safety and supervision video.

[0103] Among them, the recognition module is also used to compare the pixel value probability distribution of the target area in the current image frame with a preset probability density threshold to obtain a comparison result, wherein the pixel value probability distribution is used to represent the probability density distribution of the pixel values ​​of the target area; when the comparison result is that the pixel value probability distribution is greater than the preset probability density threshold, the target area is determined to be a static area; when the comparison result is that the probability distribution is less than or equal to the preset probability density threshold, the target area is determined to be a dynamic area.

[0104] Among them, the recognition module is also used to obtain the historical pixel value probability distribution of the target area, wherein the historical pixel value probability distribution is used to represent the probability density distribution of the pixel values ​​of the target area in at least one historical image frame in the image frame set, and at least one historical past image frame is before the current image frame; a preset probability density threshold is constructed based on the historical pixel value probability distribution.

[0105] Among them, the compression module is also used to obtain target key frames and non-target key frames in the image frame set, wherein the target key frames are used to represent image frames in which moving targets appear in the interval between two adjacent preset frames in the image frame set, and the non-target key frames are used to represent image frames other than the target key frames in the image frame set; based on the bit rate allocation result, the real-time broadband status, the target key frames and the non-target key frames, the image frame set is compressed to obtain a compressed image frame set, wherein the real-time broadband status is used to characterize the real-time data transmission rate of the transmitted compressed safety monitoring video.

[0106] Among them, the identification module is also used to determine the number of discarded non-target key frames based on the real-time broadband status; based on the discarded number, target key frames and non-target key frames, the image frame set is compressed by frame number to obtain an initial compressed image frame set; based on the bit rate allocation result, the initial compressed image frame set is compressed to obtain a compressed image frame set.

[0107] Among them, the construction module is also used to construct an initial compressed safety and surveillance video based on a compressed image frame set and audio; the initial compressed safety and surveillance video is quality enhanced based on a compressed video enhancement model to obtain a compressed safety and surveillance video, wherein the compressed video enhancement model is used to represent a network model that improves the clarity of target identification objects under the same compression ratio.

[0108] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0109] The above-mentioned memory may refer to a device inside a computer for storing data and programs, and may include memory, hard disk, etc., wherein the memory may be used to temporarily store running programs and data, the hard disk may be used to store programs and data for a long time, and the memory may be used to enable the computer to read and write data, and execute programs; the above-mentioned processor may be responsible for executing instructions in computer programs and performing data processing, and may be responsible for controlling and executing various operations, including arithmetic operations, logical operations, data transmission, etc.

[0110] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0111] The above-mentioned computer storage medium may refer to a medium in a computer memory used to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser disks, etc.; the stored program included in the computer-readable storage medium may be a set of instructions that can be recognized and executed by a computer, running on an electronic computer, and is an information tool that meets certain needs of people.

[0112] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0113] The above-mentioned computer program product may refer to a software program that has been written, tested and released and can be run on a computer or other device. The computer program product may include an application, an operating system, tool software, etc., which is used to implement specific functions or solve specific problems.

[0114] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0115] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can keep the data from being lost when the power is off, and can be used to store long-term data, such as operating systems, applications, and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0116] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0117] The above-mentioned computer program may refer to a collection of instructions used to tell a computer to perform a specific task or operation. A computer program may be written by a programmer using a specific programming language and may include algorithms, data structures, logic, and control flows. A computer program may be used for a variety of purposes, including application software, operating systems, and the like.

[0118] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for compressing and transmitting electric power safety monitoring video, characterized in that: include: Acquire a power safety monitoring video to be transmitted, wherein the power safety monitoring video is used to represent a power safety monitoring video, the power safety monitoring video at least includes an image frame set and audio, and the image frame set at least includes a current image frame; Identifying a dynamic area and a static area in the current image frame; Performing bit rate allocation on the current image frame based on the dynamic region and the static region to obtain a bit rate allocation result, wherein the bit rate allocated to the dynamic region in the bit rate allocation result is greater than the bit rate allocated to the static region; Compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set; Based on the compressed image frame set and the audio, a compressed safety and surveillance video is constructed, and the compressed safety and surveillance video is transmitted.

2. The method for compressing and transmitting electric power safety monitoring video according to claim 1, characterized in that: Identifying a dynamic area and a static area in the current image frame, comprising: Comparing the pixel value probability distribution of the target area in the current image frame with a preset probability density threshold to obtain a comparison result, wherein the pixel value probability distribution is used to represent the probability density distribution of the pixel values ​​of the target area; When the comparison result is that the pixel value probability distribution is greater than the preset probability density threshold, determining that the target area is the static area; When the comparison result is that the probability distribution is less than or equal to the preset probability density threshold, the target area is determined to be the dynamic area.

3. The method for compressing and transmitting electric power safety monitoring video according to claim 2, characterized in that: The method further comprises: Acquire a historical pixel value probability distribution of the target area, wherein the historical pixel value probability distribution is used to represent a probability density distribution of pixel values ​​of the target area in at least one historical image frame in the image frame set, and the at least one historical past image frame is before the current image frame; The preset probability density threshold is constructed based on the historical pixel value probability distribution.

4. The method for compressing and transmitting electric power safety monitoring video according to claim 1, characterized in that: Compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set includes: Acquire a target key frame and a non-target key frame in the image frame set, wherein the target key frame is used to represent an image frame in which a moving target appears in an interval between two adjacent preset frames in the image frame set, and the non-target key frame is used to represent an image frame in the image frame set other than the target key frame; Based on the bit rate allocation result, the real-time broadband status, the target key frame and the non-target key frame, the image frame set is compressed to obtain the compressed image frame set, wherein the real-time broadband status is used to characterize the real-time data transmission rate of transmitting the compressed safety monitoring video.

5. The method for compressing and transmitting electric power safety monitoring video according to claim 4, characterized in that: Based on the bit rate allocation result, the real-time broadband state, the target key frame and the non-target key frame, compressing the image frame set to obtain the compressed image frame set includes: Determine the discarded number of the non-target key frames to be discarded based on the real-time broadband state; Based on the discarded number, the target key frame and the non-target key frame, compressing the image frame set by frame number to obtain an initial compressed image frame set; The initial compressed image frame set is subjected to code rate compression based on the code rate allocation result to obtain the compressed image frame set.

6. The method for compressing and transmitting electric power safety monitoring video according to claim 1, characterized in that: Based on the compressed image frame set and the audio, a compressed safety surveillance video is constructed, and the compressed safety surveillance video is transmitted, including: Based on the compressed image frame set and the audio, construct an initial compressed safety surveillance video; The initial compressed safety surveillance video is subjected to quality enhancement processing based on a compressed video enhancement model to obtain the compressed safety surveillance video, wherein the compressed video enhancement model is used to represent a network model for improving the clarity of target identification objects under the same compression ratio.

7. A compression transmission device for electric power safety monitoring video, characterized in that: include: An acquisition module, used for acquiring a power safety monitoring video to be transmitted, wherein the power safety monitoring video is used to represent a power safety monitoring video, the power safety monitoring video at least includes an image frame set and audio, and the image frame set at least includes a current image frame; An identification module, used for identifying a dynamic area and a static area in the current image frame; an allocation module, configured to allocate a bit rate to the current image frame based on the dynamic region and the static region to obtain a bit rate allocation result, wherein the bit rate allocated to the dynamic region in the bit rate allocation result is greater than the bit rate allocated to the static region; A compression module, used for compressing the image frame set based on the bit rate allocation result to obtain a compressed image frame set; A construction module is used to construct a compressed security video based on the compressed image frame set and the audio, and transmit the compressed security video.

8. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein when the program is run, the method for compressing and transmitting electric power safety monitoring videos as described in any one of claims 1 to 6 is executed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the compression and transmission method of the power safety monitoring video according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the method for compressing and transmitting electric power safety monitoring videos according to any one of claims 1 to 6.

Images