Real-time image video compression method for dynamic frame screening

Abstract

The invention discloses a real-time image video compression method for dynamic frame screening, and relates to the technical field of data compression and transmission, and the method comprises the steps: obtaining a power transmission line monitoring video stream, carrying out the importance analysis of a plurality of continuous frames through a multi-dimensional importance evaluation index, and generating an importance index; network bandwidth and computing power information are monitored in real time, and a screening threshold value is dynamically adjusted; and positioning the key frame and the redundant frame according to the comparison of the importance index and the screening threshold. The key frame is preferentially transmitted after being encoded by a preset encoder, and the encoder retains complete image information; and the redundant frame is subjected to lightweight compression or skipping processing through the differential processing module, and is transmitted after the key frame is transmitted. The technical problem that the stability and the real-time performance of video transmission are affected due to insufficient network bandwidth and equipment computing power resources in the prior art is solved, and the technical effects of reducing unnecessary data transmission and improving the real-time performance and the stability of video transmission are achieved.

Description

Technical Field

[0001] The present application relates to the technical field of data compression and transmission, and in particular to a real-time image and video compression method with dynamic frame screening. Background Art

[0002] In the monitoring scenario of power transmission lines, the demand for real-time video data transmission is gradually increasing, especially in remote monitoring and maintenance. The contradiction between bandwidth and computing power in the transmission process is becoming more and more prominent. How to reasonably allocate transmission and processing resources under limited bandwidth and computing resources to ensure the timely transmission of high-priority key frames has become a key link in transmission line monitoring. Traditional video compression and transmission methods usually rely only on fixed compression algorithms or rules, and cannot dynamically respond to changes in the importance of different scenes in the video stream, resulting in low transmission efficiency, failure to prioritize transmission of important information, and unreasonable use of bandwidth, further exacerbating network congestion problems. In existing methods, the network environment, computing power conditions and dynamic changes in video content are not fully combined for real-time optimization, making it difficult to cope with changing monitoring needs and network conditions.

[0003] At present, relevant technologies have the problem that the data volume of transmission line monitoring video stream is large, which easily leads to insufficient network bandwidth and equipment computing resources, thus affecting the stability and real-time performance of video transmission. Summary of the invention

[0004] The present application provides a real-time image video compression method with dynamic frame screening, which obtains the transmission line monitoring video stream to solve the technical problem in the prior art that the network bandwidth and equipment computing power resources are insufficient, thereby affecting the stability and real-time performance of video transmission.

[0005] The present application provides a real-time image and video compression method for dynamic frame screening, comprising: Acquire a transmission line monitoring video stream to be transmitted, perform importance analysis on multiple continuous frames in the transmission line monitoring video stream to be transmitted according to multi-dimensional importance evaluation indicators, and generate multiple importance indicators; monitor the network bandwidth information and computing power information of the preset network in real time to perform screening threshold control and generate a preset screening threshold; compare the multiple importance indicators with the preset screening threshold, locate key frames and redundant frames, wherein the key frames are frames whose importance indicators are greater than or equal to the preset screening threshold, and the redundant frames are frames whose importance indicators are less than the preset screening threshold; input the key frames into a preset encoder for encoding and then transmit them preferentially, wherein the preset encoder adopts an encoding algorithm that retains complete image information; input the redundant frames into a differential processing module for differential compression processing, and transmit the redundant frames after differential compression processing after the key frames are transmitted, wherein the differential compression processing includes lightweight compression and skip processing.

[0006] The real-time image and video compression method with dynamic frame screening proposed in this application first obtains the transmission line monitoring video stream, performs importance analysis on multiple continuous frames through multi-dimensional importance evaluation indicators, and generates importance indicators; monitors network bandwidth and computing power information in real time, and dynamically adjusts the screening threshold; locates key frames and redundant frames based on the comparison between the importance index and the screening threshold, wherein key frames are frames with higher importance and redundant frames are frames with lower importance. Key frames are preferentially transmitted after being encoded by a preset encoder, and the encoder retains complete image information; while redundant frames are lightweight compressed or skipped through a differential processing module, and are transmitted after the key frames are transmitted, thereby achieving the technical effect of reducing unnecessary data transmission and improving the real-time and stability of video transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0008] Figure 1 A flowchart of a real-time image and video compression method with dynamic frame screening provided in an embodiment of the present application; Figure 2 A schematic diagram of the importance index generation process of a real-time image video compression method with dynamic frame screening provided in an embodiment of the present application. DETAILED DESCRIPTION

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0010] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0011] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0012] The present application embodiment provides a real-time image video compression method with dynamic frame screening, such as Figure 1 As shown, the method includes: Step S100, obtain the transmission line monitoring video stream to be transmitted, perform importance analysis on multiple continuous frames in the transmission line monitoring video stream to be transmitted according to the multi-dimensional importance evaluation index, and generate multiple importance indicators. Specifically, after obtaining the transmission line monitoring video stream to be transmitted, importance analysis is performed to generate importance indicators. For image information entropy, by statistically analyzing the probability of pixel value distribution and using formula calculation, the information entropy is large when uniformly distributed, and abnormal light and shadow changes will make the information entropy different from normal frames; in terms of motion complexity, the first frame motion complexity is initialized to 1, and the subsequent frames select the reference frame set, calculate the motion vector set through block matching and other algorithms, extract the motion vector with the largest modulus value, and determine the motion complexity according to the preset model and database; the degree of scene change is extracted by feature extraction algorithm such as SIFT feature points, and the matching of feature points of adjacent frames is calculated to quantify the scene change. Finally, a weighted fusion method is adopted to set weights for information entropy, motion complexity and scene change degree, and calculate the importance index of each frame through the formula, providing a quantitative basis for subsequent frame screening and processing.

[0013] In a possible implementation, a transmission line monitoring video stream to be transmitted is obtained, and importance analysis is performed on multiple continuous frames in the transmission line monitoring video stream to be transmitted according to a multi-dimensional importance evaluation index to generate multiple importance indicators. Step S100 further includes step S110, and the multi-dimensional importance evaluation index at least includes information entropy, motion complexity and scene change degree. Specifically, when evaluating the transmission line monitoring video stream, information entropy, motion complexity and scene change degree are important multi-dimensional importance evaluation indicators. Information entropy measures the uncertainty of picture information by calculating the probability of image pixel value distribution. The occurrence of situations such as line faults generating arcs in the picture will increase the information entropy; motion complexity is used to measure the intensity of object movement between frames. After the first frame is initialized, the subsequent frames calculate the motion vector set and extract the maximum modulus motion vector through a block matching algorithm, and the complexity value is determined according to a preset model and a database; the scene change degree uses a feature extraction algorithm to extract feature points and perform adjacent frame matching, and the scene change is quantified according to the matching result. Weather changes, equipment additions and subtractions, etc. will cause scene changes. The combination of these three indicators can comprehensively and accurately evaluate the importance of each frame and provide a basis for subsequent frame screening, compression and transmission.

[0014] In a possible implementation, a transmission line monitoring video stream to be transmitted is obtained, and importance analysis is performed on multiple consecutive frames in the transmission line monitoring video stream to be transmitted according to a multi-dimensional importance evaluation index to generate multiple importance indicators, such as Figure 2 As shown, step S100 further includes step S120, calculating the information entropy of the plurality of continuous frames to generate a plurality of information entropies. Specifically, the plurality of continuous frames of the transmission line monitoring video stream to be transmitted are sequentially taken out for analysis. For each frame of the image, the value range of its pixel value is first determined. For example, for a common 8-bit grayscale image, the pixel value range is 0-255. At the same time, a counter is set for each possible pixel value, and the initial value is 0. Each pixel point of the image is scanned row by row and column by column, and the number of occurrences of each pixel value is recorded. For example, in a certain frame of the image, after scanning, it is found that the pixel value of 10 appears 20 times, and the counter corresponding to the pixel value of 10 is increased by 20. When the scanning of the entire frame of the image is completed, After scanning, the frequency data of each pixel value in the frame image is obtained, and the probability of occurrence of each pixel value is calculated based on the obtained frequency data. For example, the total number of pixels in a frame image is 1000, and the pixel value of 20 appears 50 times, then the probability of occurrence of this pixel value is 50÷1000=0.05. Then, using the principle of the information entropy calculation formula, the probability of each pixel value is multiplied by the negative of its logarithm with base 2 and then accumulated to obtain the information entropy of the frame image. Repeat this process to calculate the information entropy for each consecutive frame, thereby generating multiple information entropies.

[0015] Step S130 , performing inter-frame motion difference analysis on the plurality of consecutive frames respectively to generate a plurality of motion complexities. Specifically, the motion complexity of the first frame image at the first position is set to 1, and it is recorded as initial data. Then, subsequent analysis is performed starting from the second frame image, and a reference frame is selected according to a preset forward frame interval. For example, the preset forward frame interval is 2. When analyzing the second frame image, the first frame image is selected as the reference frame. When analyzing the third frame image, the second frame and the first frame image are selected as the reference frame set, and so on. For each current frame, it is compared with the frames in the reference frame set, and the motion situation is determined by analyzing the position change of the object or area in the image. For example, a block matching algorithm can be used to divide the image into several small blocks, and the area with the highest matching degree with the current frame block is searched in the reference frame. The motion vector is calculated according to the position change of the block. After the motion vectors of all small blocks are calculated, the motion vector with the largest modulus value is found, and the motion complexity of the current frame is determined according to the preset motion complexity assignment rule. The rule can be based on a pre-established motion vector and complexity correspondence database. The corresponding complexity value is determined according to the interval where the modulus value of the maximum motion vector is located. The above steps are repeated for each consecutive frame to obtain multiple motion complexities.

[0016] Step S140, performing inter-frame scene change analysis on the multiple continuous frames respectively, and generating multiple scene change degrees. Specifically, the scene change degree of the first frame image is initialized to 1, stored in the scene change degree data set, and subsequent analysis is performed starting from the second frame image, and relevant frames are extracted according to the preset forward frame interval to perform feature analysis. For the brightness features of different areas in the image, the average brightness value of each area is calculated. For example, the image is divided into multiple small areas, and the average brightness value of all pixels in each small area is calculated. For color features, a color histogram can be used to count the number of pixels in each color channel in different intervals. For edge features, an edge detection algorithm (such as Sobel operator, etc.) is used to obtain edge information, and the number, direction and other features of the edge are recorded. The current frame is compared with the previous frames in terms of brightness, color, edge and other features. For example, in terms of brightness, if the average brightness value of a region in the current frame changes more than a certain threshold compared with the previous frame, the brightness change of the region is recorded. For color features, the degree of color change is determined by comparing the differences in color histograms. For edge features, if the number, direction or position of edges has changed significantly, corresponding quantitative records are also made. The changes in brightness, color, edges, etc. of each region are combined, and weighted summation is performed according to the preset weight distribution method (for example, brightness change accounts for 40%, color change accounts for 40%, and edge change accounts for 20%) to obtain the scene change degree value of the current frame relative to the previous frame. The above steps are repeated for each consecutive frame to generate multiple scene change degrees.

[0017] Step S150, weighted fusion of the multiple information entropies, the multiple motion complexities and the scene change degrees to generate the multiple importance indicators. Specifically, according to the focus of power transmission line monitoring and the importance of each evaluation indicator, a weight value is determined for the information entropy, motion complexity and scene change degree respectively. For example, if more attention is paid to the information content in the image and the stability of the scene, relatively high weights may be assigned to the information entropy and the scene change degree; if the motion of the object in the line has a greater impact on the monitoring result, a higher weight is assigned to the motion complexity. For each continuous frame, the corresponding information entropy, motion complexity and scene change degree are multiplied by their respective weight values, and then the products are added. For example, the information entropy of a frame image is A, the motion complexity is B, and the scene change degree is C. The weights assigned to them are W1, W2, and W3 respectively, then the importance index of the frame image I=A×W1+B×W2+C×W3. Such calculations are performed on all continuous frames to generate multiple importance indicators and complete the comprehensive evaluation of the importance of each frame.

[0018] In a possible implementation, the multiple consecutive frames are respectively subjected to inter-frame motion difference analysis to generate multiple motion complexities, and step S130 further includes step S131, initializing the first motion complexity of the first frame image that is at the top of the multiple consecutive frames to 1, and adding it to the multiple motion complexities. Specifically, the first motion complexity of the first frame image that is at the top of the multiple consecutive frames is initialized to 1, and added to the multiple motion complexity sets, mainly to establish a starting benchmark for subsequent calculations. Since the first frame is used as the starting point, an initial value is assigned to it in the absence of comparative motion conditions of the preceding frames to facilitate the subsequent process, and because the first frame has an important reference value in the subsequent key frame positioning, it is usually defaulted to a key frame. Assigning an initial value of 1 helps to determine a relatively basic standard in the overall motion complexity evaluation system.

[0019] Step S132, extracting a second frame image other than the first frame image from the plurality of continuous frames, and a second frame image set that satisfies a preset forward frame interval with the second frame image. Specifically, extracting a second frame image other than the first frame image from the plurality of continuous frames, preparing to perform motion complexity analysis on the second frame image, the motion complexity calculation is gradually advanced, starting from the second frame, more information can be obtained based on the forward frame interval to accurately evaluate its motion situation, and extracting a second frame image set related to the second frame image according to the preset forward frame interval, for example, setting the preset forward frame interval to 4 frames, if the video stream is a continuous 100-frame image, for the second frame image, its image set may include the first frame and other related frames before the second frame (depending on the algorithm's definition rules for related frames), including the first frame image can provide a more comprehensive reference when calculating the motion vector, which helps to more accurately evaluate the motion change of the current frame, and the forward frame interval is set to improve the accuracy of key frame positioning, if only adjacent frames are analyzed, key frames may be misjudged due to inconspicuous motion changes in a short period of time. For example, when an object is slowly accelerating, simply comparing the motion vectors of adjacent frames may not accurately identify the trend of the object's motion changes. However, by referring to multiple forward frames, the changes in motion can be captured more comprehensively, thereby accurately determining key frames and redundant frames.

[0020] Step S133, analyzing the motion vector set of the second frame image relative to each frame image in the second frame image set, and connecting the preset motion complexity assignment model to generate the second motion complexity of the second frame image, and adding it to the multiple motion complexities. Specifically, the motion vector set of the second frame image relative to each frame image in the second frame image set is analyzed, and the motion vector is calculated by comparing the pixel difference between the second frame image and each frame in the image set, using algorithms such as block matching, etc., dividing the image into several small blocks, searching the reference frame for the area that best matches the current block, thereby obtaining the motion vector of the block, and summarizing to form a motion vector set. This process can quantify information such as the motion displacement and direction of objects or picture areas between frames, and connect the obtained motion vector set to a preset motion complexity assignment model. The model determines the motion complexity based on pre-set rules and databases. For example, the database in the model stores the correspondence between different motion vector features and motion complexity. If certain features extracted from the motion vector set (such as the maximum motion vector modulus, the average motion vector direction change, etc.) meet a specific range of conditions in the database, the motion complexity of the second frame image can be determined based on the correspondence. The generated second motion complexity is added to multiple motion complexity sets to complete the evaluation and recording of the motion complexity of the second frame image.

[0021] Step S134, and so on, continue to obtain the third frame image to calculate the third motion complexity, until the multiple continuous frames are traversed to generate the multiple motion complexities. Specifically, according to the same method, the third frame image is obtained in turn and the above steps are repeated to calculate the third motion complexity, and all continuous frames are traversed backward. Each time a frame is processed, the operation of extracting the relevant frame set, analyzing the motion vector set, determining the motion complexity through the model and adding it to the set is repeated. In a step-by-step manner, the motion complexity of each frame in the video stream is evaluated, and finally multiple motion complexities covering all continuous frames are generated, so that the frames in the entire video stream have quantitative evaluation indicators in terms of the degree of motion change, which provides an important basis for distinguishing key frames from redundant frames in the subsequent process, and helps to reasonably allocate resources and operate according to the importance of frames during data transmission and processing.

[0022] In one possible implementation, the motion vector set of the second frame image relative to each frame image in the second frame image set is analyzed, and a preset motion complexity assignment model is connected to generate a second motion complexity of the second frame image, which is added to the multiple motion complexities. Step S133 further includes step S1331, extracting the motion vector with the largest modulus value in the motion vector set, connecting the preset motion complexity assignment model, calling the complexity assignment database to filter the complexity identification sample corresponding to the motion vector with the largest modulus value, and generating the second motion complexity. Specifically, after analyzing the frame image and the reference frame set to obtain the motion vector set, this set contains a lot of vector information about the movement of objects or regions between images. Each motion vector has direction and size (modulus) attributes, reflecting the displacement change of the corresponding part of the image between frames. Extracting the motion vector with the largest modulus from the vector is a key step, because in many cases, the object or region corresponding to the motion vector with the largest modulus has the most significant motion change, which can represent the main motion characteristics of the entire frame image to a certain extent. For example, in a power line monitoring video, if the line swings due to strong winds, the motion vector modulus corresponding to the part with the largest swing amplitude is likely to be the largest. The motion information of this part plays an important role in judging the importance of the frame. By traversing all the vectors in the motion vector set and comparing their modulus values, the vector with the largest modulus value can be selected, which provides core data for the subsequent connection of the preset motion complexity assignment model, so that the complexity evaluation can be focused on the most representative motion information.

[0023] Step S1332, wherein the complexity assignment database includes multiple groups of assignment samples, each group of assignment samples includes a motion vector interval sample and a complexity identification sample. Specifically, the preset motion complexity assignment model is a mechanism for determining motion complexity according to motion vector information, and the complexity assignment database, as an important support for the model, stores multiple groups of assignment samples. The motion vector interval samples in each group of samples specify a certain range of motion vector modulus value intervals, and the complexity identification samples correspond to the motion complexity level or value of the corresponding interval. For example, a motion vector interval sample is [5-10] (modulus value unit), and the corresponding complexity identification sample is "medium complexity" or a quantized value such as 3 (assuming that the complexity value range is 1-5, the larger the value, the higher the complexity). After extracting the motion vector with the largest modulus value, the model uses the motion vector as input and performs a screening operation in the complexity assignment database. Specifically, the maximum modulus motion vector is compared with each motion vector interval sample to determine the interval to which it belongs. Once the corresponding interval is found, the complexity identification sample associated with the interval is obtained. The complexity information represented by this sample is the basis for the motion complexity generated for the current frame. For example, if the extracted maximum modulus motion vector is 8, by searching in the database, it is found that it falls within the interval of [5-10], then the corresponding "medium complexity" identification or value 3 is called as the second motion complexity of the current frame. In this way, the conversion from raw motion vector data to motion complexity with clear semantics and quantitative meaning is achieved, providing an accurate data basis for subsequent operations such as distinguishing key frames and redundant frames.

[0024] Step S200, real-time monitoring of the network bandwidth information and computing power information of the preset network to adjust the screening threshold and generate a preset screening threshold. Specifically, through network monitoring tools such as SNMP, the port traffic data is read from the network device and calculated and processed to obtain accurate network bandwidth information. At the same time, the utilization rate, frequency and other parameters of the device CPU, GPU, etc. are obtained with the help of the system or third-party software and integrated into a comprehensive computing power evaluation value. A bandwidth-computing power threshold mapping table is pre-constructed, and the bandwidth interval and computing power interval are divided according to experimental data and simulation analysis, and the screening threshold is set accordingly. Match the real-time monitored bandwidth and computing power information with the mapping table, and find the corresponding screening threshold according to the interval in which it is located, so as to dynamically adjust the screening threshold according to the real-time changes in network bandwidth and computing power, and adapt to different environments to achieve reasonable frame screening and transmission.

[0025] In a possible implementation, the network bandwidth information and computing power information of the preset network are monitored in real time to perform screening threshold control and generate a preset screening threshold. Step S200 further includes step S210, obtaining a bandwidth-computing power threshold mapping table, wherein the bandwidth-computing power threshold mapping table includes multiple groups of bandwidth intervals, computing power intervals and corresponding screening thresholds with mapping relationships. Specifically, professional and technical personnel configure the bandwidth-computing power threshold mapping table through a large number of experiments and practical application experience. First, different network bandwidth situations are classified. The network bandwidth information covers key information such as the current available bandwidth, upload and download rates. For example, the bandwidth is divided into different intervals according to the numerical range. The low bandwidth interval may be set to less than 10Mbps, which is suitable for scenarios with poor network conditions; the medium bandwidth interval is 10-50Mbps, which can meet general video transmission needs; the high bandwidth interval is greater than 50Mbps, which can support the transmission of large amounts of data such as high-definition video. The division of each bandwidth interval has been tested and analyzed for data transmission capabilities in the actual network environment to ensure the rationality and accuracy of the classification. At the same time, the computing power information, that is, the computing power of the video processing device, is taken into account. The computing power information includes CPU utilization, GPU utilization, memory occupancy and other factors. For CPU utilization, different usage ratios reflect the busyness of the device in processing data; GPU utilization is crucial when tasks such as image processing are involved; memory occupancy affects the caching and processing speed of data. GPU utilization is crucial when tasks such as image processing are involved; memory occupancy affects the caching and processing speed of data. Different computing power intervals are divided according to different combinations and value ranges of these factors. For example, a low computing power interval may correspond to a situation where the CPU utilization rate is high, the GPU utilization rate is low, and the memory usage is high; a high computing power interval is a state where the CPU utilization rate is low, the GPU utilization rate is high, and the memory usage is reasonable. After dividing the bandwidth interval and computing power interval, a corresponding screening threshold is set for each group of intervals with a specific bandwidth and computing power combination. The screening threshold is determined based on experience and the requirements for video transmission and processing under different bandwidth and computing power conditions, aiming to ensure that a more optimized video frame screening effect can be achieved under different network and device conditions.

[0026] Step S220: input the network bandwidth information and the computing power information into the bandwidth-computing power threshold mapping table, obtain the bandwidth interval and computing power interval into which the network bandwidth information and the computing power information fall, and generate the preset screening threshold. Specifically, the network bandwidth information and computing power information monitored in real time are input into the bandwidth-computing power threshold mapping table. For the network bandwidth information, for example, the currently detected available bandwidth is 30Mbps, the upload rate is 5Mbps, and the download rate is 6Mbps. According to the bandwidth interval division standard, the bandwidth interval in which it is located is determined. For the computing power information, assuming that the current CPU utilization rate is 40%, the GPU utilization rate is 70%, and the memory occupancy is 60%, the computing power interval in which it is located is determined by comparing with the computing power interval division standard, and then the filtering threshold corresponding to the combination of these two intervals is found in the mapping table. Once the corresponding bandwidth interval and computing power interval combination is found in the mapping table, the corresponding filtering threshold is determined as the preset filtering threshold, and the filtering threshold is dynamically generated according to the real-time network and device status. When the network bandwidth is good and the computing power is sufficient, the filtering threshold may be appropriately increased, so that more frames are considered to be important frames for processing and transmission; when the network bandwidth is limited or the computing power is insufficient, the filtering threshold is reduced, and only key frames are processed preferentially to ensure the stability and smoothness of video transmission, while reasonably utilizing resources to improve the overall video monitoring and transmission efficiency.

[0027] In a possible implementation, the network bandwidth information and computing power information of the preset network are monitored in real time to perform screening threshold control and generate a preset screening threshold. Step S200 further includes step S230, obtaining compression feedback information of the video receiving end of the transmission line monitoring video stream to be transmitted, wherein the compression feedback information includes a video reconstruction quality index and a fluency index. Specifically, during the transmission of power line monitoring video, the video receiving end is responsible for collecting compressed feedback information. For the video reconstruction quality index, the receiving end obtains it by analyzing the received and decoded video. For example, an image quality assessment algorithm is used to calculate the image's peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and other indicators. PSNR measures image quality by calculating the difference between the corresponding pixels of the original image and the reconstructed image, while SSIM evaluates the similarity of images from three aspects: brightness, contrast and structure. These indicators are combined to obtain a quantitative value of the video reconstruction quality. In terms of the fluency index, the receiving end determines it by monitoring factors such as frame rate changes, the number of freezes and the duration of freezes during video playback. If the frame rate is stable, the number of freezes is small and the duration of freezes is short, the fluency is high; otherwise, the fluency is low. The compressed feedback information will be fed back to the sending end or the video processing center through the network for summary and analysis. Whether in a wired network or a wireless network environment, it is necessary to ensure the accuracy and timeliness of the feedback information transmission for subsequent effective processing.

[0028] Step S240, if the video reconstruction quality index or the fluency index does not meet the preset reconstruction quality constraint, a screening optimization instruction is generated. Specifically, the standards for video reconstruction quality and fluency are pre-set, that is, the preset reconstruction quality constraint. The constraint is determined according to the actual needs of transmission line monitoring and the user's expectations for video quality. For example, for the monitoring of some key components on the transmission line, the PSNR value of the video reconstruction quality may be required to be not less than a certain value, and the fluency requires the frame rate to be maintained at a certain level and the number of freezes cannot exceed a certain threshold. When the received video reconstruction quality index or fluency index does not meet the preset reconstruction quality constraint, the system will automatically generate a screening optimization instruction. If the video reconstruction quality index is lower than the required PSNR or SSIM value, or the frame rate in the fluency index is too low, freezes frequently, etc., the instruction generation mechanism will be triggered, which means that the current video transmission and compression effect fails to meet the expected standard and needs to be adjusted and optimized.

[0029] Step S250, optimize the preset screening threshold according to the preset step length according to the screening optimization instruction. Specifically, when the compression information fed back by the video receiving end shows that the video reconstruction quality or fluency does not meet the preset reconstruction quality constraint, the system will generate a screening optimization instruction. Then obtain the historical compression feedback record from the storage, covering information such as network bandwidth, computing power, screening threshold, video quality, etc. under different circumstances in the past. Use data analysis tools to analyze the changing relationship between the screening threshold and video quality, such as drawing a relationship curve and calculating related parameters. Determine the preset step length with the goal of meeting the preset reconstruction quality constraint, and optimize the step length value considering system stability and resource consumption factors. Then adjust the screening threshold according to the step length, evaluate the video transmission and compression after each adjustment, and continue to adjust and cycle according to the step length if the requirements are not met, while monitoring the changes in the network and computing power in real time, so as to adjust the step length or re-analyze the data according to the new situation until the video quality meets the standard, so as to achieve efficient, stable and high-quality video transmission and compression.

[0030] Step S260, wherein the configuration step of the preset step size includes: obtaining historical compression feedback records. Specifically, the system first collects historical compression feedback records during past video transmissions, which contain information such as video reconstruction quality indicators, fluency indicators, and the corresponding screening thresholds at different time periods, under different network and device conditions, and obtains a large amount of historical data as a basis for analysis by querying a database or storage file.

[0031] Step S270, analyzing the changing relationship between the screening threshold, video reconstruction quality and the screening threshold according to the historical compression feedback record. Specifically, analyzing the changing relationship between the screening threshold and the video reconstruction quality and fluency according to the historical compression feedback record, for example, by drawing a chart or performing a data regression analysis, observing how the video reconstruction quality and fluency change accordingly when the screening threshold is raised or lowered, and finding that when the screening threshold is too high, although key frames are processed preferentially, redundant frames are skipped too much or lightly compressed, resulting in the overall coherence and integrity of the video being affected, thereby reducing the reconstruction quality and fluency.

[0032] Step S280, based on the change relationship, the preset step length is generated with the goal that the optimized video reconstruction quality index and the fluency index meet the preset reconstruction quality constraint. Specifically, with the goal that the optimized video reconstruction quality index and the fluency index meet the preset reconstruction quality constraint, the preset step length is determined according to the change relationship obtained by analysis. If it is found that the video reconstruction quality and fluency are improved to a certain extent every time the screening threshold is reduced by a certain value, then the appropriate step length value is calculated according to the improvement range and the preset reconstruction quality constraint to be achieved. For example, after analysis, it is found that when the screening threshold is reduced by 0.1, the PSNR value of the video reconstruction quality is increased by 2dB, and the number of freezes in the fluency index is reduced by 1. To achieve the preset quality constraint, the PSNR value needs to be increased by 5dB and the number of freezes is reduced by 3. Then the appropriate step length value can be calculated accordingly, so as to gradually adjust the screening threshold until the preset reconstruction quality constraint is met. At this time, the threshold is too high and needs to be appropriately reduced. Accurate optimization is achieved through reasonable step length setting to improve the effect of video transmission and compression.

[0033] Step S300, compare the multiple importance indicators with the preset screening threshold, locate key frames and redundant frames, wherein the key frame is a frame whose importance indicator is greater than or equal to the preset screening threshold, and the redundant frame is a frame whose importance indicator is less than the preset screening threshold. Specifically, when locating key frames and redundant frames, the importance indicators of each frame obtained by multi-faceted analysis and fusion have been obtained, and the preset screening threshold is determined by using a mapping table by monitoring network bandwidth and computing power information. Starting from the first frame of the video stream, its importance index is compared with the preset screening threshold frame by frame. If the importance index of a frame is greater than or equal to the threshold, it is located as a key frame, which usually contains key information such as line failure moment and key action of equipment; if it is less than the threshold, it is a redundant frame, and its information change, motion amplitude or scene change is relatively inconspicuous. In this way, the two types of frames are accurately distinguished in the video stream, providing a basis for subsequent different processing strategies, so as to reasonably utilize resources to adapt to the network and equipment environment while ensuring the transmission of key information.

[0034] Step S400, the key frame is input into a preset encoder for encoding and then transmitted with priority, wherein the preset encoder adopts a coding algorithm that retains complete image information. Specifically, the preset encoder is constructed based on existing high-quality coding algorithms such as H.265. During encoding, in order to retain complete image information, the quantization parameters are finely adjusted, more bit resources are allocated to important areas such as key device identification, and advanced entropy coding technology is used. Data preprocessing is performed before the key frame is input into the encoder, such as converting the format and color space. In the encoding operation, the amount of data is reduced by intra-frame and inter-frame prediction, and appropriate quantization parameters are used when transforming and quantizing the residual data, and then entropy coding is used to generate a bit stream. The encoded key frame is set to high priority transmission, the priority identifier is marked at the network protocol level, the network device is processed with priority, and the flow control and congestion avoidance mechanisms are used to ensure reliable transmission. The receiving end can accurately decode and reconstruct the image because it has a decoder with the same algorithm, providing data support for transmission line monitoring.

[0035] Step S500, input the redundant frame into the differential processing module, perform differential compression processing, and transmit the redundant frame after differential compression processing after the key frame transmission is completed, wherein the differential compression processing includes lightweight compression and skip processing. Specifically, after being determined as a redundant frame, it will be sequentially input into the differential processing module through a data channel or queue and an identifier will be added for management. In the differential compression processing, lightweight compression includes reducing the resolution, such as proportionally downsampling the high-resolution redundant frame to reduce the number of pixels; compressing the color depth, from true color to index color to reduce storage space; using a simple and efficient coding algorithm to effectively compress repeated patterns, etc. When skipping, it is judged according to the importance index and system status. If the importance index is lower than the threshold and the network is tense, it is skipped, and the skipped frame is marked, and the receiving end uses interpolation or reference adjacent frames to fill it. After the key frame is transmitted, the redundant frame is transmitted in sequence, and the correct order is ensured according to the identifier during transmission. At the same time, the network bandwidth and device status are continuously monitored. When the bandwidth fluctuates or the device load changes, the transmission parameters such as rate and data packet size are dynamically adjusted to adapt to the environment to ensure smooth video stream transmission and reasonable use of resources.

[0036] In a possible implementation, the redundant frame is input into a differential processing module for differential compression processing, and the redundant frame after differential compression processing is transmitted after the key frame is transmitted, wherein the differential compression processing includes lightweight compression and skip processing, and step S500 further includes step S510, configuring a differential judgment threshold, wherein the differential judgment threshold is an importance index threshold for selecting lightweight compression or skip processing. Specifically, the differential judgment threshold must first be configured, and the threshold is an importance index threshold for selecting lightweight compression or skip processing. The setting needs to comprehensively consider multiple factors. On the one hand, a large amount of transmission line monitoring video data must be analyzed to understand the importance distribution of redundant frames in different scenarios. For example, by collecting video data under different weather conditions (sunny, rainy, windy, etc.), different time periods (daytime, nighttime), and different line operation states (normal operation, pre-fault, when a fault occurs, etc.), the information entropy, motion complexity, and scene change degree of the redundant frames are analyzed, so as to determine a suitable range as the threshold setting. At the same time, it is necessary to combine the current system resource situation and actual needs. If the network bandwidth is sufficient and the device computing power is strong, then the differentiation judgment threshold can be appropriately increased so that more redundant frames have the opportunity to be lightweight compressed instead of directly skipped, so as to improve the integrity of the video; on the contrary, if the network bandwidth is tight or the device computing power is limited, it is necessary to lower the threshold to reduce the amount of data processing and give priority to the transmission and processing of key frames and relatively important redundant frames. For example, in the transmission line fault monitoring scenario with high real-time requirements, in order to transmit key information to the monitoring center as soon as possible, the threshold may be lowered to skip more less important redundant frames.

[0037] Step S520, when the importance index of the redundant frame is less than the differential judgment threshold, the redundant frame is skipped. Specifically, when the importance index of the redundant frame is less than the differential judgment threshold, the system decides to skip the redundant frame, because under the current system resources and demand conditions, the information carried by the frame is considered to be relatively unimportant, and skipping it has little effect on the understanding of the overall video information. For example, if a frame in the video is mainly a static background sky, and its importance index is very low, lower than the set threshold, then the system will choose to skip the transmission and processing of this frame to save network bandwidth and device computing resources.

[0038] Step S530, when the importance index of the redundant frame is greater than or equal to the differential judgment threshold, the redundant frame is lightweight compressed. Specifically, when the importance index of the redundant frame is greater than or equal to the differential judgment threshold, the system will perform lightweight compression on the redundant frame, which means that the frame contains a certain degree of useful information and needs to be properly processed before transmission. For example, although most areas in the picture are static, a small part of the line components have slight movement or light changes, and its importance index meets the threshold requirement. The system will use lightweight compression methods, such as reducing resolution, compressing color depth, or using simple encoding algorithms to process the frame, while reducing the amount of data while retaining useful information as much as possible, and then transmit it, so as to balance video quality and system resource utilization efficiency, and realize reasonable disposal of redundant frames through threshold-based judgment and processing methods to adapt to different transmission line monitoring scenarios and system conditions.

[0039] The embodiment of the present application adopts the method of acquiring the video stream for monitoring the power transmission line, performing importance analysis on multiple continuous frames through multi-dimensional importance evaluation indicators, and generating importance indicators; real-time monitoring of network bandwidth and computing power information, and dynamically adjusting the screening threshold; positioning key frames and redundant frames based on the comparison between the importance indicators and the screening threshold, wherein key frames are frames with higher importance, and redundant frames are frames with lower importance. The key frames are preferentially transmitted after being encoded by a preset encoder, and the encoder retains the complete image information; while the redundant frames are lightly compressed or skipped through a differential processing module, and are transmitted after the key frames are transmitted, thereby achieving the technical effect of reducing unnecessary data transmission and improving the real-time and stability of video transmission.

[0040] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A real-time image video compression method with dynamic frame screening, characterized in that: include: Acquire a transmission line monitoring video stream to be transmitted, perform importance analysis on a plurality of consecutive frames in the transmission line monitoring video stream to be transmitted according to a multi-dimensional importance evaluation index, and generate a plurality of importance indicators; Real-time monitoring of the network bandwidth information and computing power information of the preset network to adjust the screening threshold and generate a preset screening threshold; Comparing the plurality of importance indicators with the preset screening threshold, locating key frames and redundant frames, wherein the key frames are frames whose importance indicators are greater than or equal to the preset screening threshold, and the redundant frames are frames whose importance indicators are less than the preset screening threshold; Inputting the key frame into a preset encoder for encoding and then transmitting it preferentially, wherein the preset encoder adopts an encoding algorithm that retains complete image information; The redundant frame is input into the differential processing module to perform differential compression processing, and after the key frame is transmitted, the redundant frame after differential compression processing is transmitted, wherein the differential compression processing includes lightweight compression and skip processing.

2. A real-time image video compression method with dynamic frame screening as claimed in claim 1, characterized in that: The multi-dimensional importance evaluation index includes at least information entropy, motion complexity and scene change degree.

3. A real-time image video compression method with dynamic frame screening as claimed in claim 2, characterized in that: According to the multi-dimensional importance evaluation index, importance analysis is performed on multiple continuous frames in the transmission line monitoring video stream to be transmitted to generate multiple importance indexes, including: Performing information entropy calculation on the multiple consecutive frames to generate multiple information entropies; Performing inter-frame motion difference analysis on the plurality of consecutive frames respectively to generate a plurality of motion complexities; performing inter-frame scene change analysis on the plurality of consecutive frames respectively to generate a plurality of scene change degrees; The multiple information entropies, the multiple motion complexities and the scene change degrees are weightedly fused to generate the multiple importance indicators.

4. A real-time image video compression method with dynamic frame screening as claimed in claim 3, characterized in that: Performing inter-frame motion difference analysis on the plurality of consecutive frames respectively to generate a plurality of motion complexities, including: Initialize the first motion complexity of the first frame image in the plurality of consecutive frames to 1, and add the first motion complexity to the plurality of motion complexities; Extracting a second frame image other than the first frame image and a second frame image set that meets a preset forward frame interval with the second frame image from the plurality of continuous frames; Analyze the motion vector set of the second frame image relative to each frame image in the second frame image set, connect to a preset motion complexity assignment model, generate a second motion complexity of the second frame image, and add it to the multiple motion complexities; And so on, continue to acquire the third frame image and calculate the third motion complexity until the multiple continuous frames are traversed to generate the multiple motion complexities.

5. A real-time image video compression method with dynamic frame screening as claimed in claim 4, characterized in that: Analyzing the motion vector set of the second frame image relative to each frame image in the second frame image set, and connecting a preset motion complexity assignment model to generate a second motion complexity of the second frame image, including: Extracting the motion vector with the largest modulus value from the motion vector set, connecting the preset motion complexity assignment model, calling the complexity assignment database to filter the complexity identification sample corresponding to the motion vector with the largest modulus value, and generating the second motion complexity; The complexity assignment database includes multiple groups of assignment samples, and each group of assignment samples includes a motion vector interval sample and a complexity identification sample.

6. A real-time image video compression method with dynamic frame screening as claimed in claim 1, characterized in that: Inputting the redundant frame into a differential processing module for differential compression processing includes: Configuring a difference judgment threshold, wherein the difference judgment threshold is an importance index threshold for selecting lightweight compression or skipping processing; When the importance index of the redundant frame is less than the difference judgment threshold, skipping the redundant frame; When the importance index of the redundant frame is greater than or equal to the difference judgment threshold, lightweight compression is performed on the redundant frame.

7. A real-time image video compression method with dynamic frame screening as claimed in claim 1, characterized in that: Real-time monitoring of the network bandwidth information and computing power information of the preset network to adjust the screening threshold and generate the preset screening threshold, including: Obtaining a bandwidth-computing power threshold mapping table, wherein the bandwidth-computing power threshold mapping table includes multiple groups of bandwidth intervals, computing power intervals, and corresponding screening thresholds having a mapping relationship; The network bandwidth information and the computing power information are input into the bandwidth-computing power threshold mapping table, the bandwidth interval and computing power interval into which the network bandwidth information and the computing power information fall are obtained, and the preset screening threshold is generated.

8. A real-time image video compression method with dynamic frame screening as claimed in claim 1, characterized in that: Also includes: Obtaining compression feedback information of a video receiving end of the power transmission line monitoring video stream to be transmitted, wherein the compression feedback information includes a video reconstruction quality index and a fluency index; If the video reconstruction quality index or the fluency index does not meet the preset reconstruction quality constraint, generating a screening optimization instruction; Optimizing the preset screening threshold according to the preset step size according to the screening optimization instruction; The step of configuring the preset step size includes: Get historical compression feedback records; Analyzing the relationship between the screening threshold, the video reconstruction quality and the screening threshold according to the historical compression feedback record; According to the change relationship, the preset step size is generated with the goal that the optimized video reconstruction quality index and smoothness index meet the preset reconstruction quality constraint.

Images