Video compression method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence and research and development, and relates to a video compression method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a target video frame in a to-be-compressed video stream, and generating a thermodynamic diagram according to the target video frame; dividing the thermodynamic diagram into a plurality of thermodynamic diagram macro blocks, and calculating a thermodynamic value of each thermodynamic diagram macro block; correspondingly determining a compression strategy of each thermodynamic diagram macro block according to the thermodynamic value of each thermodynamic diagram macro block; and compressing the target video frame according to the compression strategy of each thermodynamic diagram macro block. In addition, the invention also relates to a block chain technology, and data such as a target video frame and a thermodynamic diagram can be stored in a block chain. According to the invention, the problem of quality reduction of video key content in a dynamic video processing process can be solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and research and development technology, and in particular to a video compression method, apparatus, computer equipment and storage medium. Background Art

[0002] Video compression technology is widely used in various fields such as streaming media, video conferencing, and video surveillance in modern society, providing an effective solution for the transmission and storage of digital content. For example, in various businesses of enterprises, clients use streaming media as a carrier to support business information exchange; when handling affairs within the enterprise, network video is often used for meetings. All these production scenarios use video compression technology. However, the existing compression algorithms still have some shortcomings when processing dynamic video content. For example, when processing portrait videos, the existing compression algorithms often find it difficult to effectively retain details and edges, resulting in blurred or distorted face and skin details. Since users pay much more attention to portraits than other objects, the loss of details during the compression process makes users ultimately dissatisfied with the clarity of the portraits in the video.

[0003] Therefore, when processing dynamic video content, how to ensure the quality of key content in the video becomes an urgent problem to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video compression method, apparatus, computer equipment and storage medium, which can solve the problem of quality degradation of key video content during dynamic video processing.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a video compression method, which adopts the following technical solution:

[0006] Obtain a target video frame in a video stream to be compressed, and generate a heat map according to the target video frame;

[0007] Dividing the heat map into a plurality of heat map macroblocks, and calculating the heat value of each of the heat map macroblocks;

[0008] Determining compression strategies for the respective heat map macroblocks according to the heat values ​​of the respective heat map macroblocks;

[0009] The target video frame is compressed according to the compression strategy of each heat map macroblock.

[0010] In order to solve the above technical problems, the embodiment of the present application further provides a video compression device, which adopts the following technical solution:

[0011] An acquisition module, used for acquiring a target video frame in a video stream to be compressed, and generating a heat map according to the target video frame;

[0012] A calculation module, used for dividing the heat map into a plurality of heat map macro blocks, and calculating the heat value of each heat map macro block;

[0013] A determination module, configured to determine a compression strategy for each of the heat map macroblocks according to the heat values ​​of each of the heat map macroblocks;

[0014] A compression module is used to compress the target video frame according to the compression strategy of each heat map macroblock.

[0015] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, which adopts the following technical solution: the computer device includes a memory and a processor, the memory stores computer-readable instructions, and the processor implements the steps of the video compression method described in any one of the above when executing the computer-readable instructions.

[0016] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which adopts the following technical solution: the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the video compression method described in any one of the above items are implemented.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0018] The embodiment of the present application obtains the target video frame in the video stream to be compressed, generates a heat map according to the target video frame, and can reflect the attention or importance of each video content in the video frame by generating a heat map; by dividing the heat map into a number of heat map macroblocks and calculating the thermal value of each heat map macroblock, it is possible to further determine the video content in the video frame that requires higher compression quality; by determining the compression strategy of each heat map macroblock according to the thermal value of each heat map macroblock, it is possible to dynamically determine the compression rate and compression mode of each heat map macroblock; by compressing the target video frame according to the compression strategy of each heat map macroblock, it is possible to select a lower compression rate for macroblocks with high thermal values ​​to retain more details; for macroblocks with low thermal values, a higher compression rate is selected to reduce the size of the video stream, thereby achieving the goal of effectively reducing the size of the video stream while ensuring the quality of key content in the dynamic video processing process, and significantly improving the compression effect of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0021] Figure 2 is a flowchart of an embodiment of the video compression method of the present application;

[0022] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S201;

[0023] Figure 4 is a structural schematic diagram of an embodiment of the video compression device of the present application;

[0024] Figure 5 It is a structural diagram of an embodiment of a computer device of the present application. DETAILED DESCRIPTION

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0026] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0028] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0029] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0030] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0031] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0032] It should be noted that the video compression method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the video compression device is generally set in the server / terminal device.

[0033] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0034] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for video compression according to the present application. The video compression method comprises the following steps:

[0035] Step S201, obtaining a target video frame in a video stream to be compressed, and generating a heat map according to the target video frame;

[0036] In this embodiment, the electronic device (eg, Figure 1 The server / terminal device shown in the figure can obtain the video stream to be compressed through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 2G / 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0037] In this embodiment, the video stream to be compressed refers to video data that has not been compressed, usually in the form of continuous frames. The video stream can be captured in real time (such as camera input) or read from a file or network stream. The target video frame refers to a specific frame selected from the video stream to be compressed. Among them, the target video frame can be selected based on a timestamp, a frame number or other specific conditions (such as based on motion detection, based on face detection, etc.). A heat map is a data visualization tool for representing the data density or intensity of different areas in a two-dimensional space. In this embodiment, a heat map is used to represent the distribution of specific features of pixels in an image (such as brightness, color intensity, motion intensity, etc.). A heat map can represent the size of different values ​​by different color shades or other visual forms, thereby intuitively displaying the hot spots in the image.

[0038] Specifically, for the obtained video stream to be compressed, the video stream to be compressed is decomposed into a video frame sequence. The selection conditions of the target video frame are determined, such as timestamp, frame number, specific event trigger, etc., and one or more target video frames are determined from the video frame sequence according to the selection conditions in a traversal manner, and a heat map is generated according to the target video frame.

[0039] More specifically, a heat map is generated based on a target video frame. For example, an image processing algorithm is used to extract features from the target video frame, wherein the image processing algorithm used may be a Sobel operator, a Canny edge detection, and a convolutional neural network, etc. The extracted features may include but are not limited to pixel brightness, color intensity, motion vector, edge detection, etc. Based on the extracted features, the target video frame is converted into a heat map. Optionally, each pixel value in the heat map corresponds to the size of the eigenvalue, which is represented by color mapping (such as red for high eigenvalues ​​and blue for low eigenvalues).

[0040] In some optional implementations, reference Figure 3 The above-mentioned step of generating a heat map according to the target video frame may include the following steps:

[0041] Step S2011, preprocessing the target video frame to obtain a preprocessed video frame;

[0042] Specifically, the acquired target video frame is preprocessed to obtain a preprocessed video frame, for example, the target video frame is converted into a grayscale image to reduce the amount of calculation; a filter (such as a Gaussian filter, a median filter, etc.) is used to remove noise in the target video frame; the target video frame is enhanced by contrast adjustment, histogram equalization, etc.; and the image size of the target video frame is adjusted according to the input requirements of the convolutional neural network.

[0043] Step S2012, using a convolutional neural network to extract features from the preprocessed video frames to obtain a plurality of feature maps;

[0044] Specifically, the preprocessed video frame is input into a convolutional neural network (CNN), and several feature maps are extracted from the middle layer (such as the convolution layer or the pooling layer) of the convolutional neural network. The feature map contains feature information of different levels in the preprocessed video frame.

[0045] Step S2013, performing principal component analysis on each of the feature graphs to obtain a number of key feature vectors;

[0046] Specifically, principal component analysis is performed on each feature map. For example, the Eigen-CAM algorithm is used. First, principal component analysis (PCA) is performed on each feature map, which includes flattening the feature map, concatenating it into a feature matrix, and performing PCA dimensionality reduction on the feature matrix to obtain several key eigenvectors (i.e., principal components).

[0047] It should be noted that the Eigen-CAM algorithm is a method for improving the interpretability of convolutional neural network (CNN) models. The Eigen-CAM algorithm can find the most activated area of ​​the input image in the CNN intermediate layer feature map through the tomographic decomposition method.

[0048] Step S2014, calculating the weight value of each feature channel, and performing weighted summation on each of the key feature vectors according to the weight value to obtain a feature contribution matrix;

[0049] Specifically, the weight value of each feature channel is calculated. For example, in Eigen-CAM, the input feature map is subjected to global average pooling (GAP), and then the weight value of each feature channel is calculated. The weighted sum of each key feature vector is performed according to the calculated weight value to obtain a feature contribution matrix. Each element in the feature contribution matrix represents the feature contribution of the pixel at the corresponding position.

[0050] Step S2015: Map the feature contribution matrix to the target video frame to generate a heat map of the target video frame.

[0051] Specifically, the feature contribution matrix is ​​mapped to the target video frame. For example, the feature contribution matrix is ​​normalized so that the values ​​in the feature contribution matrix are within an appropriate numerical range (such as 0-255), and the normalized values ​​are mapped to a color space (such as a red, green, and blue RGB color model, an HSV color model, etc.) to generate a heat map for the target video frame.

[0052] The embodiment of the present application helps to remove interference information in the video frame by preprocessing the target video frame, so that the convolutional neural network can more accurately capture the key features in the target video frame; by using the convolutional neural network to extract features from the preprocessed video frame, it can automatically learn and extract information from different regions and levels in the video frame; the Eigen-CAM algorithm combining the principal component analysis (PCA) and class activation mapping (CAM) ideas is used to reduce the dimension of the feature map through PCA to extract the most critical feature vector, and then the Eigen-CAM algorithm is used to calculate the weight value of each feature channel, and each key feature vector is weighted and summed according to these weights to obtain a feature contribution matrix, which not only reduces feature redundancy and improves the efficiency of subsequent calculations, but also highlights the important features in the video frame, providing a reliable basis for generating high-quality heat maps. The heat map generated based on the Eigen-CAM algorithm can more intuitively display the feature contribution of different regions in the video frame, that is, the difference between the main object that the user is concerned about and the edge background picture.

[0053] Step S202, dividing the heat map into a plurality of heat map macro blocks, and calculating the heat value of each heat map macro block;

[0054] In this embodiment, the heat map macroblock is a sub-region of the heat map, that is, the entire heat map is divided into several smaller, uniform or non-uniform regions. These regions can be rectangular, square or other shapes. The thermal value is used to quantify the data intensity or density within the heat map macroblock. For each macroblock, the thermal value can be the average, maximum, minimum, median or other statistics of the values ​​of all points in the macroblock.

[0055] Specifically, determine the macroblock division method, which may include uniform division (such as dividing the heat map into several rectangular areas of the same size) and non-uniform division (such as division based on data distribution or a specific algorithm). According to the selected division method, the heat map is divided into several heat map macroblocks. For each heat map macroblock, extract the values ​​of all points in the heat map macroblock, and calculate the thermal value according to the values ​​of all points in each heat map macroblock.

[0056] In some optional implementations, the above-mentioned dividing the heat map into a plurality of heat map macroblocks may include the following steps:

[0057] Perform feature extraction on each pixel in the thermal map to obtain thermal features of each pixel;

[0058] Specifically, the heat map is composed of a two-dimensional array, in which each element (i.e., pixel) represents the heat value of the location. Each pixel in the heat map is traversed and the heat value of each pixel is read as the heat feature.

[0059] It should be noted that the thermal characteristics of each pixel read out have a wide range or exist in different dimensions. In order to ensure that the thermal characteristics of all pixels are within a unified and comparable range, the read thermal characteristics need to be further processed.

[0060] Mapping the thermal characteristics to a preset characteristic value range to obtain the thermal characteristic value of each pixel;

[0061] Specifically, the thermal characteristics are mapped to a preset characteristic value range to obtain the thermal characteristic value of each pixel. For example, the thermal characteristics can be mapped to a preset characteristic value range through linear transformation, logarithmic transformation, normalization, etc., where the characteristic value range can be [0, 255], to obtain the thermal characteristic value of each pixel.

[0062] Obtaining a preset characteristic threshold, comparing the thermal characteristic value with the characteristic threshold, and obtaining a comparison result;

[0063] Specifically, a preset characteristic threshold is obtained, and the characteristic threshold is used to divide the thermal characteristic value into different categories or regions. The thermal characteristic value of each pixel is compared with the characteristic threshold to obtain a comparison result. The comparison result can be a binary value (using 0 and 1 to indicate belonging to / not belonging to a specific region, respectively), and the comparison result can also be that the thermal characteristic value is greater than / equal to / less than the characteristic threshold.

[0064] Divide each pixel in the thermal map into regions according to the comparison result to obtain a thermal map after region division;

[0065] Specifically, each pixel in the heat map is divided into regions according to the comparison result. For example, if the comparison result is a binary value, a new two-dimensional array is created based on the heat map, wherein each element in the two-dimensional array represents the region to which the pixel at the corresponding position of the heat map belongs, and the value of each element in the two-dimensional array is determined according to the binary value corresponding to the comparison result, so as to obtain a two-dimensional array composed of binary values; the image region represented by each binary value is determined, such as 0 represents the key area of ​​the image, and 1 represents the background area of ​​the image, so as to obtain the heat map after region division.

[0066] According to the heat map after the area division, the macroblock size corresponding to each divided image area is obtained;

[0067] Specifically, according to the heat map after the region division, the macroblock size corresponding to each divided image region is obtained, where different image region types can correspond to different macroblock sizes. For example, a smaller macroblock size is selected for the key image region to retain more details and edges during video compression; a larger macroblock size is selected for the background image region to improve video compression efficiency.

[0068] The image area is divided into a plurality of heat map macroblocks according to the macroblock size.

[0069] Specifically, after determining the macroblock size of each image area in the heat map, the image area is divided into a number of heat map macroblocks according to the macroblock size.

[0070] The embodiment of the present application extracts features from each pixel in the thermal map and maps the thermal features to a preset feature value range, thereby eliminating the dimensional differences and range differences between the thermal features of different pixels, so that the thermal features of all pixels are within a unified and comparable range. By obtaining a preset feature threshold and comparing the thermal feature value with the feature threshold, the pixels in the thermal map can be divided into different regions. By obtaining the macroblock size corresponding to each divided image region based on the thermal map after region division, adaptive adjustment of the macroblock size can be achieved. For key image areas, selecting a smaller macroblock size can retain more details and edge information; for image background areas, selecting a larger macroblock size can improve video compression efficiency. This adaptive macroblock division method helps to improve video compression efficiency while ensuring image quality.

[0071] In some optional implementations, the heat map after the region division includes an image key region and an image background region, and the region division of each pixel in the heat map according to the comparison result to obtain the heat map after the region division may include the following steps:

[0072] If the comparison result is that the thermal characteristic value is greater than or equal to the characteristic threshold, the pixels corresponding to the thermal characteristic value are divided into the key area of ​​the image;

[0073] If the comparison result is that the thermal characteristic value is less than the characteristic threshold, the pixels corresponding to the thermal characteristic value are divided into the image background area.

[0074] Specifically, the thermal characteristic value of each pixel in the thermal map is traversed, and the thermal characteristic value is compared with the characteristic threshold. When the comparison result is that the thermal characteristic value is greater than or equal to the characteristic threshold, it means that the thermal intensity of the pixel is high, and the pixel corresponding to the thermal characteristic value is divided into the key area of ​​the image. When the comparison result is that the thermal characteristic value is less than the characteristic threshold, it means that the thermal intensity of the pixel is low, and the pixel corresponding to the thermal characteristic value is divided into the image background area. After performing image area division on each pixel in the thermal map, a thermal map after area division is obtained.

[0075] In some optional implementations, the thermal value of the above-mentioned thermal map macroblock includes a total thermal value and a thermal mean value, wherein the calculation formula of the total thermal value includes:

[0076]

[0077] The calculation formula of thermal mean includes:

[0078]

[0079] The heat map macroblock is a two-dimensional matrix, i refers to the row of the two-dimensional matrix, j refers to the column of the two-dimensional matrix, H t is the total thermal value of the heat map macroblock, is the thermal mean value of the heat map macroblock, H i，j is the thermal value of the i-th row and j-th column in the two-dimensional matrix.

[0080] Step S203, determining a compression strategy for each of the heat map macroblocks according to the heat value of each of the heat map macroblocks;

[0081] In this embodiment, compression strategies corresponding to different thermal value ranges are predefined, and the compression strategies may include but are not limited to lossless compression strategies, lossy compression strategies with different compression levels, content-based compression strategies, etc. The thermal values ​​of each thermal map macroblock are traversed, and the compression strategy of each thermal map macroblock is determined according to the thermal value.

[0082] In some optional implementations, the above-mentioned determining the compression strategy of each heat map macroblock according to the thermal value of each heat map macroblock may include the following steps:

[0083] Obtaining a preset thermal threshold, and comparing the thermal value of each of the thermal map macroblocks with the thermal threshold;

[0084] If the thermal value is greater than or equal to the thermal threshold, the compression strategy of the heat map macroblock corresponding to the thermal value is determined to be a lossless compression strategy;

[0085] If the thermal value is less than the thermal threshold, the compression strategy of the heat map macroblock corresponding to the thermal value is determined as a lossy compression strategy.

[0086] Specifically, one or more thermal thresholds are preset according to the characteristics of the thermal map data and the compression requirements. The thermal threshold is used to divide the thermal map macroblock into different thermal intervals. The thermal values ​​of each thermal map macroblock are traversed and the thermal value is compared with the thermal threshold. When the thermal value is greater than or equal to the thermal threshold, it means that the thermal characteristics of the thermal map macroblock are more significant and a higher image quality fidelity is required. Therefore, the compression strategy of the thermal map macroblock is determined to be a lossless compression strategy. The lossless compression strategy can retain all the details of the image to the greatest extent. When the thermal value is less than the thermal threshold, it means that the thermal characteristics of the thermal map macroblock are relatively weak and the requirements for image quality are relatively low. Therefore, the compression strategy of the thermal map macroblock is determined to be a lossy compression strategy. The lossy compression strategy can reduce the use of storage space to a certain extent.

[0087] The embodiment of the present application can significantly optimize storage efficiency by presetting a thermal threshold and selecting a lossless or lossy compression strategy according to the size of the thermal value. For macroblocks with higher thermal values, a lossless compression strategy can retain all details, although this may take up more storage space. However, for macroblocks with lower thermal values, a lossy compression strategy can reduce the use of storage space to a certain extent while maintaining sufficient image quality, because these macroblocks usually contain fewer details or have less impact on the overall quality of the image. In addition, the preset thermal threshold can be adjusted according to different application scenarios and requirements, which enables the algorithm to flexibly adapt to different video content and compression requirements, thereby providing the best compression effect and image quality. Based on the scheme of the present application, by dividing the thermal map macroblocks into different thermal intervals and selecting a compression strategy according to the interval, a single compression strategy is avoided in the entire image or video sequence, but customized processing is performed according to the thermal characteristics of each macroblock, meeting the user's clarity requirements for different objects of interest.

[0088] In some optional implementations, determining the compression strategy of the heat map macroblock corresponding to the thermal value as a lossy compression strategy may include the following steps:

[0089] Normalizing the thermal value to obtain a normalized thermal value;

[0090] A compression ratio is determined according to the normalized thermal value, and a lossy compression strategy is generated according to the compression ratio as a compression strategy for the thermal map macroblock corresponding to the thermal value.

[0091] Specifically, a normalized range of thermal values ​​is determined so that all thermal values ​​are mapped to a standard range (such as 0 to 1). When the thermal value of a thermal map macroblock is less than a thermal threshold, a normalization formula is used to convert the thermal value to a normalized thermal value. The normalization formula may be to subtract the minimum value of all thermal values ​​from the target thermal value, and then divide it by the difference between the maximum value and the minimum value of all thermal values, thereby mapping the target thermal value to a range of 0 to 1. Based on the normalized thermal value, a mapping relationship is established to determine the compression ratio. The mapping relationship may be linear, nonlinear, or based on a specific functional relationship. For example, a threshold may be set, and when the normalized thermal value is lower than the threshold, a linearly increasing compression ratio is used; when the thermal value is higher than the threshold, a higher compression ratio is used. A lossy compression strategy is generated based on the determined compression ratio as the compression strategy for the thermal map macroblock corresponding to the thermal value.

[0092] Exemplarily, when the thermal value of the thermal map macroblock is higher than the set threshold α, a lossless compression strategy is selected. The compression ratio CR can be determined by:

[0093]

[0094] If the heat map macroblock heat mean is equal to α, then At this time, the linear compression ratio CR is the maximum compression ratio CR max Equal to 1, that is, the lossless compression strategy is selected; if the thermal mean of the heat map macroblock is less than α, then At this time, the thermal mean is normalized to the range of [0, 1], and the linear compression ratio CR will linearly increase the compression ratio according to the decrease of the thermal mean.

[0095] Step S204: compress the target video frame according to the compression strategy of each heat map macroblock.

[0096] In this embodiment, the target video frame is pre-divided according to the macroblock size of the heat map macroblock at the same spatial position to obtain a video frame macroblock, wherein the spatial position, content and size of the video frame macroblock correspond to the heat map macroblock one by one. The compression strategy of each heat map macroblock is traversed, and the compression strategy may include but is not limited to information such as a serial number identifier, a compression algorithm, a compression ratio, and a quantization parameter. According to the compression strategy of each heat map macroblock, the video frame macroblock corresponding to the heat map macroblock is compressed, and the compressed video frame macroblocks are combined to form a complete video frame to complete the compression of the target video frame.

[0097] The embodiment of the present application adopts the above scheme, specifically by obtaining the target video frame in the video stream to be compressed, and generating a heat map according to the target video frame, so as to reflect the attention or importance of each video content in the video frame by generating a heat map; by dividing the heat map into a number of heat map macroblocks and calculating the thermal value of each heat map macroblock, it is possible to further determine the video content in the video frame that requires higher compression quality; by determining the compression strategy of each heat map macroblock according to the thermal value of each heat map macroblock, it is possible to dynamically determine the compression rate and compression mode of each heat map macroblock; by compressing the target video frame according to the compression strategy of each heat map macroblock, it is possible to select a lower compression rate for macroblocks with high thermal values ​​to retain more details; for macroblocks with low thermal values, a higher compression rate is selected to reduce the size of the video stream, thereby achieving the goal of effectively reducing the size of the video stream while ensuring the quality of key content in the dynamic video processing process, and significantly improving the compression effect of the video.

[0098] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned video stream to be compressed, target video frame, heat map and other information as well as other related data, the above-mentioned video stream to be compressed, target video frame, heat map and other information as well as other related data can also be stored in a blockchain node.

[0099] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0101] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0102] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a video compression device, and the embodiment of the device is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0103] like Figure 4 As shown, the video compression device 400 described in this embodiment includes: an acquisition module 401, a calculation module 402, a determination module 403 and a compression module 404. Among them:

[0104] An acquisition module 401 is used to acquire a target video frame in a video stream to be compressed, and generate a heat map according to the target video frame;

[0105] A calculation module 402 is used to divide the heat map into a plurality of heat map macroblocks and calculate the heat value of each heat map macroblock;

[0106] A determination module 403, configured to determine a compression strategy for each of the heat map macroblocks according to the heat values ​​of each of the heat map macroblocks;

[0107] The compression module 404 is used to compress the target video frame according to the compression strategy of each heat map macroblock.

[0108] In this embodiment, the video stream to be compressed refers to video data that has not been compressed, usually in the form of continuous frames. The video stream can be captured in real time (such as camera input) or read from a file or network stream. The target video frame refers to a specific frame selected from the video stream to be compressed. Among them, the target video frame can be selected based on a timestamp, a frame number or other specific conditions (such as based on motion detection, based on face detection, etc.). A heat map is a data visualization tool for representing the data density or intensity of different areas in a two-dimensional space. In this embodiment, a heat map is used to represent the distribution of specific features of pixels in an image (such as brightness, color intensity, motion intensity, etc.). A heat map can represent the size of different values ​​by different color shades or other visual forms, thereby intuitively displaying the hot spots in the image.

[0109] Specifically, the acquisition module 401 is used to obtain the video stream to be compressed, decompose the video stream to be compressed into a video frame sequence, determine the selection conditions of the target video frame, such as timestamp, frame number, specific event trigger, etc., use a traversal method to determine one or more target video frames from the video frame sequence according to the selection conditions, and generate a heat map according to the target video frame.

[0110] More specifically, a heat map is generated based on a target video frame. For example, an image processing algorithm is used to extract features from the target video frame, wherein the image processing algorithm used may be a Sobel operator, a Canny edge detection, and a convolutional neural network, etc. The extracted features may include but are not limited to pixel brightness, color intensity, motion vector, edge detection, etc. Based on the extracted features, the target video frame is converted into a heat map. Optionally, each pixel value in the heat map corresponds to the size of the eigenvalue, which is represented by color mapping (such as red for high eigenvalues ​​and blue for low eigenvalues).

[0111] Specifically, a heatmap macroblock is a sub-region of a heatmap, that is, the entire heatmap is divided into several smaller, uniform or non-uniform regions. These regions can be rectangular, square or other shapes. The thermal value is used to quantify the data intensity or density within the heatmap macroblock. For each macroblock, the thermal value can be the average, maximum, minimum, median or other statistics of the values ​​of all points in the macroblock.

[0112] Specifically, the calculation module 402 is used to determine the macroblock division method, which may include uniform division (such as dividing the heat map into a number of rectangular areas of the same size) and non-uniform division (such as division based on data distribution or a specific algorithm). According to the selected division method, the heat map is divided into a number of heat map macroblocks. For each heat map macroblock, the values ​​of all points in the heat map macroblock are extracted, and the thermal value is calculated according to the values ​​of all points in each heat map macroblock.

[0113] Specifically, the determination module 403 is used to predefine compression strategies corresponding to different thermal value ranges, and the compression strategies may include but are not limited to lossless compression strategies, lossy compression strategies with different compression levels, content-based compression strategies, etc. The thermal values ​​of each thermal map macroblock are traversed, and the compression strategy of each thermal map macroblock is determined according to the thermal value.

[0114] Specifically, the compression module 404 is used to pre-segment the target video frame according to the macroblock size of the heat map macroblock at the same spatial position to obtain a video frame macroblock, wherein the spatial position, content and size of the video frame macroblock correspond to the heat map macroblock one by one. The compression strategy of each heat map macroblock is traversed, and the compression strategy may include but is not limited to information such as a serial number identifier, a compression algorithm, a compression ratio, and a quantization parameter. According to the compression strategy of each heat map macroblock, the video frame macroblock corresponding to the heat map macroblock is compressed, and the compressed video frame macroblocks are combined to form a complete video frame to complete the compression of the target video frame.

[0115] The video compression device 400 of the embodiment of the present application obtains the target video frame in the video stream to be compressed, generates a heat map according to the target video frame, and can reflect the attention or importance of each video content in the video frame by generating a heat map; by dividing the heat map into a number of heat map macroblocks and calculating the thermal value of each heat map macroblock, it can further determine the video content in the video frame that requires higher compression quality; by determining the compression strategy of each heat map macroblock according to the thermal value of each heat map macroblock, it can dynamically determine the compression rate and compression mode of each heat map macroblock; by compressing the target video frame according to the compression strategy of each heat map macroblock, it can select a lower compression rate for macroblocks with high thermal values ​​to retain more details; for macroblocks with low thermal values, a higher compression rate is selected to reduce the size of the video stream, thereby achieving the goal of effectively reducing the size of the video stream while ensuring the quality of key content in the dynamic video processing process, and significantly improving the compression effect of the video.

[0116] In some optional implementations of this embodiment, the acquisition module 401 may include a preprocessing submodule, a first extraction submodule, an analysis submodule, a weighting submodule, and a generation submodule.

[0117] in:

[0118] A preprocessing submodule, used to preprocess the target video frame to obtain a preprocessed video frame;

[0119] A first extraction submodule, configured to extract features from the preprocessed video frame using a convolutional neural network to obtain a plurality of feature maps;

[0120] An analysis submodule, used for performing principal component analysis on each of the feature graphs to obtain a number of key feature vectors;

[0121] A weighting submodule is used to calculate the weight value of each feature channel, and perform weighted summation on each of the key feature vectors according to the weight value to obtain a feature contribution matrix;

[0122] A generation submodule is used to map the feature contribution matrix to the target video frame to generate a heat map of the target video frame.

[0123] Specifically, the preprocessing submodule is used to preprocess the acquired target video frame to obtain a preprocessed video frame. For example, the target video frame can be converted into a grayscale image to reduce the amount of calculation; a filter (such as a Gaussian filter, a median filter, etc.) is used to remove noise in the target video frame; the target video frame is enhanced by contrast adjustment, histogram equalization, etc.; and the image size of the target video frame is adjusted according to the input requirements of the convolutional neural network.

[0124] Specifically, the first extraction submodule is used to input the preprocessed video frame into a convolutional neural network (CNN) and extract several feature maps from the middle layer (such as the convolution layer or the pooling layer) of the convolutional neural network. The feature map contains feature information of different levels in the preprocessed video frame.

[0125] Specifically, the analysis submodule is used to perform principal component analysis on each feature map. For example, using the Eigen-CAM algorithm, each feature map is first subjected to principal component analysis (PCA) processing, which includes flattening the feature map, concatenating it into a feature matrix, and performing PCA dimensionality reduction on the feature matrix to obtain several key feature vectors (i.e., principal components).

[0126] It should be noted that the Eigen-CAM algorithm is a method for improving the interpretability of convolutional neural network (CNN) models. The Eigen-CAM algorithm can find the most activated area of ​​the input image in the CNN intermediate layer feature map through the tomographic decomposition method.

[0127] Specifically, the weighted submodule is used to calculate the weight value of each feature channel. For example, in Eigen-CAM, the input feature map is subjected to global average pooling (GAP), and then the weight value of each feature channel is calculated. The weighted sum of each key feature vector is performed according to the calculated weight value to obtain the feature contribution matrix. Each element in the feature contribution matrix represents the feature contribution of the pixel at the corresponding position.

[0128] Specifically, a submodule is generated to map the feature contribution matrix to a target video frame. For example, the feature contribution matrix is ​​normalized so that the values ​​in the feature contribution matrix are within a suitable numerical range (such as 0-255), and the normalized values ​​are mapped to a color space (such as a red, green, and blue RGB color model, an HSV color model, etc.) to generate a heat map for the target video frame.

[0129] The acquisition module 401 of the embodiment of the present application, by preprocessing the target video frame, helps to remove the interference information in the video frame, so that the convolutional neural network can more accurately capture the key features in the target video frame; by using the convolutional neural network to extract features from the preprocessed video frame, it can automatically learn and extract information from different regions and levels in the video frame; using the Eigen-CAM algorithm that combines the principal component analysis (PCA) and class activation mapping (CAM) ideas, PCA is used to reduce the dimension of the feature map and extract the most critical feature vector, and then the Eigen-CAM algorithm is used to calculate the weight value of each feature channel, and each key feature vector is weighted and summed according to these weights to obtain a feature contribution matrix, which not only reduces feature redundancy and improves the efficiency of subsequent calculations, but also highlights the important features in the video frame, providing a reliable basis for generating high-quality heat maps. The heat map generated based on the Eigen-CAM algorithm can more intuitively show the feature contribution of different regions in the video frame, that is, the difference between the main object and the edge background picture that the user is concerned about.

[0130] In some optional implementations of this embodiment, the calculation module 402 may include a second extraction submodule, a mapping submodule, a first comparison submodule, a first division submodule, an acquisition submodule, and a second division submodule.

[0131] The second extraction submodule is used to extract features from each pixel in the thermal map to obtain thermal features of each pixel;

[0132] A mapping submodule, used to map the thermal characteristics to a preset characteristic value range to obtain the thermal characteristic value of each pixel;

[0133] A first comparison submodule is used to obtain a preset characteristic threshold, compare the thermal characteristic value with the characteristic threshold, and obtain a comparison result;

[0134] A first division submodule, used to divide each pixel in the thermal map into regions according to the comparison result to obtain a thermal map after region division;

[0135] An acquisition submodule, used for acquiring the macroblock size corresponding to each divided image area according to the heat map after the area division;

[0136] The second division submodule is used to divide the image area into a plurality of heat map macroblocks according to the macroblock size.

[0137] Specifically, the heat map is composed of a two-dimensional array, in which each element (ie, pixel) represents the heat value of the position. The second extraction submodule is used to traverse each pixel in the heat map and read the heat value of each pixel as a heat feature.

[0138] It should be noted that the thermal characteristics of each pixel read out have a wide range or exist in different dimensions. In order to ensure that the thermal characteristics of all pixels are within a unified and comparable range, the read thermal characteristics need to be further processed.

[0139] Specifically, the mapping submodule is used to map the thermal characteristics to a preset characteristic value range to obtain the thermal characteristic value of each pixel. For example, the thermal characteristics can be mapped to a preset characteristic value range through linear transformation, logarithmic transformation, normalization, etc., wherein the characteristic value range can be [0, 255], to obtain the thermal characteristic value of each pixel.

[0140] Specifically, the first comparison submodule is used to obtain a preset feature threshold, which is used to divide the thermal feature value into different categories or regions. The thermal feature value of each pixel is compared with the feature threshold to obtain a comparison result. The comparison result can be a binary value (using 0 and 1 to represent belonging to / not belonging to a specific area, respectively), and the comparison result can also be that the thermal feature value is greater than / equal to / less than the feature threshold.

[0141] Specifically, the first division submodule is used to divide each pixel in the heat map into regions according to the comparison result. For example, if the comparison result is a binary value, a new two-dimensional array is created based on the heat map, wherein each element in the two-dimensional array represents the region to which the pixel at the corresponding position of the heat map belongs, and the value of each element in the two-dimensional array is determined according to the binary value corresponding to the comparison result to obtain a two-dimensional array composed of binary values; the image area represented by each binary value is determined, such as 0 represents the key area of ​​the image, and 1 represents the background area of ​​the image, to obtain the heat map after region division.

[0142] Specifically, the acquisition submodule is used to obtain the macroblock size corresponding to each divided image area according to the heat map after area division, where different image area types can correspond to different macroblock sizes. For example, a smaller macroblock size is selected for the key area of ​​the image to retain more details and edges during video compression; a larger macroblock size is selected for the background area of ​​the image to improve the video compression efficiency.

[0143] Specifically, the second division submodule is used to divide the image area into a number of heat map macroblocks according to the macroblock size after determining the macroblock size of each image area in the heat map.

[0144] The calculation module 402 of the embodiment of the present application can eliminate the dimensional differences and range differences between the thermal features of different pixels by extracting features from each pixel in the thermal map and mapping the thermal features to a preset feature value range, so that the thermal features of all pixels are within a unified and comparable range. By obtaining a preset feature threshold and comparing the thermal feature value with the feature threshold, the pixels in the thermal map can be divided into different regions. By obtaining the macroblock size corresponding to each divided image region according to the thermal map after region division, adaptive adjustment of the macroblock size can be achieved. For key image areas, selecting a smaller macroblock size can retain more details and edge information; for image background areas, selecting a larger macroblock size can improve video compression efficiency. This adaptive macroblock division method helps to improve video compression efficiency while ensuring image quality.

[0145] In some optional implementations of this embodiment, the first division submodule may include a key area division unit and a background area division unit.

[0146] A key area division unit, configured to divide the pixels corresponding to the thermal characteristic value into the key area of ​​the image if the comparison result is that the thermal characteristic value is greater than or equal to the characteristic threshold;

[0147] The background area division unit is used to divide the pixels corresponding to the thermal characteristic value into the image background area if the comparison result is that the thermal characteristic value is less than the characteristic threshold.

[0148] Specifically, the thermal characteristic value of each pixel in the thermal map is traversed, and the thermal characteristic value is compared with the characteristic threshold. The key area division unit is used to divide the pixel corresponding to the thermal characteristic value into the key area of ​​the image when the comparison result is that the thermal characteristic value is greater than or equal to the characteristic threshold, indicating that the thermal intensity of the pixel is high. The background area division unit is used to divide the pixel corresponding to the thermal characteristic value into the background area of ​​the image when the comparison result is that the thermal characteristic value is less than the characteristic threshold, indicating that the thermal intensity of the pixel is low. After image area division is performed on each pixel in the thermal map, a thermal map after area division is obtained.

[0149] In some optional implementations, the thermal value of the above-mentioned thermal map macroblock includes a total thermal value and a thermal mean value, wherein the calculation formula of the total thermal value includes:

[0150]

[0151] The calculation formula of thermal mean includes:

[0152]

[0153] The heat map macroblock is a two-dimensional matrix, i refers to the row of the two-dimensional matrix, j refers to the column of the two-dimensional matrix, H t is the total thermal value of the heat map macroblock, is the thermal mean value of the heat map macroblock, H i，j is the thermal value of the i-th row and j-th column in the two-dimensional matrix.

[0154] In some optional implementations of this embodiment, the determination module 403 may include a second comparison submodule, a first strategy determination submodule, and a second strategy determination submodule.

[0155] A second comparison submodule is used to obtain a preset thermal threshold value and compare the thermal value of each thermal map macroblock with the thermal threshold value;

[0156] A first strategy determination submodule, configured to determine the compression strategy of the heat map macroblock corresponding to the thermal value as a lossless compression strategy if the thermal value is greater than or equal to the thermal threshold;

[0157] The second strategy determination submodule is used to determine the compression strategy of the heat map macroblock corresponding to the thermal value as a lossy compression strategy if the thermal value is less than the thermal threshold.

[0158] Specifically, the second comparison submodule is used to preset one or more thermal thresholds according to the characteristics of the thermal map data and the compression requirements. The thermal threshold is used to divide the thermal map macroblock into different thermal intervals. The thermal values ​​of each thermal map macroblock are traversed and the thermal value is compared with the thermal threshold. The first strategy determination submodule is used to determine the compression strategy of the thermal map macroblock as a lossless compression strategy when the thermal value is greater than or equal to the thermal threshold, indicating that the thermal characteristics of the thermal map macroblock are more significant and require higher image quality fidelity. The lossless compression strategy can retain all the details of the image to the greatest extent. The second strategy determination submodule is used to determine the compression strategy of the thermal map macroblock as a lossy compression strategy when the thermal value is less than the thermal threshold, indicating that the thermal characteristics of the thermal map macroblock are relatively weak and the requirements for image quality are relatively low. The lossy compression strategy can reduce the use of storage space to a certain extent.

[0159] The determination module 403 of the embodiment of the present application can significantly optimize storage efficiency by presetting a thermal threshold and selecting a lossless or lossy compression strategy according to the size of the thermal value. For macroblocks with higher thermal values, a lossless compression strategy can retain all details, although this may take up more storage space. However, for macroblocks with lower thermal values, a lossy compression strategy can reduce the use of storage space to a certain extent while maintaining sufficient image quality, because these macroblocks usually contain fewer details or have less impact on the overall quality of the image. In addition, the preset thermal threshold can be adjusted according to different application scenarios and requirements, which enables the algorithm to flexibly adapt to different video content and compression requirements, thereby providing the best compression effect and image quality. Based on the solution of the present application, by dividing the thermal map macroblocks into different thermal intervals and selecting a compression strategy according to the interval, a single compression strategy is avoided in the entire image or video sequence, but customized processing is performed according to the thermal characteristics of each macroblock, meeting the user's clarity requirements for different objects of interest.

[0160] In some optional implementations of this embodiment, the second strategy determination submodule may further include a normalization unit and a strategy generation unit.

[0161] A normalization unit, used for normalizing the thermal value to obtain a normalized thermal value;

[0162] A strategy generation unit is used to determine a compression ratio according to the normalized thermal value, and to generate a lossy compression strategy according to the compression ratio as a compression strategy for the thermal map macroblock corresponding to the thermal value.

[0163] Specifically, the normalization unit is used to determine the normalization range of the thermal value so that all thermal values ​​are mapped to a standard range (such as 0 to 1). When the thermal value of the thermal map macroblock is less than the thermal threshold, the normalization formula is used to convert the thermal value into a normalized thermal value. The normalization formula can be to subtract the minimum value of all thermal values ​​from the target thermal value, and then divide it by the difference between the maximum value and the minimum value of all thermal values, so as to map the target thermal value to the range of 0 to 1. The strategy generation unit is used to establish a mapping relationship based on the normalized thermal value to determine the compression ratio. The mapping relationship can be linear, nonlinear, or based on a specific functional relationship. For example, a threshold can be set, and when the normalized thermal value is lower than the threshold, a linearly increasing compression ratio is adopted; when the thermal value is higher than the threshold, a higher compression ratio is adopted. A lossy compression strategy is generated according to the determined compression ratio as the compression strategy of the thermal map macroblock corresponding to the thermal value.

[0164] Exemplarily, when the thermal value of the thermal map macroblock is higher than the set threshold α, a lossless compression strategy is selected. The compression ratio CR can be determined by:

[0165]

[0166] If the heat map macroblock heat mean is equal to α, then At this time, the linear compression ratio CR is the maximum compression ratio CR max Equal to 1, that is, the lossless compression strategy is selected; if the thermal mean of the heat map macroblock is less than α, then At this time, the thermal mean is normalized to the range of [0, 1], and the linear compression ratio CR will linearly increase the compression ratio according to the decrease of the thermal mean.

[0167] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0168] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 with a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0169] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0170] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of a video compression method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0171] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running the video compression method.

[0172] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0173] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the video compression method as described above.

[0174] The computer device, computer-readable storage medium and computer-readable instructions provided in the embodiments of the present application obtain the target video frame in the video stream to be compressed through the processor execution, generate a heat map according to the target video frame, and can reflect the attention or importance of each video content in the video frame by generating a heat map; by dividing the heat map into a number of heat map macroblocks and calculating the thermal value of each heat map macroblock, it is possible to further determine the video content in the video frame that requires higher compression quality; by determining the compression strategy of each heat map macroblock according to the thermal value of each heat map macroblock, it is possible to dynamically determine the compression rate and compression mode of each heat map macroblock; by compressing the target video frame according to the compression strategy of each heat map macroblock, it is possible to select a lower compression rate for macroblocks with high thermal values ​​to retain more details; for macroblocks with low thermal values, a higher compression rate is selected to reduce the size of the video stream, thereby achieving the goal of effectively reducing the size of the video stream while ensuring the quality of key content during dynamic video processing, and significantly improving the compression effect of the video.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0176] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0177] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A video compression method, characterized in that: The steps include: Obtain a target video frame in a video stream to be compressed, and generate a heat map according to the target video frame; Dividing the heat map into a plurality of heat map macroblocks, and calculating the heat value of each of the heat map macroblocks; Determining compression strategies for the respective heat map macroblocks according to the heat values ​​of the respective heat map macroblocks; The target video frame is compressed according to the compression strategy of each heat map macroblock.

2. The video compression method according to claim 1, characterized in that: The step of generating a heat map according to the target video frame comprises: Preprocessing the target video frame to obtain a preprocessed video frame; Using a convolutional neural network to extract features from the preprocessed video frames to obtain a plurality of feature maps; Performing principal component analysis on each of the feature graphs to obtain a number of key feature vectors; Calculate the weight value of each feature channel, and perform weighted summation on each of the key feature vectors according to the weight value to obtain a feature contribution matrix; The feature contribution matrix is ​​mapped to the target video frame to generate a heat map of the target video frame.

3. The video compression method according to claim 1, characterized in that: The step of dividing the heat map into a plurality of heat map macroblocks comprises: Perform feature extraction on each pixel in the thermal map to obtain thermal features of each pixel; Mapping the thermal characteristics to a preset characteristic value range to obtain the thermal characteristic value of each pixel; Obtaining a preset characteristic threshold, comparing the thermal characteristic value with the characteristic threshold, and obtaining a comparison result; Divide each pixel in the thermal map into regions according to the comparison result to obtain a thermal map after region division; According to the heat map after the area division, the macroblock size corresponding to each divided image area is obtained; The image area is divided into a plurality of heat map macroblocks according to the macroblock size.

4. The video compression method according to claim 3, characterized in that: The heat map after region division includes an image key region and an image background region. The step of performing region division on each pixel in the heat map according to the comparison result to obtain the heat map after region division includes: If the comparison result is that the thermal characteristic value is greater than or equal to the characteristic threshold, the pixels corresponding to the thermal characteristic value are divided into the key area of ​​the image; If the comparison result is that the thermal characteristic value is less than the characteristic threshold, the pixels corresponding to the thermal characteristic value are divided into the image background area.

5. The video compression method according to claim 1, characterized in that: The thermal value of the thermal map macroblock includes a total thermal value and a thermal mean value. The calculation formula of the total thermal value includes: The calculation formula of the thermal mean value includes: The heat map macroblock is a two-dimensional matrix, i refers to the row of the two-dimensional matrix, j refers to the column of the two-dimensional matrix, H t is the total thermal value of the heat map macroblock, is the thermal mean value of the heat map macroblock, H i，j is the thermal value of the i-th row and j-th column in the two-dimensional matrix.

6. The video compression method according to claim 1, characterized in that: The step of determining the compression strategy of each heat map macroblock according to the heat value of each heat map macroblock includes: Obtaining a preset thermal threshold, and comparing the thermal value of each of the thermal map macroblocks with the thermal threshold; If the thermal value is greater than or equal to the thermal threshold, the compression strategy of the heat map macroblock corresponding to the thermal value is determined to be a lossless compression strategy; If the thermal value is less than the thermal threshold, the compression strategy of the heat map macroblock corresponding to the thermal value is determined as a lossy compression strategy.

7. The video compression method according to claim 6, characterized in that: The step of determining the compression strategy of the heat map macroblock corresponding to the thermal value as a lossy compression strategy includes: Normalizing the thermal value to obtain a normalized thermal value; A compression ratio is determined according to the normalized thermal value, and a lossy compression strategy is generated according to the compression ratio as a compression strategy for the thermal map macroblock corresponding to the thermal value.

8. A video compression device, characterized in that: The device comprises: An acquisition module, used for acquiring a target video frame in a video stream to be compressed, and generating a heat map according to the target video frame; A calculation module, used for dividing the heat map into a plurality of heat map macro blocks, and calculating the heat value of each heat map macro block; A determination module, configured to determine a compression strategy for each of the heat map macroblocks according to the heat values ​​of each of the heat map macroblocks; A compression module is used to compress the target video frame according to the compression strategy of each heat map macroblock.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the video compression method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video compression method according to any one of claims 1 to 7 are implemented.

Images