Video encoding method, device, electronic device and storage medium

Abstract

The present invention relates to the field of video processing technology, and provides a video encoding method, device, electronic device, and storage medium. The method comprises: first determining the target area where the target object is located in the video frame to be processed, and determining the positional relationship between all its macroblocks and the target area; then obtaining the two-dimensional heat information of the previous video frame of the video frame to be processed from the three-dimensional heat information to obtain reference two-dimensional heat information; then obtaining the two-dimensional heat information of the video frame to be processed based on the positional relationship and the reference two-dimensional heat information, and updating it to the three-dimensional heat information; finally, determining the quantization parameter value of each macroblock in the video frame to be processed based on the positional relationship and the two-dimensional heat information of the video frame to be processed, and encoding the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed. This achieves the rational allocation of encoding resources and improves the adaptability of video encoding to dynamically changing scenes.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video encoding method, device, electronic device and storage medium. Background Art

[0002] The Quantization Parameter (QP) value is a key parameter in video encoding and image processing, used to control the accuracy of the quantization process. Quantization is the process of converting a continuous, high-precision signal into a discrete, low-precision signal. In video encoding, the QP value determines the degree of compression and quality loss of video data. Currently, video encoding typically uses a fixed QP value, but this approach is not suitable for dynamically changing scenes and can result in poor video encoding performance and wasteful resources. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a video encoding method, apparatus, electronic device and storage medium.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, the present invention provides a video encoding method, the method comprising:

[0006] Taking each video frame in the video as a to-be-processed video frame in turn, determining a target region where a target object is located in the to-be-processed video frame, and determining a positional relationship between all macroblocks in the to-be-processed video frame and the target region;

[0007] Obtaining the 2D heat information of the video frame preceding the video frame to be processed from the 2D heat information of each historical video frame included in the 3D heat information to obtain reference 2D heat information; the 2D heat information indicates the importance of the image content corresponding to all macroblocks in the corresponding video frame;

[0008] Obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information;

[0009] According to the positional relationship and the two-dimensional heat information of the video frame to be processed, the quantization parameter value of each macroblock in the video frame to be processed is determined, and the video frame to be processed is encoded according to the quantization parameter value of each macroblock in the video frame to be processed.

[0010] In an optional embodiment, the two-dimensional heat information includes a heat value of each macroblock in the corresponding video frame, and the heat value of the macroblock represents the importance of the image content corresponding to the macroblock;

[0011] The step of obtaining the two-dimensional heat information of the video frame to be processed according to the position relationship and the reference two-dimensional heat information includes:

[0012] Determining a target indicator value for each macroblock in the to-be-processed video frame according to the positional relationship;

[0013] Determining a reference heat value of each macroblock in the to-be-processed video frame according to the reference two-dimensional heat information; wherein the reference heat value of the macroblock is the heat value of the macroblock at the same position as the macroblock in the reference two-dimensional heat information;

[0014] According to the target indication value and reference heat value of each macroblock in the video frame to be processed, the heat value of each macroblock in the video frame to be processed is calculated to obtain two-dimensional heat information of the video frame to be processed.

[0015] In an optional embodiment, the step of calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed includes:

[0016] For each macroblock in the video frame to be processed, a heat value of the macroblock is calculated according to a preset heat formula based on a preset time decay factor, a target indicator value of the macroblock, and a reference heat value, to obtain a heat value of each macroblock in the video frame to be processed; the heat formula is:

[0017] ;

[0018] in, Indicates the heat value of the macroblock in the i-th row and j-th column of the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at row i and column j in the video frame to be processed, Represents the target indicator value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0019] In an optional embodiment, the two-dimensional heat information of the video frame to be processed includes a heat value of each macroblock in the video frame to be processed; the position relationship includes whether each macroblock in the video frame to be processed intersects with the target area;

[0020] The step of determining the quantization parameter value of each macroblock in the video frame to be processed based on the positional relationship and the two-dimensional heat information of the video frame to be processed includes:

[0021] For each macroblock in the to-be-processed video frame, if the macroblock intersects with the target area, determining a quantization parameter value of the macroblock based on a heat value of the macroblock;

[0022] If the macroblock does not intersect with the target area, a reference quantization parameter value of the macroblock is obtained, and the quantization parameter value of the macroblock is determined based on the reference quantization parameter value of the macroblock; wherein the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock in the previous video frame of the video frame to be processed, which has the same position as the macroblock.

[0023] In an optional embodiment, the step of determining the quantization parameter value of the macroblock based on the heat value of the macroblock includes:

[0024] obtaining a reference adjustment value of the macroblock according to a type of a target object in a target area intersecting the macroblock;

[0025] A quantization parameter value of the macroblock is determined according to the heat value and the reference adjustment value of the macroblock.

[0026] In an optional embodiment, the step of determining the quantization parameter value of the macroblock according to the heat value and the reference adjustment value of the macroblock includes:

[0027] The quantization parameter value of the macroblock is calculated according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset thermal impact coefficient, the thermal value of the macroblock, and a reference adjustment value; the first adjustment formula is:

[0028] ;

[0029] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at row i and column j in the video frame to be processed, Indicates the thermal influence coefficient, Indicates the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0030] In an optional implementation manner, the step of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock includes:

[0031] According to a preset second adjustment formula, the quantization parameter value of the macroblock is calculated based on a preset basic quantization parameter value, a preset decay rate coefficient, and a reference quantization parameter value of the macroblock; the second adjustment formula is:

[0032] ;

[0033] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the decay rate coefficient; The reference quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed.

[0034] In a second aspect, the present invention provides a video encoding device, comprising:

[0035] a determination module, configured to sequentially use each video frame in the video as a to-be-processed video frame, determine a target region where a target object is located in the to-be-processed video frame, and determine a positional relationship between all macroblocks in the to-be-processed video frame and the target region;

[0036] a processing module configured to obtain, from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, the two-dimensional heat information of the video frame preceding the video frame to be processed, to obtain reference two-dimensional heat information; the two-dimensional heat information indicating the importance of image content corresponding to all macroblocks in the corresponding video frame;

[0037] Obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information;

[0038] An encoding module is used to determine the quantization parameter value of each macroblock in the video frame to be processed based on the position relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0039] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the video encoding method described in any one of the aforementioned embodiments is implemented.

[0040] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the video encoding method described in any one of the aforementioned embodiments is implemented.

[0041] The video encoding method, device, electronic device and storage medium provided by the embodiment of the present invention include: first, taking each video frame in the video as a to-be-processed video frame in turn, determining the target area where the target object is located in the to-be-processed video frame, and determining the positional relationship between all macroblocks in the to-be-processed video frame and the target area; then, obtaining the two-dimensional heat information of the previous video frame of the to-be-processed video frame from the two-dimensional heat information of each historical video frame contained in the three-dimensional heat information, and obtaining reference two-dimensional heat information; the two-dimensional heat information represents the importance of the image content corresponding to all macroblocks in the corresponding video frame; then, based on the positional relationship and the reference two-dimensional heat information, obtaining the two-dimensional heat information of the to-be-processed video frame, and updating the two-dimensional heat information of the to-be-processed video frame to the three-dimensional heat information; finally, based on the positional relationship and the two-dimensional heat information of the to-be-processed video frame, determining the quantization parameter value of each macroblock in the to-be-processed video frame, and encoding the to-be-processed video frame according to the quantization parameter value of each macroblock in the to-be-processed video frame. The embodiment of the present invention determines the positional relationship between the macroblock and the target area so as to reasonably allocate encoding resources according to the association between the two. In addition, three-dimensional heat information is used to reflect the changes of the target object in the time and space dimensions. At the same time, the two-dimensional heat information of adjacent video frames is used to determine the quantization parameter value of each macroblock, thereby improving the adaptability of video encoding to dynamically changing scenes and realizing adaptive control of video encoding quality.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0045] Figure 2 FIG1 shows one of the flow charts of the video encoding method provided by an embodiment of the present invention;

[0046] Figure 3 FIG2 shows a second flow chart of a video encoding method provided by an embodiment of the present invention;

[0047] Figure 4 A functional module diagram of a video encoding device provided by an embodiment of the present invention is shown.

[0048] Icons: 100 - electronic device; 110 - processor; 120 - memory; 130 - communication module; 300 - video encoding device; 310 - determination module; 330 - processing module; 350 - encoding module. DETAILED DESCRIPTION

[0049] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0051] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0052] The QP value (Quantization Parameter) is an important parameter in video coding and image processing, used to control the accuracy of the quantization process. Quantization is the process of converting a continuous or high-precision signal into a discrete or low-precision signal. In video coding, the QP value determines the degree of compression and quality loss of video data. Currently, a fixed QP value is usually used to encode videos, but this method is not suitable for dynamically changing scenes. Although there is also a method of encoding videos based on ROI (Region of Interest), it has the following problems: (1) the video is divided into fixed regions for processing, resulting in poor adaptability to dynamically changing scenes; (2) the video quality before and after the target appears is large and there is jitter, which affects the visual experience. (3) the allocation of encoding resources is not intelligent enough, resulting in resource waste. Therefore, the embodiment of the present invention provides a video encoding method to solve the above problems.

[0053] See also Figure 1 is a block diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes a processor 110, a memory 120, and a communication module 130. Each of these components is electrically connected to one another, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to one another via one or more communication buses or signal lines.

[0054] The processor 110 is used to read / write data or programs stored in the memory 120 and execute corresponding functions. It can be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it can also be a DSP digital signal processor, an ASIC application-specific integrated circuit, an FPGA off-the-shelf programmable gate array or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0055] The memory 120 is used to store programs or data, and can be RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0056] The communication module 130 is used to communicate signaling or data with other devices.

[0057] It should be noted that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0058] It is understood that, in one implementation, the electronic device of the embodiment of the present invention can receive and encode the video captured by the shooting device. In another implementation, the electronic device of the embodiment of the present invention can also integrate a shooting module and can capture and encode the video in real time.

[0059] The following will take the above-mentioned electronic device 100 as the execution entity to introduce the various steps in the various methods provided by the embodiments of the present invention and the corresponding technical effects.

[0060] See also Figure 2 , is a flow chart of a video encoding method provided by an embodiment of the present invention.

[0061] In step S202 , each video frame in the video is sequentially used as a video frame to be processed, a target region where the target object is located in the video frame to be processed is determined, and a positional relationship between all macroblocks in the video frame to be processed and the target region is determined.

[0062] Step S204, obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame contained in the three-dimensional heat information, and obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance of the image content corresponding to all macroblocks in the corresponding video frame.

[0063] Step S206 , obtaining the two-dimensional heat information of the video frame to be processed according to the position relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to three-dimensional heat information.

[0064] Step S208 , determining the quantization parameter value of each macroblock in the video frame to be processed according to the position relationship and the two-dimensional heat information of the video frame to be processed, and encoding the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0065] In this embodiment, for each video frame in the captured video, each video frame can be sequentially treated as a to-be-processed video frame and encoded to implement video encoding. That is, the processing method for each video frame in this embodiment of the present invention is similar. For the sake of simplicity, the following description uses the to-be-processed video frame as an example.

[0066] First, a preset detection algorithm or detection module can be used to perform target detection on the video frame to be processed. For example, a preset detection model such as the YOLOv7 lightweight detection model can be used to perform target detection on the video frame to be processed, thereby determining the area where the target object is located in the video frame to be processed, that is, obtaining the target area.

[0067] It is understandable that the embodiments of the present invention can be used not only for single-type target detection, but also for multi-type target detection. For example, the preset model can also output the confidence that the target object in the video frame to be processed belongs to various preset types, and the type with the highest confidence is used as the type of the target object. Among them, the target object can be understood as the image content that is focused on in the video screen. For example, in the monitoring scenario of a substation, the collected video is the monitoring video of the substation, then the target object can be a digital or pointer-shaped instrument, a switch device of the equipment, and personnel, etc.

[0068] Moreover, the size and number of macroblocks in a video frame can be determined based on the video coding standard used. Macroblocks refer to the basic unit of video coding. Dividing a video frame into multiple macroblocks means dividing a video frame into multiple independently coded areas. For example, assuming that the video coding standard used is H.264, the size of the macroblock is 16×16, that is, 256 pixels, and the width of the video frame is W and the height is H, then the number of macroblocks in a video frame is Moreover, the size and number of macroblocks in each video frame are the same, so the positional relationship between all macroblocks in the video frame to be processed and the target area can be determined according to the target area where the target object in the video frame to be processed is located.

[0069] Then, from the two-dimensional heat information of each historical video frame contained in the three-dimensional heat information, the two-dimensional heat information of the previous video frame of the video frame to be processed is obtained, that is, the reference two-dimensional heat information is obtained. Among them, the historical video frame refers to the video frame before the video frame to be processed. The two-dimensional heat information can be understood as a data structure constructed for a specific video frame, which represents the importance of the image content corresponding to each macroblock in the video frame. Three-dimensional heat information refers to a data structure constructed by spatiotemporally combining the two-dimensional heat information of multiple consecutive video frames, which contains the distribution of the target object in the time dimension and the spatial dimension. And a three-dimensional heat map can be used to realize data visualization of three-dimensional heat information.

[0070] Next, based on the positional relationship between all macroblocks in the processed video frame and the target region, and the importance of the image content corresponding to each macroblock in the previous video frame as indicated by the 2D heat information, the importance of the image content corresponding to each macroblock in the processed video frame can be determined, thereby obtaining the 2D heat information for the processed video frame. The 2D heat information of the processed video frame is then added to the 3D heat information to obtain the 2D heat information for the subsequent video frame.

[0071] Finally, based on the positional relationship and the two-dimensional heat information of the video frame to be processed, the quantization parameter value of each macroblock in the video frame to be processed is determined, and the video frame to be processed is encoded according to the quantization parameter value of each macroblock in the video frame to be processed, that is, the encoded video frame is obtained. In a similar manner, each video frame in the video is processed to obtain each encoded video frame, that is, the encoded video is obtained. It can be understood that the embodiment of the present invention uses different quantization parameter values ​​for encoding different areas in a video frame to compress the video picture to different degrees so that the quality of each area is different.

[0072] It can be understood that the embodiments of the present invention determine the positional relationship between macroblocks and target regions to rationally allocate coding resources based on their association. Furthermore, three-dimensional heat information is used to reflect the temporal and spatial changes of the target object, while two-dimensional heat information from adjacent video frames is used to determine the quantization parameter values ​​for each macroblock. This improves the adaptability of video coding to dynamically changing scenes and enables adaptive control of video coding quality.

[0073] Optionally, for the process of obtaining the two-dimensional heat information of the video frame to be processed according to the position relationship and the reference two-dimensional heat information in step S206, the embodiment of the present invention provides a possible implementation method, please refer to Figure 3 .

[0074] Step S206 - 1 : determining a target indicator value of each macroblock in the video frame to be processed according to the position relationship.

[0075] Step S206-2, determining the reference heat value of each macroblock in the video frame to be processed based on the reference two-dimensional heat information; wherein the reference heat value of the macroblock is the heat value of the macroblock at the same position as the macroblock in the reference two-dimensional heat information.

[0076] Step S206-3, calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and reference heat value of each macroblock in the video frame to be processed, and obtaining two-dimensional heat information of the video frame to be processed.

[0077] In this embodiment, the positional relationship includes whether each macroblock in the to-be-processed video frame intersects with the target region. Then, the target indicator value of each macroblock can be determined based on whether each macroblock in the to-be-processed video frame intersects with the target region.

[0078] For example, for each macroblock in a video frame to be processed, if the macroblock intersects with the target area, it indicates that there are pixels in the macroblock that represent the target object, and the target object is considered to exist in the macroblock. Then, the target indication value of the macroblock is set to a preset first value, such as 1. If the macroblock does not intersect with the target area, it indicates that there are no pixels in the macroblock that represent the target object, and the target object is considered to not exist in the macroblock. Then, the target indication value of the macroblock is set to a preset second value, such as 0. That is, the target indication value of the macroblock is used to indicate whether the target object exists in the macroblock.

[0079] Then, the two-dimensional heat information includes the heat value of each macroblock in the corresponding video frame, and the reference two-dimensional heat information includes the heat value of each macroblock in the previous video frame of the video frame to be processed. Then, the reference heat value of each macroblock in the video frame to be processed can be determined based on the reference two-dimensional heat information. For example, since the number of macroblocks in each video frame is the same, for each macroblock in the video frame to be processed, the heat value of the macroblock with the same position as the macroblock in the reference two-dimensional heat information can be used as the reference heat value of the macroblock to obtain the reference heat value of each macroblock in the video frame to be processed.

[0080] Finally, according to the target indication value and reference heat value of each macroblock in the video frame to be processed, the heat value of each macroblock in the video frame to be processed is calculated to obtain the importance of the image content corresponding to each macroblock in the video frame to be processed, that is, to obtain the two-dimensional heat information of the video frame to be processed.

[0081] Optionally, for the process of calculating the heat value of each macroblock in the video frame to be processed based on the target indication value and reference heat value of each macroblock in the video frame to be processed in step S206-5, an embodiment of the present invention provides a possible implementation method, namely: for each macroblock in the video frame to be processed, according to a preset heat formula, based on a preset time attenuation factor, the target indication value and reference heat value of the macroblock, the heat value of the macroblock is calculated to obtain the heat value of each macroblock in the video frame to be processed.

[0082] And the heat formula is:

[0083] ;

[0084] in, Indicates the heat value of the macroblock in the i-th row and j-th column of the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at row i and column j in the video frame to be processed, Represents the target indicator value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0085] It is understandable that the embodiment of the present invention is similar in the manner of calculating the heat value of each macroblock in the video frame to be processed. For the sake of brief description, one macroblock is used as an example for illustration below.

[0086] For example, a weighted fusion method can be used to calculate the heat value of a macroblock based on its target indicator value and reference heat value. First, the preset time attenuation factor The reference heat value of the macroblock Multiplying together, we get the first product and express it as ; Then, add a preset time decay factor Subtract the first difference value, and multiply the first difference value by the target indication value of the macroblock to obtain a second product, which is expressed as Finally, add the first product and the second product to get the heat value of the macroblock. .

[0087] It should be understood that since the first video frame of the video has no previous video frame, the heat formula for calculating each macroblock in the first video frame can be expressed as: ;in, Represents the heat value of the macroblock in the i-th row and j-th column in the first video frame; The target indicator value of the macroblock at row i and column j in the first video frame; Represents the time decay factor.

[0088] It should be noted that the time decay factor The value range is (0,1). And, the time decay factor Used to adjust the weight relationship between historical data and current data; when the time decay factor When it is close to 0, it means that the target indicator value of the macroblock in the currently processed video frame has a great influence on the heat value of the macroblock. In this case, this method is suitable for scenes that require a quick response to sudden targets. When it is close to 1, it indicates that the reference heat value obtained based on the historical video frame has a great influence on the heat value of the macroblock. In this case, this method is suitable for scenarios with high stability requirements.

[0089] For example, for substation monitoring, which requires high stability, the time decay factor can be Set to 0.95. Based on the above heat formula, it can be concluded that when a target object continues to exist in a macroblock at a certain position, the heat value of the macroblock at that position will gradually increase over time; if there is no target object at that macroblock, the heat value of the macroblock at that position will gradually decrease over time. This can make the change of the heat value of the macroblock smoother and avoid instantaneous fluctuations. It should be understood that the time decay factor The specific value of can be set according to the actual application scenario, and the embodiment of the present invention is not limited to this.

[0090] It can be understood that the embodiment of the present invention combines historical heat information with the existence status of the current target object through the time attenuation factor, so as to determine the importance of each area in the currently processed video frame in a spatiotemporal manner, thereby providing an important basis for the reasonable allocation of subsequent coding resources, so that coding resources can be more efficiently focused on the image content of concern.

[0091] Optionally, for the process of determining the quantization parameter value of each macroblock in the video frame to be processed according to the position relationship and the two-dimensional heat information of the video frame to be processed in step S208, an embodiment of the present invention provides a possible implementation method.

[0092] Step S208 - 1 : for each macroblock in the video frame to be processed, if the macroblock intersects with the target area, a quantization parameter value of the macroblock is determined based on the heat value of the macroblock.

[0093] Step S208-2: If the macroblock does not intersect with the target area, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock with the same position as the macroblock in the previous video frame of the video frame to be processed.

[0094] It will be appreciated that the embodiments of the present invention employ a similar method for calculating the quantization parameter value for each macroblock in the video frame to be processed. For simplicity, the following description uses a macroblock as an example. As previously mentioned, if a macroblock intersects with a target region, it is considered that a target object exists within the macroblock, and the macroblock can be classified as a target-containing macroblock. If the macroblock does not intersect with the target region, it is considered that no target object exists within the macroblock, and the macroblock can be classified as a target-free macroblock.

[0095] For targeted macroblocks, since they have image content that is of particular concern, their quantization parameter values ​​can be calculated using the heat value of the macroblock. It can be understood that, for macroblocks containing important image content, the embodiment of the present invention further subdivides the importance of the image content based on the heat value of the macroblock, so as to ensure that when encoding macroblocks with higher heat values, i.e., those with higher importance, the quality of the image can be guaranteed, and at the same time, when encoding macroblocks with lower heat values, i.e., those with lower importance, the encoding accuracy is appropriately reduced, thereby achieving dynamic adjustment of the quantization parameters and ensuring the reasonable allocation of encoding resources. Moreover, the heat values ​​of macroblocks at the same position will change over time, so the heat value can be used as a dynamic indicator to reflect the importance of each macroblock in real time, thereby making the adjustment of the quantization parameters more flexible and efficient.

[0096] For non-target macroblocks, since they do not contain any image content of particular interest, the image content in the macroblock can be considered a static background area. Therefore, the quantization parameter value of the macroblock at the same position as the macroblock in the previous video frame can be used as the reference quantization parameter value for the macroblock to calculate the quantization parameter value for the macroblock. It can be understood that, for macroblocks that do not contain important image content, the embodiment of the present invention calculates the quantization parameter value for the macroblock in the currently processed video frame by referencing the quantization parameter value of the macroblock at the same position in the previous video frame, thereby ensuring the consistency and stability of the encoding of non-interest areas in the video.

[0097] It can be understood that the embodiment of the present invention adopts different coding strategies for macroblocks of different categories, thereby achieving differentiated coding of different areas in the video frame, improving the reasonable allocation of coding resources, and enhancing the overall performance of video coding.

[0098] Optionally, for the process of determining the quantization parameter value of the macroblock based on the heat value of the macroblock in step S208-1, an embodiment of the present invention provides a possible implementation method, namely, obtaining a baseline adjustment value for the macroblock based on the type of the target object in the target area intersecting the macroblock. The quantization parameter value of the macroblock is determined based on the heat value of the macroblock and the baseline adjustment value.

[0099] It will be appreciated that the embodiments of the present invention classify different types of target objects into different importance levels and set corresponding benchmark adjustment values ​​for each type, so as to determine the quantization parameter value of the macroblock based on the type of target object in the macroblock. Furthermore, the higher the importance level of the target object, the higher the benchmark adjustment value.

[0100] For example, in the above-mentioned monitoring scenario of the substation, the target objects can be three types: instruments, switch devices, and personnel. Moreover, since the accuracy of the data in the instrument is more important, the importance level of the instrument type is high, and the benchmark adjustment value corresponding to the instrument can be set to 6; the switch device can reflect the operating status of the equipment, so the importance level of the switch device type is medium, and the benchmark adjustment value corresponding to the switch device can be set to 4; the frequency of personnel appearing in this scenario is low, so the importance level of the personnel type is low, and the benchmark adjustment value corresponding to the personnel can be set to 3. It can be understood that the type of target object and its corresponding benchmark adjustment value can be set according to actual conditions, and the embodiments of the present invention do not limit this.

[0101] In this embodiment, for a macroblock with a target, since the macroblock intersects the target region, the benchmark adjustment value corresponding to the type of target object in the target region intersecting the macroblock is obtained to obtain the benchmark adjustment value for the macroblock. The quantization parameter value for the macroblock is then calculated based on the heat value and the benchmark adjustment value for the macroblock.

[0102] It can be understood that the embodiments of the present invention use heat values ​​to reflect the varying importance of image content, and use baseline adjustment values ​​to reflect the varying importance levels of target objects. In other words, the importance reflected by these two factors is used to jointly determine the quantization parameter value for the macroblock. This improves the accuracy of quantization parameter adjustment.

[0103] Optionally, for the above-mentioned process of determining the quantization parameter value of the macroblock based on the heat value and benchmark adjustment value of the macroblock, an embodiment of the present invention adopts a possible implementation method, namely: according to a preset first adjustment formula, based on a preset basic quantization parameter value, a preset heat influence coefficient, the heat value of the macroblock and the benchmark adjustment value, the quantization parameter value of the macroblock is calculated.

[0104] And, the first adjustment formula is:

[0105] ;

[0106] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at row i and column j in the video frame to be processed, represents the thermal influence coefficient, Indicates the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0107] In this embodiment, for a target macroblock, the first adjustment formula can be used to calculate the quantization parameter value of the macroblock. First, the heat value of the macroblock is Thermal influence coefficient Multiplying together yields the third product and is expressed as , and adding one to the third product yields a first sum value and is expressed as ; Then, the first sum is added to the baseline adjustment value Multiplying together, we get the fourth product and express it as ;Finally, the basic quantization parameter value Subtracting the fourth product, the quantization parameter value of the macroblock is Among them, the heat influence coefficient The value range is (0,1), such as the heat impact coefficient It can be set to 0.5, or can be set according to actual conditions, and the embodiment of the present invention is not limited to this.

[0108] It can be understood that the embodiment of the present invention determines the quantization parameter value of the macroblock by the heat value and the reference adjustment value of the macroblock. On the one hand, the higher the heat value of the macroblock, the more important the image content corresponding to the macroblock is. Based on the first adjustment formula, it can be concluded that The larger the result of The larger the calculated result of , the smaller the quantization parameter value of the macroblock will be, that is, the higher the quantization accuracy will be, and the lower the compression level will be. On the other hand, the higher the benchmark adjustment value of the macroblock, the higher the importance level of the target object in the macroblock, and the higher the benchmark adjustment value The larger the value, based on the principle similar to the above, the smaller the quantization parameter value of the macroblock will be, that is, the higher the quantization accuracy will be, and the lower the compression level will be. This allows the targeted macroblock to retain more details during encoding, thus ensuring the image quality of the key areas.

[0109] Optionally, for the process of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock in step S208-3, an embodiment of the present invention provides a possible implementation method, namely: according to a preset second adjustment formula, based on a preset basic quantization parameter value, a preset attenuation rate coefficient, and the reference quantization parameter value of the macroblock, the quantization parameter value of the macroblock is calculated.

[0110] And, the second adjustment formula is:

[0111] ;

[0112] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the decay rate coefficient; The reference quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed.

[0113] In this embodiment, for macroblocks without a target, the second adjustment formula can be used to calculate the quantization parameter value of the macroblock. First, the basic quantization parameter value, i.e. The reference quantization parameter value of the macroblock Subtract, get the second difference and express it as ; Then, the decay rate coefficient Multiplying by the second difference, the fifth product is obtained and expressed as ; Finally, the reference quantization parameter value of the macroblock Added to the fifth multiplication, the quantization parameter value of the macroblock is .

[0114] It can be understood that after the target object disappears from the video screen, the attenuation rate coefficient can be used , to gradually restore the quantization parameter value of the macroblock to the basic quantization parameter value, so as to avoid sudden changes in video quality and visual jumps. In addition, the attenuation rate coefficient The value range is (0,1), which is used to adjust the speed of change of the quantization parameter value; the attenuation rate coefficient The larger the value, the faster the quantization parameter value changes; the decay rate coefficient The smaller it is, the slower the change of the quantization parameter value will be.

[0115] For ease of understanding, the present invention provides an example. For example, assuming that the decay rate coefficient is 0.2, the basic quantization parameter value is 32. For a macroblock at a certain position, in the nth frame, there is a target object at this macroblock, and its quantization parameter value is 24; in the n+1th frame, the target object disappears from the video screen, that is, there is no target object at this macroblock, and the quantization parameter value of the macroblock at this position is calculated using the second adjustment formula to be 25.6; in the n+2th frame, there is no target object at this macroblock, and the quantization parameter value of the macroblock at this position is calculated using the second adjustment formula to be 26.88. Calculating in a similar manner, it can be concluded that, starting from the nth frame, after 7 video frames, the quantization parameter value of the macroblock at this position is restored to 30.4; after 30 videos, the quantization parameter value of the macroblock at this position is approximately 32.

[0116] Moreover, the decay rate coefficient The specific value of can also be set according to the type of target object. For example, the higher the importance level, the lower the decay rate coefficient. The smaller the value, the slower the change in video quality will be after the target object disappears, thus improving the visual experience. For example, in the above substation monitoring scenario, the target objects can be three types: instruments, switch devices, and personnel. Since the importance level of the instrument type is high, the attenuation rate coefficient can be set to Set to 0.3; Since the importance level of this type of switch device is medium, the decay rate coefficient can be set to is 0.2; since the importance level of personnel is low, the attenuation rate coefficient can be set to It should be understood that the decay rate coefficient The specific value of can also be set according to actual conditions, and the embodiment of the present invention is not limited to this.

[0117] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of a video encoding device is given below. Figure 4 , is a functional block diagram of a video encoding device provided by an embodiment of the present invention. It should be noted that the basic principles and technical effects of the video encoding device 300 provided by this embodiment are the same as those of the above-mentioned embodiments. For the sake of simplicity, any details not mentioned in this embodiment may be referred to the corresponding contents of the above-mentioned embodiments. The video encoding device 300 includes:

[0118] The determination module 310 is configured to sequentially use each video frame in the video as a to-be-processed video frame, determine a target region where a target object is located in the to-be-processed video frame, and determine a positional relationship between all macroblocks in the to-be-processed video frame and the target region.

[0119] The processing module 330 is used to obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame contained in the three-dimensional heat information, and obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance of the image content corresponding to all macroblocks in the corresponding video frame; based on the position relationship and the reference two-dimensional heat information, the two-dimensional heat information of the video frame to be processed is obtained, and the two-dimensional heat information of the video frame to be processed is updated to the three-dimensional heat information.

[0120] The encoding module 350 is used to determine the quantization parameter value of each macroblock in the video frame to be processed based on the position relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0121] Optionally, the processing module 330 is also used to: determine the target indication value of each macroblock in the video frame to be processed based on the position relationship; determine the reference heat value of each macroblock in the video frame to be processed based on the reference two-dimensional heat information; wherein the reference heat value of the macroblock is the heat value of the macroblock with the same position as the macroblock in the reference two-dimensional heat information; calculate the heat value of each macroblock in the video frame to be processed based on the target indication value and reference heat value of each macroblock in the video frame to be processed, and obtain the two-dimensional heat information of the video frame to be processed.

[0122] Optionally, the processing module 330 is further configured to: for each macroblock in the video frame to be processed, calculate the heat value of the macroblock according to a preset heat formula based on a preset time decay factor, a target indicator value of the macroblock, and a reference heat value, to obtain the heat value of each macroblock in the video frame to be processed; the heat formula is:

[0123] ;

[0124] in, Indicates the heat value of the macroblock in the i-th row and j-th column of the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at row i and column j in the video frame to be processed, Represents the target indicator value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0125] Optionally, the encoding module 350 is also used to: for each macroblock in the video frame to be processed, if the macroblock intersects with the target area, determine the quantization parameter value of the macroblock based on the heat value of the macroblock; if the macroblock does not intersect with the target area, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock in the previous video frame of the video frame to be processed with the same position as the macroblock.

[0126] Optionally, the encoding module 350 is further configured to: obtain a benchmark adjustment value of the macroblock according to the type of the target object in the target area intersecting the macroblock; and determine a quantization parameter value of the macroblock according to the heat value and the benchmark adjustment value of the macroblock.

[0127] Optionally, the encoding module 350 is further configured to calculate the quantization parameter value of the macroblock according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset heat impact coefficient, the heat value of the macroblock, and the reference adjustment value; the first adjustment formula is:

[0128] ;

[0129] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at row i and column j in the video frame to be processed, Indicates the thermal influence coefficient, Indicates the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0130] Optionally, the encoding module 350 is further configured to calculate the quantization parameter value of the macroblock according to a preset second adjustment formula based on a preset basic quantization parameter value, a preset decay rate coefficient, and a reference quantization parameter value of the macroblock; the second adjustment formula is:

[0131] ;

[0132] in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the decay rate coefficient; The reference quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed.

[0133] An embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the video encoding method disclosed in the embodiment of the present invention is implemented.

[0134] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the video encoding method disclosed in the embodiment of the present invention is implemented.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0137] If the functions are implemented as software modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0138] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A video encoding method, characterized in that: The method comprises: Taking each video frame in the video as a to-be-processed video frame in turn, determining a target region where a target object is located in the to-be-processed video frame, and determining a positional relationship between all macroblocks in the to-be-processed video frame and the target region; Obtaining the 2D heat information of the video frame preceding the video frame to be processed from the 2D heat information of each historical video frame included in the 3D heat information to obtain reference 2D heat information; the 2D heat information indicates the importance of the image content corresponding to all macroblocks in the corresponding video frame; Obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information; According to the positional relationship and the two-dimensional heat information of the video frame to be processed, the quantization parameter value of each macroblock in the video frame to be processed is determined, and the video frame to be processed is encoded according to the quantization parameter value of each macroblock in the video frame to be processed.

2. The video encoding method according to claim 1, wherein: The two-dimensional heat information includes a heat value of each macroblock in the corresponding video frame, where the heat value of the macroblock represents the importance of the image content corresponding to the macroblock; The step of obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information includes: Determining a target indicator value for each macroblock in the to-be-processed video frame according to the positional relationship; Determining a reference heat value of each macroblock in the to-be-processed video frame according to the reference two-dimensional heat information; wherein the reference heat value of the macroblock is the heat value of the macroblock at the same position as the macroblock in the reference two-dimensional heat information; According to the target indication value and reference heat value of each macroblock in the video frame to be processed, the heat value of each macroblock in the video frame to be processed is calculated to obtain two-dimensional heat information of the video frame to be processed.

3. The video encoding method according to claim 2, wherein: The step of calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed comprises: For each macroblock in the video frame to be processed, a heat value of the macroblock is calculated according to a preset heat formula based on a preset time decay factor, a target indicator value of the macroblock, and a reference heat value, to obtain a heat value of each macroblock in the video frame to be processed; the heat formula is: ； in, Indicates the heat value of the macroblock in the i-th row and j-th column of the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at row i and column j in the video frame to be processed, Represents the target indicator value of the macroblock in the i-th row and j-th column in the video frame to be processed.

4. The video encoding method according to claim 1, wherein: The two-dimensional heat information of the video frame to be processed includes the heat value of each macroblock in the video frame to be processed; the position relationship includes whether each macroblock in the video frame to be processed intersects with the target area; The step of determining the quantization parameter value of each macroblock in the video frame to be processed based on the positional relationship and the two-dimensional heat information of the video frame to be processed includes: For each macroblock in the to-be-processed video frame, if the macroblock intersects with the target area, determining a quantization parameter value of the macroblock based on a heat value of the macroblock; If the macroblock does not intersect with the target area, a reference quantization parameter value of the macroblock is obtained, and the quantization parameter value of the macroblock is determined based on the reference quantization parameter value of the macroblock; wherein the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock in the previous video frame of the video frame to be processed, which has the same position as the macroblock.

5. The video encoding method according to claim 4, wherein: The step of determining the quantization parameter value of the macroblock based on the heat value of the macroblock includes: obtaining a reference adjustment value of the macroblock according to a type of a target object in a target area intersecting the macroblock; A quantization parameter value of the macroblock is determined according to the heat value and the reference adjustment value of the macroblock.

6. The video encoding method according to claim 5, wherein: The step of determining the quantization parameter value of the macroblock according to the heat value and the reference adjustment value of the macroblock includes: The quantization parameter value of the macroblock is calculated according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset thermal impact coefficient, the thermal value of the macroblock, and a reference adjustment value; the first adjustment formula is: ； in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at row i and column j in the video frame to be processed, Indicates the thermal influence coefficient, Indicates the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed.

7. The video encoding method according to claim 4, wherein: The step of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock comprises: According to a preset second adjustment formula, the quantization parameter value of the macroblock is calculated based on a preset basic quantization parameter value, a preset decay rate coefficient, and a reference quantization parameter value of the macroblock; the second adjustment formula is: ； in, Indicates the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, Represents the basic quantization parameter value, represents the decay rate coefficient; Indicates the reference quantization parameter value of the macroblock in the i-th row and j-th column in the video frame to be processed.

8. A video encoding device, characterized in that: The device comprises: a determination module, configured to sequentially use each video frame in the video as a to-be-processed video frame, determine a target region where a target object is located in the to-be-processed video frame, and determine a positional relationship between all macroblocks in the to-be-processed video frame and the target region; a processing module configured to obtain, from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, the two-dimensional heat information of the video frame preceding the video frame to be processed, to obtain reference two-dimensional heat information; the two-dimensional heat information indicating the importance of image content corresponding to all macroblocks in the corresponding video frame; Obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information; An encoding module is used to determine the quantization parameter value of each macroblock in the video frame to be processed based on the position relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the video encoding method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the video encoding method according to any one of claims 1 to 7.

Images