Video coding method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of video processing, and provides a video coding method and device, electronic equipment and a storage medium. The method comprises the following steps: firstly, determining a target area where a target object is located in a video frame to be processed, and determining a position relationship between all macro blocks and the target area; obtaining two-dimensional heat information of a previous video frame of the to-be-processed video frame from the three-dimensional heat information to obtain reference two-dimensional heat information; obtaining two-dimensional heat information of the video frame to be processed according to the position relation and the reference two-dimensional heat information, and updating the two-dimensional heat information to three-dimensional heat information; and finally, according to the position relationship and the two-dimensional heat information of the video frame to be processed, determining a quantization parameter value of each macro block in the video frame to be processed, and according to the quantization parameter value of each macro block in the video frame to be processed, coding the video frame to be processed. Therefore, reasonable allocation of coding resources is realized, and the adaptation degree of video coding and a dynamic change scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular, to a video encoding method, apparatus, electronic device, and storage medium. Background Art

[0002] The QP value (Quantization Parameter) is an important parameter in video encoding and image processing, which is used to control the accuracy of the quantization process. Quantization is the process of converting a continuous or high-precision signal into a discrete or low-precision signal. In video encoding, the QP value determines the compression degree and quality loss of video data. Currently, a fixed QP value is usually used to encode videos, but this method is not applicable to dynamically changing scenarios, and there are problems such as poor video encoding effect and resource waste. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a video encoding method, apparatus, electronic device, and storage medium.

[0004] To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a video encoding method, and the method includes: Taking each video frame in the video as a to-be-processed video frame in sequence, determining a target area where a target object is located in the to-be-processed video frame, and determining the positional relationship between all macroblocks in the to-be-processed video frame and the target area; Obtaining the two-dimensional heat information of the video frame immediately preceding the to-be-processed video frame from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, to obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame; Obtaining the two-dimensional heat information of the to-be-processed video frame according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the to-be-processed video frame to the three-dimensional heat information; Determining the quantization parameter value of each macroblock in the to-be-processed video frame according to the positional relationship and the two-dimensional heat information of the to-be-processed video frame, and encoding the to-be-processed video frame according to the quantization parameter value of each macroblock in the to-be-processed video frame.

[0005] In an optional implementation manner, the two-dimensional heat information includes the heat value of each macroblock in the corresponding video frame, and the heat value of the macroblock represents the importance degree of the image content corresponding to the macroblock; The step of obtaining the two-dimensional heat information of the to-be-processed video frame according to the positional relationship and the reference two-dimensional heat information includes: Determine the target indication value of each macroblock in the video frame to be processed according to the position relationship; Determine the reference heat value of each macroblock in the video frame to be processed according to the reference two-dimensional heat information; wherein, the reference heat value of the macroblock is the heat value of the macroblock with the same position as this macroblock in the reference two-dimensional heat information; Calculate the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed, and obtain the two-dimensional heat information of the video frame to be processed.

[0006] In an alternative embodiment, the step of calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed includes: For each macroblock in the video frame to be processed, calculate the heat value of the macroblock according to a preset heat formula based on a preset time decay factor, the target indication value and the reference heat value of the macroblock, so as to obtain the heat value of each macroblock in the video frame to be processed; the heat formula is: ; wherein, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the target indication value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0007] In an alternative embodiment, the two-dimensional heat information of the video frame to be processed includes the heat value of each macroblock in the video frame to be processed; the position relationship includes whether each macroblock in the video frame to be processed intersects with the target area; The step of determining the quantization parameter value of each macroblock in the video frame to be processed according to the position relationship and the two-dimensional heat information of the video frame to be processed includes: For each macroblock in the video frame to be processed, if the macroblock intersects with the target area, determine the quantization parameter value of the macroblock based on the heat value of the macroblock; If the macroblock does not intersect with the target area, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein, the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock with the same position as this macroblock in the previous video frame of the video frame to be processed.

[0008] In an alternative embodiment, the step of determining the quantization parameter value of the macroblock based on the heat value of the macroblock includes: Obtaining a reference adjustment value of the macroblock according to the type of the target object in the target area intersecting with the macroblock; Determining the quantization parameter value of the macroblock according to the heat value and the reference adjustment value of the macroblock.

[0009] In an alternative embodiment, the step of determining the quantization parameter value of the macroblock according to the heat value and the reference adjustment value of the macroblock includes: Calculating the quantization parameter value of the macroblock according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset heat influence coefficient, the heat value and the reference adjustment value of the macroblock; the first adjustment formula is: ; Wherein, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the heat influence coefficient, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0010] In an alternative embodiment, the step of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock includes: Calculating the quantization parameter value of the macroblock according to a preset second adjustment formula based on a preset basic quantization parameter value, a preset attenuation rate coefficient, and the reference quantization parameter value of the macroblock; the second adjustment formula is: ; Wherein, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the attenuation rate coefficient; the reference quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0011] In a second aspect, the present invention provides a video encoding device, the device includes: A determination module, configured to sequentially use each video frame in the video as a video frame to be processed, determine the target area where the target object is located in the video frame to be processed, and determine the positional relationship between all the macroblocks in the video frame to be processed and the target area; A processing module is configured to obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, so as to obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame. Obtain the two-dimensional heat information of the video frame to be processed according to the position relationship and the reference two-dimensional heat information, and update the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information. An encoding module is configured to determine the quantization parameter value of each macroblock in the video frame to be processed according to the position relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0012] In a third aspect, the present invention provides an electronic device, including a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the video encoding method according to any one of the foregoing embodiments is implemented.

[0013] In a fourth aspect, the present invention provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the video encoding method according to any one of the foregoing embodiments is implemented.

[0014] The video encoding method, device, electronic device and storage medium provided by the embodiments of the present invention, the method includes: first, sequentially taking each video frame in the video as the video frame to be processed, determining the target area where the target object is located in the video frame to be processed, and determining the position relationship between all macroblocks in the video frame to be processed and the target area; then, obtaining the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, so as to obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame; then, obtaining the two-dimensional heat information of the video frame to be processed according to the position relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information; finally, determining the quantization parameter value of each macroblock in the video frame to be processed according to the position relationship and the two-dimensional heat information of the video frame to be processed, and encoding the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed. The embodiments of the present invention determine the position relationship between the macroblock and the target area, so as to reasonably allocate encoding resources according to the correlation between the two. And the three-dimensional heat information is used to reflect the change of the target object in the spatio-temporal dimension, and at the same time, the two-dimensional heat information of adjacent video frames is used to determine the quantization parameter value of each macroblock, thereby improving the adaptability of video encoding to dynamic change scenarios and realizing the adaptive regulation of video encoding quality.

[0015] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides preferred embodiments in conjunction with the accompanying drawings and describes them in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0017] Figure 1 Shows a block diagram of an electronic device provided by an embodiment of the present invention; Figure 2 Shows one of the flow diagrams of a video encoding method provided by an embodiment of the present invention; Figure 3 Shows another flow diagram of a video encoding method provided by an embodiment of the present invention; Figure 4 Shows a functional module diagram of a video encoding device provided by an embodiment of the present invention.

[0018] Reference numerals: 100 - electronic device; 110 - processor; 120 - memory; 130 - communication module; 300 - video encoding device; 310 - determination module; 330 - processing module; 350 - encoding module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0021] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0022] The QP value (Quantization Parameter) is an important parameter in video coding and image processing, and is used to control the precision of the quantization process. Quantization is the process of converting a continuous or high-precision signal into a discrete or low-precision signal. In video coding, the QP value determines the compression degree and quality loss of video data. Currently, a fixed QP value is usually used to encode videos, but this method is not applicable to dynamically changing scenarios. Although there is also a method of video coding based on ROI (Region of Interest), it has the following problems: (1) dividing the video into fixed regions for processing results in poor adaptability to dynamically changing scenarios; (2) before and after the target appears, there are large differences in video quality and jitter, etc., affecting the visual experience. (3) The allocation of coding resources is not intelligent enough, resulting in resource waste. Therefore, the embodiments of the present invention provide a video coding method to solve the above problems.

[0023] Please refer to Figure 1 , which is a block diagram of the electronic device provided by the embodiments of the present invention. The electronic device 100 includes a processor 110, a memory 120 and a communication module 130. Each element is electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0024] The processor 110 is used to read / write data or programs stored in the memory 120 and perform corresponding functions. It can be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0025] The memory 120 is used to store programs or data. It can be a RAM (Random Access Memory), a ROM (Read Only Memory), a PROM (Programmable Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0026] The communication module 130 is used to communicate signaling or data with other devices.

[0027] It should be noted that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include more or fewer components than Figure 1 those shown, or have a configuration different from Figure 1 that shown. Figure 1 Each component shown can be implemented by hardware, software, or a combination thereof.

[0028] It can be understood that, in one implementation, the electronic device according to the embodiments of the present invention can receive the video collected by the shooting device and encode it. In another implementation, the electronic device according to the embodiments of the present invention can also integrate a shooting module and be capable of collecting video in real time and encoding it. Next, taking the above-mentioned electronic device 100 as the execution subject, each step in each method provided by the embodiments of the present invention will be introduced, as well as the corresponding technical effects.

[0029] Please refer to Figure 2 , which is a schematic flowchart of a video encoding method provided by an embodiment of the present invention.

[0030] Step S202: Take each video frame in the video as a video frame to be processed in turn, determine the target area where the target object is located in the video frame to be processed, and determine the positional relationship between all macroblocks in the video frame to be processed and the target area.

[0031] Step S204: Obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, so as to obtain the reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame.

[0032] Step S206: Obtain the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and update the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information.

[0033] Step S208: Determine the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0034] In this embodiment, for each video frame in the collected video, each video frame can be taken as a video frame to be processed in turn and encoded to implement video encoding. That is, the processing manner of each video frame in the embodiments of the present invention is similar. For the sake of brief description, the following takes the video frame to be processed as an example for illustration.

[0035] First, a preset detection algorithm or detection module can be used to perform target detection on the video frame to be processed. For example, a preset detection model such as the YOLOv7 lightweight detection model can be used to perform target detection on the video frame to be processed to determine the area where the target object is located in the video frame to be processed, that is, obtain the target area.

[0036] It can be understood that the embodiments of the present invention can be used not only for single-type target detection but also for multi-type target detection. For example, the preset model can also output the confidence levels of the target object in the video frame to be processed belonging to various preset types, and take the type with the highest confidence level as the type of the target object. Among them, the target object can be understood as the image content that is the key focus in the video picture. For example, in the monitoring scenario of a substation, the collected video is the monitoring video of the substation, then the target object can be meters in digital or pointer form, switch devices of equipment, and personnel, etc.

[0037] Moreover, according to the adopted video coding standard, the size and quantity of macroblocks in a video frame can be determined. Among them, a macroblock refers to the basic unit of video coding. Dividing a video frame into multiple macroblocks means dividing a video frame into multiple independently codable regions. For example, assuming the adopted video coding standard is the H.264 coding standard, then the size of a macroblock is 16×16, that is, 256 pixel points, and if the width of the resolution of the video frame is W and the height is H, then the number of macroblocks in a video frame is pieces. Moreover, since the size and quantity of macroblocks in each video frame are the same, the positional relationship between all the macroblocks in the video frame to be processed and the target region can be determined according to the target region where the target object is located in the video frame to be processed.

[0038] Then, from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, the two-dimensional heat information of the video frame before the video frame to be processed is obtained, that is, the reference two-dimensional heat information is obtained. Among them, a historical video frame refers to a video frame before the video frame to be processed. The two-dimensional heat information can be understood as a data structure constructed for a specific video frame, which represents the importance degree of the image content corresponding to each macroblock in this video frame. The three-dimensional heat information is a data structure constructed by performing spatio-temporal joint on the two-dimensional heat information of multiple consecutive video frames, which includes the distribution of the target object in the time dimension and the space dimension. And a three-dimensional heat map can be used to realize the data visualization of the three-dimensional heat information.

[0039] Next, according to the positional relationship between all the macroblocks in the video frame to be processed and the target region, and the importance degree of the image content corresponding to each macroblock in the previous video frame represented by the reference two-dimensional heat information, the importance degree of the image content corresponding to each macroblock in the video frame to be processed can be determined, and then the two-dimensional heat information of the video frame to be processed is obtained. And the two-dimensional heat information of the video frame to be processed is added to the three-dimensional heat information to facilitate obtaining the two-dimensional heat information of the next video frame.

[0040] Finally, according to the positional relationship and the two-dimensional heat information of the video frame to be processed, the quantization parameter value of each macroblock in the video frame to be processed is determined, and the video frame to be processed is encoded according to the quantization parameter value of each macroblock in the video frame to be processed, that is, the encoded video frame is obtained. In a similar manner, each video frame in the video is processed to obtain each encoded video frame, that is, the encoded video is obtained. It can be understood that in the embodiments of the present invention, different quantization parameter values are used for encoding different regions in a video frame to perform different degrees of compression on the video picture, so that the quality of each region is different.

[0041] It can be understood that in the embodiments of the present invention, by determining the positional relationship between the macroblock and the target area, it is convenient to reasonably allocate coding resources according to the association relationship between the two. And the three-dimensional heat information is used to reflect the changes of the target object in the spatio-temporal dimension. At the same time, the two-dimensional heat information of adjacent video frames is used to determine the quantization parameter values of each macroblock, thereby improving the adaptability of video coding to dynamically changing scenes and realizing the adaptive regulation of video coding quality.

[0042] Optionally, for the process of obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information in step S206, the embodiments of the present invention provide a possible implementation manner. Please refer to Figure 3 .

[0043] Step S206-1: Determine the target indication value of each macroblock in the video frame to be processed according to the positional relationship.

[0044] Step S206-2: Determine the reference heat value of each macroblock in the video frame to be processed according to the reference two-dimensional heat information; wherein, the reference heat value of a macroblock is the heat value of the macroblock with the same position as this macroblock in the reference two-dimensional heat information.

[0045] Step S206-3: Calculate the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed, and obtain the two-dimensional heat information of the video frame to be processed.

[0046] In this embodiment, the positional relationship includes whether each macroblock in the video frame to be processed intersects with the target area. Then, it is possible to determine the target indication value of each macroblock according to whether each macroblock in the video frame to be processed intersects with the target area.

[0047] For example, for each macroblock in the video frame to be processed, if this macroblock intersects with the target area, it means that there are pixel points representing the target object in this macroblock, then it is considered that there is a target object in this macroblock, and the target indication value of this macroblock is set to a preset first value such as 1. If this macroblock does not intersect with the target area, it means that there are no pixel points representing the target object in this macroblock, then it is considered that there is no target object in this macroblock, and the target indication value of this macroblock is set to a preset second value such as 0. That is, the target indication value of a macroblock is used to indicate whether there is a target object in this macroblock.

[0048] Then, the two-dimensional heat information includes the heat value of each macroblock in the corresponding video frame, and the reference two-dimensional heat information includes the heat value of each macroblock in the previous video frame of the video frame to be processed. Then, based on the reference two-dimensional heat information, the reference heat value of each macroblock in the video frame to be processed can be determined. For example, since the number of macroblocks in each video frame is the same, for each macroblock in the video frame to be processed, the heat value of the macroblock with the same position as this macroblock in the reference two-dimensional heat information can be used as the reference heat value of this macroblock, so as to obtain the reference heat value of each macroblock in the video frame to be processed.

[0049] Finally, according to the target indication value and the reference heat value of each macroblock in the video frame to be processed, calculate the heat value of each macroblock in the video frame to be processed, so as to obtain the importance degree of the image content corresponding to each macroblock in the video frame to be processed, that is, obtain the two-dimensional heat information of the video frame to be processed.

[0050] Optionally, for the process of calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed in step S206-5, the embodiment of the present invention provides a possible implementation manner, that is: for each macroblock in the video frame to be processed, according to a preset heat formula, based on a preset time decay factor, the target indication value and the reference heat value of the macroblock, calculate the heat value of the macroblock, so as to obtain the heat value of each macroblock in the video frame to be processed.

[0051] And, the heat formula is: ; Wherein, represents the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock in the i-th row and j-th column in the video frame to be processed, represents the target indication value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0052] It can be understood that the embodiment of the present invention is similar to the method for calculating the heat value of each macroblock in the video frame to be processed. For the sake of brief description, the following takes a macroblock as an example for illustration.

[0053] For example, a weighted fusion method can be adopted to calculate its heat value based on the target indication value and the reference heat value of the macroblock. First, multiply the preset time decay factor by the reference heat value of this macroblock to obtain a first product and denote it as ; then, multiply one by the preset time decay factor Subtract to obtain a first difference, and multiply the first difference by the target indication value of the macroblock to obtain a second product, which is expressed as ; finally, add the first product and the second product to obtain the heat value of the macroblock, which is .

[0054] It should be understood that since there is no previous video frame for the first video frame of the video, the heat formula for calculating each macroblock in the first video frame can be expressed as: ; where represents the heat value of the macroblock at the i-th row and j-th column in the first video frame; represents the target indication value of the macroblock at the i-th row and j-th column in the first video frame; represents the time decay factor.

[0055] It should be noted that the time decay factor has a value range of (0, 1). And the time decay factor is used to adjust the weight relationship between historical data and current data; when the time decay factor is close to 0, it indicates that the target indication value of the macroblock in the currently processed video frame has a great influence on the heat value of the macroblock. Then this method is suitable for scenarios that require quick response to sudden targets. When the time decay factor is close to 1, it indicates that the reference heat value obtained based on historical video frames has a great influence on the heat value of the macroblock. Then this method is suitable for scenarios with high stability requirements.

[0056] For example, for the scenario of substation monitoring with relatively high stability requirements, the time decay factor can be set to 0.95. Then based on the above heat formula, it can be obtained that when a target object continuously exists in a macroblock at a certain position, as time goes by, the heat value of the macroblock at that position will gradually increase; if there is no target object in the macroblock at that position, as time goes by, the heat value of the macroblock at that position will gradually decrease. This can make the change of the heat value of the macroblock relatively smooth and avoid instantaneous fluctuations. It should be understood that the specific value of the time decay factor can be set according to the actual application scenario, and the embodiments of the present invention are not limited thereto.

[0057] It can be understood that the embodiments of the present invention combine historical heat information with the existence state of the current target object through the time decay factor, so as to determine the importance degree of each region in the currently processed video frame in a spatio-temporal joint manner, thereby providing an important basis for the reasonable allocation of subsequent coding resources, and enabling the coding resources to be more efficiently concentrated on the concerned image content.

[0058] Optionally, for the process of determining the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed in step S208, an embodiment of the present invention provides a possible implementation manner.

[0059] Step S208-1: For each macroblock in the video frame to be processed, if the macroblock intersects with the target area, determine the quantization parameter value of the macroblock based on the heat value of the macroblock.

[0060] Step S208-2: If the macroblock does not intersect with the target area, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein, the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock at the same position as the macroblock in the previous video frame of the video frame to be processed.

[0061] It can be understood that the embodiments of the present invention are similar in the way of calculating the quantization parameter value of each macroblock in the video frame to be processed. For the sake of brief description, the following takes a macroblock as an example for illustration. As introduced above, if a macroblock intersects with the target area, it is considered that there is a target object in the macroblock, then the macroblock can be classified as a macroblock with a target. If the macroblock does not intersect with the target area, it is considered that there is no target object in the macroblock, then the macroblock can be classified as a macroblock without a target.

[0062] For a macroblock with a target, since there is key image content of concern in it, the quantization parameter value of the macroblock can be calculated through the heat value of the macroblock. It can be understood that for a macroblock containing important image content, the embodiments of the present invention further subdivide the importance of the image content according to the heat value of the macroblock, so as to ensure that when encoding a macroblock with a higher heat value, that is, a higher importance, the quality of the image can be guaranteed, and when encoding a macroblock with a lower heat value, that is, a lower importance, the encoding accuracy is appropriately reduced, so as to realize the dynamic adjustment of the quantization parameter and ensure the reasonable allocation of encoding resources. And, the heat value of the macroblock at the same position changes with time, so the heat value can be used as a dynamic index to reflect the importance of each macroblock in real time, so that the adjustment of the quantization parameter is more flexible and efficient.

[0063] For a macroblock without a target, since there is no key image content of concern in it, it can be considered that the image content in the macroblock is a static background area, then the quantization parameter value of the macroblock at the same position as the macroblock in the previous video frame can be used as the reference quantization parameter value of the macroblock to calculate the quantization parameter value of the macroblock. It can be understood that for a macroblock that does not contain important image content, the embodiments of the present invention calculate the quantization parameter value of the macroblock in the currently processed video frame by referring to the quantization parameter value of the macroblock at the same position in the previous video frame, so as to ensure the consistency and stability of the encoding of the non-concerned area in the video.

[0064] It can be understood that in the embodiments of the present invention, different coding strategies are adopted for different categories of macroblocks, thereby realizing differential coding for different regions in a video frame, improving the reasonable allocation of coding resources, and enhancing the overall performance of video coding.

[0065] Optionally, for the process of determining the quantization parameter value of a macroblock based on the popularity value of the macroblock in step S208-1, the embodiments of the present invention provide a possible implementation manner, that is: obtain the reference adjustment value of the macroblock according to the type of the target object in the target region intersecting the macroblock. Determine the quantization parameter value of the macroblock according to the popularity value and the reference adjustment value of the macroblock.

[0066] It can be understood that in the embodiments of the present invention, different importance levels are divided for different types of target objects, and corresponding reference adjustment values are respectively set, so as to determine the quantization parameter value of the macroblock according to the type of the target object in the macroblock. Moreover, the higher the importance level of a type, the higher its reference adjustment value.

[0067] For example, in the above-mentioned monitoring scenario of a substation, the target objects can be three types: instruments, switching devices, and personnel. Moreover, since the accuracy of the data in the instruments is relatively important, the importance level of the type of instrument is high, and the reference adjustment value corresponding to the instrument can be set to 6; the switching devices can reflect the operating state of the equipment, so the importance level of the type of switching device is medium, and the reference adjustment value corresponding to the switching device can be set to 4; the frequency of personnel appearing in this scenario is low, so the importance level of the type of personnel is low, and the reference adjustment value corresponding to the personnel can be set to 3. It can be understood that the type of the target object and its corresponding reference adjustment value can be set according to the actual situation, and the embodiments of the present invention do not limit this.

[0068] In this embodiment, for a macroblock with a target, since this macroblock intersects the target region, the reference adjustment value corresponding to the type of the target object in the target region intersecting this macroblock can be obtained to obtain the reference adjustment value of this macroblock. And based on the popularity value and the reference adjustment value of this macroblock, calculate the quantization parameter value of this macroblock.

[0069] It can be understood that in the embodiments of the present invention, the different importance of the image content is reflected by the popularity value, and the different importance levels of the target objects are reflected by the reference adjustment value. That is, the quantization parameter value of the macroblock is jointly determined by the importance degrees reflected by these two aspects. Thereby improving the adjustment accuracy of the quantization parameter.

[0070] Optionally, for the above-mentioned process of determining the quantization parameter value of the macroblock based on the heat value and the benchmark adjustment value of the macroblock, an embodiment of the present invention adopts a possible implementation method, namely: according to a preset first adjustment formula, based on a preset basic quantization parameter value, a preset heat influence coefficient, the heat value of the macroblock and the benchmark adjustment value, the quantization parameter value of the macroblock is calculated.

[0071] And, the first adjustment formula is: ; in, represents the quantization parameter value of the macroblock in the i-th row and j-th column of the video frame to be processed, represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the thermal influence coefficient, Indicates the heat value of the macroblock in the i-th row and j-th column in the video frame to be processed.

[0072] In this embodiment, for a target macroblock, the first adjustment formula can be used to calculate the quantization parameter value of the macroblock. First, the heat value of the macroblock Thermal influence coefficient The third product is multiplied and expressed as , and adding one to the third product yields the first sum and is expressed as ; Then, the first sum is added to the reference adjustment value Multiplying together, we get the fourth product and express it as ;Finally, the basic quantization parameter value Subtracting the fourth product, the quantization parameter value of the macroblock is Among them, the thermal influence coefficient The value range is (0,1), such as the heat influence coefficient It can be set to 0.5, or can be set according to actual conditions, which is not limited in the embodiment of the present invention.

[0073] It can be understood that the embodiment of the present invention determines the quantization parameter value of the macroblock by the heat value and the reference adjustment value of the macroblock. On the one hand, the higher the heat value of the macroblock, the more important the image content corresponding to the macroblock is. Based on the first adjustment formula, it can be concluded that: The larger the result of the calculation, the The larger the calculated result of , the smaller the quantization parameter value of the macroblock will be, that is, the higher the quantization accuracy will be, and the lower the compression will be. On the other hand, the higher the benchmark adjustment value of the macroblock, the higher the importance level of the target object in the macroblock, and the higher the benchmark adjustment value The larger it is, based on the similar principle described above, the smaller the quantization parameter value of the macroblock will be, that is, the higher the quantization accuracy will be, and the lower the compression degree will be. In this way, the targeted macroblock can retain more details during encoding to ensure the image quality of the key attention area.

[0074] Optionally, for the process of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock in step S208-3, the embodiment of the present invention provides a possible implementation manner, that is: according to a preset second adjustment formula, based on a preset basic quantization parameter value, a preset attenuation rate coefficient, and the reference quantization parameter value of the macroblock, calculate the quantization parameter value of the macroblock.

[0075] And the second adjustment formula is: ; Wherein, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the attenuation rate coefficient; the reference quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0076] In this embodiment, for the macroblock without a target, the second adjustment formula can be used to calculate the quantization parameter value of the macroblock. First, subtract the reference quantization parameter value of the macroblock from the basic quantization parameter value to obtain a second difference denoted as ; then, multiply the attenuation rate coefficient by the second difference to obtain a fifth product denoted as ; finally, add the reference quantization parameter value of the macroblock to the fifth product to obtain the quantization parameter value of the macroblock, which is .

[0077] It can be understood that after the target object disappears from the video picture, the attenuation rate coefficient can be used to gradually restore the quantization parameter value of the macroblock to the basic quantization parameter value, which can avoid visual jumps caused by sudden changes in video quality. And the attenuation rate coefficient has a value range of (0, 1), which is used to adjust the speed of change of the quantization parameter value; the larger the attenuation rate coefficient , the faster the change of the quantization parameter value will be; the smaller the attenuation rate coefficient , the slower the change of the quantization parameter value will be.

[0078] For the sake of easy understanding, the embodiment of the present invention provides an example. For example, assume that the attenuation rate coefficient is 0.2, the basic quantization parameter value is 32. For a macroblock at a certain position, at the n-th frame, there is a target object at this position, and its quantization parameter value is 24; at the (n + 1)-th frame, the target object disappears from the video picture, that is, there is no target object at this position, and the quantization parameter value of the macroblock at this position calculated by the second adjustment formula is 25.6; at the (n + 2)-th frame, there is no target object at this position, and the quantization parameter value of the macroblock at this position calculated by the second adjustment formula is 26.88. Calculating in a similar way, it can be obtained that starting from the n-th frame, after 7 video frames, the quantization parameter value of the macroblock at this position returns to 30.4; after 30 video frames, the quantization parameter value of the macroblock at this position is about 32.

[0079] Moreover, the attenuation rate coefficient The specific value of can also be set according to the type of the target object. For example, for a type with a higher importance level, the attenuation rate coefficient is smaller, so as to ensure that after the target object disappears, the change of the video quality is slow, thereby improving the visual experience. For example, in the above-mentioned substation monitoring scenario, the target objects can be three types: instruments, switch devices, and personnel. Since the importance level of the instrument type is high, the attenuation rate coefficient can be set to 0.3; since the importance level of the switch device type is medium, the attenuation rate coefficient can be set to 0.2; since the importance level of the personnel type is low, the attenuation rate coefficient can be set to 0.1. It should be understood that the specific value of the attenuation rate coefficient can also be set according to the actual situation, and the embodiments of the present invention are not limited thereto.

[0080] In order to execute the corresponding steps in the above embodiments and each possible way, an implementation manner of a video encoding device is given below. Please refer to Figure 4 , which is a functional block diagram of the video encoding device provided by the embodiments of the present invention. It should be noted that for the video encoding device 300 provided in this embodiment, its basic principle and the generated technical effects are the same as those in the above embodiments. For a brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding content in the above embodiments. The video encoding device 300 includes: A determination module 310, configured to sequentially use each video frame in the video as a to-be-processed video frame, determine the target area where the target object is located in the to-be-processed video frame, and determine the positional relationship between all macroblocks in the to-be-processed video frame and the target area.

[0081] The processing module 330 is configured to obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, so as to obtain the reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame; according to the positional relationship and the reference two-dimensional heat information, obtain the two-dimensional heat information of the video frame to be processed, and update the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information.

[0082] The encoding module 350 is configured to determine the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

[0083] Optionally, the processing module 330 is further configured to: determine the target indication value of each macroblock in the video frame to be processed according to the positional relationship; determine the reference heat value of each macroblock in the video frame to be processed according to the reference two-dimensional heat information; wherein, the reference heat value of the macroblock is the heat value of the macroblock with the same position as the macroblock in the reference two-dimensional heat information; calculate the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed, so as to obtain the two-dimensional heat information of the video frame to be processed.

[0084] Optionally, the processing module 330 is further configured to: for each macroblock in the video frame to be processed, calculate the heat value of the macroblock according to a preset heat formula based on a preset time decay factor, the target indication value of the macroblock, and the reference heat value, so as to obtain the heat value of each macroblock in the video frame to be processed; the heat formula is: ; Wherein, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the target indication value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0085] Optionally, the encoding module 350 is further configured to: for each macroblock in the video frame to be processed, if the macroblock intersects with the target area, determine the quantization parameter value of the macroblock based on the heat value of the macroblock; if the macroblock does not intersect with the target area, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein, the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock with the same position as the macroblock in the previous video frame of the video frame to be processed.

[0086] Optionally, the encoding module 350 is further configured to: obtain a reference adjustment value of a macroblock according to the type of the target object in the target region intersecting the macroblock; determine a quantization parameter value of the macroblock according to the heat value of the macroblock and the reference adjustment value.

[0087] Optionally, the encoding module 350 is further configured to: calculate the quantization parameter value of the macroblock according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset heat influence coefficient, the heat value of the macroblock, and the reference adjustment value; the first adjustment formula is: ; Wherein, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the heat influence coefficient, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0088] Optionally, the encoding module 350 is further configured to: calculate the quantization parameter value of the macroblock according to a preset second adjustment formula based on a preset basic quantization parameter value, a preset attenuation rate coefficient, and the reference quantization parameter value of the macroblock; the second adjustment formula is: ; Wherein, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the attenuation rate coefficient; the reference quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed.

[0089] An embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the video encoding method disclosed in the embodiment of the present invention is implemented.

[0090] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the video encoding method disclosed in the embodiment of the present invention is implemented.

[0091] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0092] In addition, in each embodiment of the present invention, the various functional modules can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0093] If the above functions are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0094] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video encoding method, characterized in that, The method includes: Taking each video frame in the video as a video frame to be processed in sequence, determining a target area where a target object is located in the video frame to be processed, and determining the positional relationship between all macroblocks in the video frame to be processed and the target area; Obtaining the two-dimensional heat information of the video frame immediately preceding the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, to obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame; Obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and updating the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information; Determining the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed, and encoding the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

2. The video encoding method according to claim 1, wherein The two-dimensional heat information includes the heat value of each macroblock in the corresponding video frame, and the heat value of the macroblock represents the importance degree of the image content corresponding to the macroblock; The step of obtaining the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information includes: Determining the target indication value of each macroblock in the video frame to be processed according to the positional relationship; Determining the reference heat value of each macroblock in the video frame to be processed according to the reference two-dimensional heat information; wherein, the reference heat value of the macroblock is the heat value of the macroblock with the same position as this macroblock in the reference two-dimensional heat information; Calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed, to obtain the two-dimensional heat information of the video frame to be processed.

3. The video encoding method according to claim 2, wherein The step of calculating the heat value of each macroblock in the video frame to be processed according to the target indication value and the reference heat value of each macroblock in the video frame to be processed includes: For each macroblock in the video frame to be processed, calculating the heat value of the macroblock according to a preset heat formula based on a preset time decay factor, the target indication value and the reference heat value of the macroblock, to obtain the heat value of each macroblock in the video frame to be processed; the heat formula is: ； Among them, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the time decay factor, represents the reference heat value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the target indication value of the macroblock at the i-th row and j-th column in the video frame to be processed.

4. The video encoding method according to claim 1, wherein The two-dimensional heat information of the video frame to be processed includes the heat value of each macroblock in the video frame to be processed; the positional relationship includes whether each macroblock in the video frame to be processed intersects with the target area; The step of determining the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed includes: For each macroblock in the video frame to be processed, if the macroblock intersects with the target area, determining the quantization parameter value of the macroblock based on the heat value of the macroblock; If the macroblock does not intersect with the target region, obtain the reference quantization parameter value of the macroblock, and determine the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock; wherein, the reference quantization parameter value of the macroblock is the quantization parameter value of the macroblock at the same position as this macroblock in the previous video frame of the video frame to be processed.

5. The video encoding method according to claim 4, wherein The step of determining the quantization parameter value of the macroblock based on the heat value of the macroblock includes: Obtain the baseline adjustment value of the macroblock according to the type of the target object in the target region intersecting with the macroblock; Determine the quantization parameter value of the macroblock according to the heat value and the baseline adjustment value of the macroblock.

6. The video encoding method according to claim 5, wherein The step of determining the quantization parameter value of the macroblock according to the heat value and the baseline adjustment value of the macroblock includes: Calculate the quantization parameter value of the macroblock according to a preset first adjustment formula based on a preset basic quantization parameter value, a preset heat influence coefficient, the heat value of the macroblock, and the baseline adjustment value; the first adjustment formula is: ； Among them, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the basic quantization parameter value, represents the reference adjustment value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the heat influence coefficient, represents the heat value of the macroblock at the i-th row and j-th column in the video frame to be processed.

7. The video encoding method according to claim 4, wherein The step of determining the quantization parameter value of the macroblock based on the reference quantization parameter value of the macroblock includes: Calculate the quantization parameter value of the macroblock according to a preset second adjustment formula based on a preset basic quantization parameter value, a preset attenuation rate coefficient, and the reference quantization parameter value of the macroblock; the second adjustment formula is: ； Among them, represents the quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed, represents the base quantization parameter value, represents the attenuation rate coefficient; The reference quantization parameter value of the macroblock at the i-th row and j-th column in the video frame to be processed.

8. A video encoding device, characterized in that The device includes: A determination module, configured to sequentially use each video frame in the video as a video frame to be processed, determine the target region where the target object is located in the video frame to be processed, and determine the positional relationship between all macroblocks in the video frame to be processed and the target region; A processing module, configured to obtain the two-dimensional heat information of the previous video frame of the video frame to be processed from the two-dimensional heat information of each historical video frame included in the three-dimensional heat information, to obtain reference two-dimensional heat information; the two-dimensional heat information represents the importance degree of the image content corresponding to all macroblocks in the corresponding video frame; Obtain the two-dimensional heat information of the video frame to be processed according to the positional relationship and the reference two-dimensional heat information, and update the two-dimensional heat information of the video frame to be processed to the three-dimensional heat information; An encoding module, configured to determine the quantization parameter value of each macroblock in the video frame to be processed according to the positional relationship and the two-dimensional heat information of the video frame to be processed, and encode the video frame to be processed according to the quantization parameter value of each macroblock in the video frame to be processed.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, it implements the video encoding method according to any one of claims 1-7.

10. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the video encoding method according to any one of claims 1-7.

Images