Video coding method and decoding method, bit stream storage method and device, electronic equipment and storage medium

Abstract

The invention relates to a video coding method, a video decoding method, a bit stream storage method, a bit stream storage device, electronic equipment and a storage medium, and the method comprises the following steps: for each pixel block in a plurality of pixel blocks contained in a current video frame, executing the following operations: identifying a target semantic type of the content contained in the pixel block by using a semantic segmentation model, detecting the target saliency degree of the content contained in the pixel block by using a saliency detection model; calculating a quantization parameter increment based on the target semantic type and the target saliency degree; adjusting the quantization parameter value of the pixel block based on the quantization parameter increment; and encoding the pixel block based on the adjusted quantization parameter value.

Description

Technical Field

[0001] This disclosure relates to the field of video encoding and decoding, and more specifically, to video encoding methods, decoding methods, methods for storing bit streams, apparatus, electronic devices, and storage media. Background Technology

[0002] With the increasing richness of video content and the rising demands for visual quality from users, improving the subjective quality of video coding under limited bandwidth and storage resources has become an important research direction in the field of video coding. Traditional video coding methods mainly focus on optimizing the overall rate-distortion of video coding, ignoring the differences in attention paid to different areas by the Human Visual System (HVS), resulting in uneven allocation of coding resources, which affects the subjective viewing experience.

[0003] To address the aforementioned issues, a region-aware coding method based on visual saliency has been proposed in related technologies. Saliency detection is a bottom-up attention model that identifies the most attention-grabbing regions in an image or video by simulating the human eye's attention mechanism. Specifically, the saliency detection scheme primarily utilizes computer vision algorithms to simulate the bottom-up attention mechanism of the human visual system, automatically identifying visually prominent regions in an image that are easily attracted to human attention as Regions of Interest (ROIs).

[0004] However, saliency detection cannot distinguish the semantic meaning of image regions. For example, a billboard in an image may be more salient than a face, but for the user, the face is usually more important. In such scenarios, the regions obtained through saliency detection are not necessarily the regions the user is truly interested in. Especially in complex scenes, with significant lighting variations, or with heavy background interference, saliency detection algorithms may produce misclassifications. That is, unimportant regions may be incorrectly assigned high bitrates, while important regions may be ignored, leading to poor encoding quality. Summary of the Invention

[0005] This disclosure provides video encoding methods, decoding methods, methods, apparatuses, electronic devices, and storage media for storing bit streams, in order to at least solve the problem in the aforementioned related technologies that saliency detection algorithms may produce misjudgments, thereby leading to poor encoding quality.

[0006] According to a first aspect of the present disclosure, a video encoding method is provided, comprising: for each pixel block among a plurality of pixel blocks contained in a current video frame, performing the following operations: identifying the target semantic type of the content contained in the pixel block using a semantic segmentation model, and detecting the target saliency of the content contained in the pixel block using a saliency detection model; calculating a quantization parameter increment based on the target semantic type and the target saliency; adjusting the quantization parameter value of the pixel block based on the quantization parameter increment; and encoding the pixel block based on the adjusted quantization parameter value.

[0007] Optionally, calculating the quantization parameter increment based on the target semantic type and the target saliency includes: determining the target subjective value weight corresponding to the pixel block based on the target semantic type, wherein each semantic type is preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight; calculating the quantization parameter increment based on the target subjective value weight and the target saliency, wherein the quantization parameter increment is negatively correlated with the target subjective value weight and the quantization parameter increment is negatively correlated with the target saliency.

[0008] Optionally, it further includes: determining a first number of pixel blocks in the current video frame that have the same saliency as the target saliency; the step of calculating the quantization parameter increment based on the target subjective value weight and the target saliency includes: calculating the quantization parameter increment based on the target subjective value weight, the target saliency, and the first number, wherein the quantization parameter increment is negatively correlated with the first number.

[0009] Optionally, calculating the quantization parameter increment based on the target subjective value weight, the target salience, and the first quantity includes: calculating the ratio between the first quantity and the second quantity, wherein the second quantity is the number of the plurality of pixel blocks; and calculating the quantization parameter increment based on the target subjective value weight, the target salience, and the ratio, wherein the quantization parameter increment is negatively correlated with the ratio.

[0010] Optionally, it further includes: obtaining the target texture complexity of the content contained in the pixel block; the step of calculating the quantization parameter increment based on the target subjective value weight and the target saliency includes: calculating the quantization parameter increment based on the target subjective value weight, the target saliency and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

[0011] According to a second aspect of the present disclosure, a video decoding method is provided, comprising: for each pixel block among a plurality of pixel blocks contained in a current video frame, performing the following operations: receiving a target semantic type and a target saliency of the pixel block, wherein the target semantic type is the semantic type of the content contained in the pixel block identified using a semantic segmentation model, and the target saliency is the saliency of the content contained in the pixel block detected using a saliency detection model; calculating a quantization parameter increment based on the target semantic type and the target saliency; adjusting the quantization parameter value of the pixel block based on the quantization parameter increment; and decoding the pixel block based on the adjusted quantization parameter value.

[0012] Optionally, calculating the quantization parameter increment based on the target semantic type and the target saliency includes: determining the target subjective value weight corresponding to the pixel block based on the target semantic type, wherein each semantic type is preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight; calculating the quantization parameter increment based on the target subjective value weight and the target saliency, wherein the quantization parameter increment is negatively correlated with the target subjective value weight and the quantization parameter increment is negatively correlated with the target saliency.

[0013] Optionally, it further includes: receiving a first quantity, wherein the first quantity is the number of pixel blocks in the plurality of pixel blocks contained in the current video frame that have the same saliency as the target saliency; the step of calculating the quantization parameter increment based on the target subjective value weight and the target saliency includes: calculating the quantization parameter increment based on the target subjective value weight, the target saliency and the first quantity, wherein the quantization parameter increment is negatively correlated with the first quantity.

[0014] Optionally, calculating the quantization parameter increment based on the target subjective value weight, the target salience, and the first quantity includes: calculating the ratio between the first quantity and the second quantity, wherein the second quantity is the number of the plurality of pixel blocks; and calculating the quantization parameter increment based on the target subjective value weight, the target salience, and the ratio, wherein the quantization parameter increment is negatively correlated with the ratio.

[0015] Optionally, it further includes: receiving the target texture complexity of the content contained in the pixel block; the step of calculating the quantization parameter increment based on the target subjective value weight and the target saliency includes: calculating the quantization parameter increment based on the target subjective value weight, the target saliency and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

[0016] According to a third aspect of the present disclosure, a video encoding apparatus is provided, comprising: a semantic type and saliency acquisition module, configured to, for each pixel block among a plurality of pixel blocks contained in a current video frame, identify the target semantic type of the content contained in the pixel block using a semantic segmentation model, and detect the target saliency of the content contained in the pixel block using a saliency detection model; a quantization parameter increment calculation module, configured to calculate a quantization parameter increment based on the target semantic type and the target saliency; an adjustment module, configured to adjust the quantization parameter value of the pixel block based on the quantization parameter increment; and an encoding module, configured to encode the pixel block based on the adjusted quantization parameter value.

[0017] Optionally, the quantization parameter increment calculation module is configured to: determine the target subjective value weight corresponding to the pixel block based on the target semantic type, wherein each semantic type is preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight; calculate the quantization parameter increment based on the target subjective value weight and the target saliency, wherein the quantization parameter increment is negatively correlated with the target subjective value weight and the quantization parameter increment is negatively correlated with the target saliency.

[0018] Optionally, the video encoding apparatus further includes: a quantity determination module configured to determine a first number of pixel blocks in the plurality of pixel blocks contained in the current video frame that have the same saliency as the target saliency; and a quantization parameter increment calculation module configured to calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the first number, wherein the quantization parameter increment is negatively correlated with the first number.

[0019] Optionally, the quantization parameter increment calculation module is configured to: calculate the ratio between the first quantity and the second quantity, wherein the second quantity is the number of the plurality of pixel blocks; and calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the ratio, wherein the quantization parameter increment is negatively correlated with the ratio.

[0020] Optionally, the video encoding apparatus further includes: a texture complexity acquisition module, configured to acquire the target texture complexity of the content contained in the pixel block; and a quantization parameter increment calculation module configured to: calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

[0021] According to a fourth aspect of the present disclosure, a video decoding apparatus is provided, comprising: a semantic type and saliency receiving module configured to receive a target semantic type and a target saliency for each pixel block among a plurality of pixel blocks contained in a current video frame, wherein the target semantic type is the semantic type of the content contained in the pixel block identified by a semantic segmentation model, and the target saliency is the saliency of the content contained in the pixel block detected by a saliency detection model; an incremental calculation module configured to calculate a quantization parameter increment based on the target semantic type and the target saliency; a parameter value adjustment module configured to adjust the quantization parameter value of the pixel block based on the quantization parameter increment; and a decoding module configured to decode the pixel block based on the adjusted quantization parameter value.

[0022] Optionally, the incremental calculation module is configured to: determine the target subjective value weight corresponding to the pixel block based on the target semantic type, wherein each semantic type is preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight; calculate the quantization parameter increment based on the target subjective value weight and the target saliency, wherein the quantization parameter increment is negatively correlated with the target subjective value weight and the quantization parameter increment is negatively correlated with the target saliency.

[0023] Optionally, the video decoding device further includes: a quantity receiving module configured to receive a first quantity, wherein the first quantity is the number of pixel blocks in the plurality of pixel blocks contained in the current video frame that have the same saliency as the target saliency; and an increment calculation module configured to: calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the first quantity, wherein the quantization parameter increment is negatively correlated with the first quantity.

[0024] Optionally, the incremental calculation module is configured to: calculate the ratio between the first quantity and the second quantity, wherein the second quantity is the number of the plurality of pixel blocks; and calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the ratio, wherein the quantization parameter increment is negatively correlated with the ratio.

[0025] Optionally, the video decoding device further includes: a texture complexity receiving module configured to receive the target texture complexity of the content contained in the pixel block; and an incremental calculation module configured to calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

[0026] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a video encoding method or a video decoding method according to the present disclosure.

[0027] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform a video encoding method or a video decoding method according to the present disclosure.

[0028] According to a seventh aspect of the present disclosure, a method for storing a bitstream is provided, comprising: generating a bitstream according to a video encoding method of the present disclosure; and storing the bitstream.

[0029] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In this disclosure, when encoding video frames, by fusing information from both saliency (visual attention) and semantic segmentation (semantic understanding), not only can the natural attention of the human visual system to salient regions be considered, but semantic importance scoring is also introduced. This involves more comprehensive and richer factors, thus effectively reducing misjudgments. In this way, encoding resources can be allocated as much as possible to areas that better align with human subjective perception, ensuring that areas truly of human attention receive higher encoding quality, thereby improving the user's viewing experience. Simultaneously, it avoids the incorrect allocation of high bitrates to unimportant areas, conserving bitrate resources.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0032] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure.

[0033] Figure 2 This is a schematic diagram illustrating a video coding architecture according to an exemplary embodiment of the present disclosure.

[0034] Figure 3 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0035] Figure 4 This is a flowchart illustrating a video decoding method according to an exemplary embodiment of the present disclosure.

[0036] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment of the present disclosure.

[0037] Figure 6 This is a block diagram illustrating a video decoding apparatus according to an exemplary embodiment of the present disclosure.

[0038] Figure 7 This is a schematic diagram illustrating a computing environment coupled with a user interface. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0040] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0041] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0042] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1As shown, system 10 includes a source device 12 that generates and encodes video data that will later be decoded by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0043] In some implementations, the target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14.

[0044] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the target device 14 can access the encoded video data in the storage device 32 via the input interface 28.

[0045] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0046] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. Alternatively, the encoded video data can be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback.

[0047] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0048] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4 Part 10, AVC) or extensions of such standards. It should be understood that this application is not limited to any particular video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally understood that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally understood that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0049] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0050] Most related technologies utilize either visual saliency or semantic understanding information alone for video coding, failing to fully integrate the advantages of both. For visual saliency, video coding can be performed using, but is not limited to, the Itti-Koch model, Phase-based Quaternion Fourier Transform (PQFT), and deep learning methods.

[0051] The Itti-Koch model is a classic method for modeling biological visual attention. It primarily generates saliency maps by simulating the human eye's response to low-level features such as brightness, color, and orientation. For example, this model can be used to generate frame-level saliency maps and mark Regions of Interest (ROIs). Based on these marked ROIs, adaptive bitrate control strategies for High Efficiency Video Coding (HEVC) encoders can then be developed.

[0052] PQFT is a saliency modeling method in the frequency domain, primarily suitable for processing video content rich in high-frequency features such as motion, texture, and edges. For example, a PQFT-based HEVC subjective-driven complexity control method has been proposed, which can concentrate coding complexity on user-perceived salient regions, thereby achieving a trade-off between perceptual quality and computational resources.

[0053] Regarding deep learning-based methods, with the development of deep learning technology in recent years, more and more research is using convolutional neural networks (CNNs) to detect salient target regions in video frames. These deep learning models can learn complex features and output high-precision pixel-level saliency maps. For example, the Global Context-Aware Progressive Aggregation Network (GCPANet) can also be used to extract salient regions and guide the image ROI compressor to allocate more bitrate resources to important regions, thereby achieving better subjective compression benefits.

[0054] However, as mentioned earlier, in some scenarios, the regions obtained through saliency detection may not be the regions that the user is truly interested in. Especially in complex scenes, with significant lighting variations, or with heavy background interference, saliency detection algorithms may produce misjudgments. That is, unimportant regions may be incorrectly assigned high bitrates, while important regions may be ignored, leading to poor encoding quality.

[0055] Region-aware coding based on semantic understanding primarily utilizes semantic segmentation models (top-down attention models) to identify specific semantic regions (e.g., people, sky, ground, etc.) in images or videos, and then differentiates these regions during the encoding process. For example, a semantic-aware compression (SAC) method specifically designed for autonomous driving scenarios has been proposed. This method identifies regions with key driving significance (e.g., vehicles, pedestrians, road signs, etc.) in images through semantic segmentation and uses higher-quality configurations for these regions during encoding, thereby optimizing the overall perception effect. However, although semantic understanding has made significant progress in the field of vision, its application in video coding remains relatively limited.

[0056] In order to solve the above-mentioned problems in the related technologies, this disclosure provides video encoding methods, decoding methods, methods, apparatuses, electronic devices and storage media for storing bit streams. Figure 2 A schematic diagram illustrating a video coding architecture according to an exemplary embodiment of the present disclosure is shown. (Refer to...) Figure 2 The video coding architecture provided in this disclosure may include five parts: input video frame sequence, pre-coding processing, construction of a unified Map array, generation of coding control parameters, and differential coding execution.

[0057] For the "pre-encoding processing" part, a semantic segmentation model (e.g., Bilateral Segmentation Network, BiSeNet) can be used to generate a semantic segmentation map of the current video frame. For example, it can identify various types of semantic regions contained in the current video frame, such as, but not limited to, human regions, sky regions, ground regions, etc.; and a saliency detection model (e.g., deep learning models such as U-squared Net, U2-Net, etc.) can be used to generate a saliency map of the current video frame to identify the regions of human eye focus in the current video frame.

[0058] Regarding the "constructing a unified Map array" part, the aforementioned saliency map and semantic segmentation map can be merged into a unified multi-channel array map[], where each channel can store different types of information under different dimensions. For example, it can include, but is not limited to: saliency dimension, semantic understanding dimension (e.g., it can include, but is not limited to: character type, sky type, grass type), texture complexity dimension, etc.

[0059] The "Generation of Coding Control Parameters" section can be further divided into four parts: subjective value weighting, normalized area weight calculation, calculation of quantization parameter increment ΔQP, and quantization parameter (QP) range amplification mechanism.

[0060] "Subjective value weighting" can refer to assigning a subjective value weight to each pixel block based on a large number of offline experimental results. This subjective value weight can be used to characterize the user's preference for the corresponding semantic type. Furthermore, the higher the user's preference for the corresponding semantic type, the greater the subjective value weight can be.

[0061] "Normalized area weight calculation" mainly refers to the calculation of the proportion of pixel blocks with the same degree of salience in the entire video frame. The larger this proportion is, the more bitrate resources can be allocated to the pixel blocks with the same degree of salience, that is, the better the encoding quality can be.

[0062] "Calculating the quantization parameter increment ΔQP" can refer to calculating the quantization parameter increment for each pixel block based on the aforementioned semantic segmentation results, saliency detection results, texture complexity detection results, etc.

[0063] The "QP range amplification mechanism" can refer to the fact that the quantization parameter increment ΔQP originally needs to be controlled within a certain range, but in some scenarios, the value of ΔQP can break through the above range limit. For example, it can be taken as a smaller value, thereby ensuring better coding quality.

[0064] Regarding the "differentiated encoding execution" part, since the quantization parameter increment can be calculated separately for different pixel blocks according to attributes such as the semantic type, saliency, and texture complexity of the pixel block, and then the corresponding pixel block can be encoded based on the calculated quantization parameter increment. Therefore, this disclosure can realize differentiated encoding for each pixel block in a video frame, meaning that the encoding quality obtained by different pixel blocks may be different from each other.

[0065] Thus, in this disclosure, by fusing visual saliency and high-level semantic understanding during video frame encoding, a coding control strategy oriented towards human visual attention can be implemented. This provides a more intelligent, robust, and human-perceived video / image coding bitrate control scheme. This scheme not only considers the natural focus of the human visual system on salient regions but also introduces semantic importance scores for key semantic targets (e.g., but not limited to: people, sky, ground, etc.), enabling coding resources to be allocated to regions that better align with human subjective perception values, thereby improving coding quality.

[0066] Figure 3 This is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0067] Reference Figure 3 It can obtain a sequence of video frames to be encoded, which may contain multiple video frames. For each pixel block in the current video frame, the following operations can be performed.

[0068] In step 301, a semantic segmentation model can be used to identify the target semantic type of the content contained in the pixel block, and a saliency detection model can be used to detect the target saliency of the content contained in the pixel block.

[0069] For example, as mentioned earlier, a semantic segmentation model can be used to generate a semantic segmentation map of the current video frame, which can identify various types of semantic regions contained in the current video frame. Furthermore, a saliency detection model can be used to generate a saliency map of the current video frame to identify regions of human visual interest within the current video frame.

[0070] Next, a unified Map array can be constructed. For example, as mentioned earlier, the aforementioned saliency map and semantic segmentation map can be merged into a unified multi-channel array map[], where each channel can store different types of information under different dimensions. For example, it can include, but is not limited to: saliency dimension, semantic understanding dimension (e.g., it can include, but is not limited to: person type, sky type, grass type), texture complexity dimension, etc.

[0071] It should be noted that the aforementioned dimensions and types are merely exemplary classification methods. Other dimensions can be added or classified into other types based on actual needs, and this disclosure does not impose specific limitations in this regard. Furthermore, in this disclosure, the classification can be based on, but is not limited to, 4 pixels. 4 pixels, 16 pixels The video frame is divided into multiple pixel blocks in units of 16 pixels, and this disclosure does not impose any specific restrictions on this.

[0072] In step 302, the quantization parameter increment ΔQP can be calculated based on the aforementioned target semantic type and target saliency.

[0073] According to an exemplary embodiment of this disclosure, the target subjective value weight corresponding to a pixel block can be determined based on the target semantic type, wherein each semantic type can be preset with a corresponding subjective value weight. This subjective value weight can be used to characterize the user's preference for the corresponding semantic type, and the higher the user's preference for the corresponding semantic type, the greater the subjective value weight can be.

[0074] It should be noted that, based on offline experimental results, in a large number of short video samples, although the frequency of human figures in the videos is relatively low (detection rate of approximately 40%), their subjective coding performance improvement is the most significant; while salient regions, although occurring more frequently (80%), are not entirely equivalent to high-value regions. Therefore, in this disclosure, subjective value weights corresponding to various semantic types can be pre-set based on a large number of offline experimental results, that is, subjective value weights can be assigned to regions of different semantic types. For example, since the human eye may have a higher preference for human figures, the subjective value weight of human figure regions can be set higher, so that human figures can dominate in the final coding control chart.

[0075] Next, the quantitative parameter increment ΔQP can be calculated based on the target subjective value weight and the target significance. This quantitative parameter increment can be negatively correlated with the target subjective value weight and also negatively correlated with the target significance.

[0076] It should be noted that the smaller the QP value, the more bitrate resources are allocated, and therefore, the higher the encoding quality. Since the quantization parameter increment ΔQP can be negatively correlated with the target subjective value weight and the target saliency, the larger the subjective value weight (i.e., the higher the user's preference for the corresponding semantic type) and the greater the target saliency (i.e., the higher the user's attention to the corresponding region), the smaller the quantization parameter increment ΔQP can be. Consequently, the quantization parameter adjusted based on this increment ΔQP will also be smaller, allowing for the allocation of more bitrate resources to the corresponding pixel block, thus improving the encoding quality of that pixel block. In other words, this disclosure allows for the allocation of more bitrate resources to pixel blocks with higher user preference and attention, thereby improving the encoding quality of those pixel blocks and ensuring a better user viewing experience.

[0077] According to exemplary embodiments of this disclosure, a first number of pixel blocks having the same salience as the target salience among a plurality of pixel blocks contained in the current video frame can also be determined. Exemplarily, salience can also be referred to as salience level, and salience level can include, but is not limited to: not salience, relatively salience, and very salience. Therefore, a first number of pixel blocks having the same salience level as the target salience among a plurality of pixel blocks contained in the current video frame can be determined.

[0078] Then, based on the aforementioned subjective value weight of the target, the aforementioned saliency of the target, the first quantity, and the second quantity corresponding to multiple pixel blocks, the quantization parameter increment can be calculated, wherein the quantization parameter increment can be negatively correlated with the first quantity.

[0079] It should be noted that the more pixel blocks with the same salience in the current video frame, the larger the frame size occupied by the content with that salience, and thus the more likely that the content with that salience is the core content that the current video frame wants to present. Therefore, the quantization parameter increment ΔQP can be negatively correlated with the first number of pixel blocks with the same salience; that is, the larger the first number, the smaller the quantization parameter increment ΔQP can be, and consequently, the smaller the quantization parameter after adjustment based on this increment ΔQP. As mentioned earlier, the smaller the QP value, the more bitrate resources are allocated, and the higher the encoding quality. Therefore, in this disclosure, more bitrate resources can be allocated to pixel blocks with the same salience, ensuring that pixel blocks with the same salience receive better encoding quality, which in turn ensures that the core content in the video frame receives better encoding performance, thereby ensuring a better user viewing experience.

[0080] According to an exemplary embodiment of this disclosure, a ratio between the first quantity and the second quantity can also be calculated; that is, a ratio can also be calculated between the first quantity of pixel blocks with the same salience and the second quantity of multiple pixel blocks contained in the current video frame. Then, a quantization parameter increment can be calculated based on the aforementioned target subjective value weight, the aforementioned target salience, and the ratio, wherein the quantization parameter increment ΔQP can be negatively correlated with the ratio.

[0081] It should be noted that if the proportion of pixels with the same salience in the current video frame is larger, then the content with that salience is likely to be the core content that the current video frame wants to present. Therefore, the quantization parameter increment ΔQP can be set to be negatively correlated with the aforementioned ratio. That is, the larger the proportion of content with the same salience in the current video frame, the smaller the quantization parameter increment ΔQP can be, and consequently, the smaller the quantization parameter after adjustment based on this increment ΔQP will also be. As mentioned earlier, the smaller the QP value, the more bitrate resources are allocated, and the higher the encoding quality. Therefore, in this disclosure, more bitrate resources can be allocated to content with the same salience, which ensures that content with the same salience receives better encoding quality, and thus ensures that the core content in the video frame receives better encoding performance, thereby ensuring a better user viewing experience.

[0082] According to an exemplary embodiment of this disclosure, the target texture complexity of the content contained in the pixel block can also be obtained. Then, the quantization parameter increment ΔQP can be calculated based on the aforementioned target subjective value weight, the aforementioned target saliency, and the target texture complexity, wherein the quantization parameter increment ΔQP can be negatively correlated with the target texture complexity.

[0083] It should be noted that the higher the texture complexity of a scene region, the more complex the ideas it may contain, meaning the richer the meaning it can represent. Therefore, the quantization parameter increment ΔQP can be negatively correlated with the texture complexity; that is, the higher the texture complexity of a scene region, the smaller the quantization parameter increment ΔQP can be, and consequently, the smaller the quantization parameter after adjustment based on this increment ΔQP. As mentioned earlier, the smaller the QP value, the more bitrate resources are allocated, and the higher the encoding quality. Therefore, in this disclosure, more bitrate resources can be allocated to scene regions with higher texture complexity, ensuring better encoding quality for these regions, and thus better encoding performance for them within video frames. This ensures that the rich ideas contained in these texture regions can be expressed more completely and accurately, resulting in a better user viewing experience.

[0084] For example, the quantization parameter increment can be calculated using the following formula: ΔQP= .

[0085] Where ΔQP is the increment of the quantization parameter, The target subjective value weight of the aforementioned pixel blocks, This is the ratio between the first number of pixel blocks with the target saliency level mentioned above and the second number of pixel blocks contained in the current video frame. The salience of the aforementioned pixel blocks, As a preset weighting factor, The complexity of the target texture of the aforementioned pixel blocks, This refers to the first number of pixel blocks with a saliency level equal to the target saliency level, as described above. This refers to the second number of pixel blocks contained in the current video frame.

[0086] In addition, the complexity of the target texture α can be the gradient variance normalization value. α can be obtained from a large number of offline experimental results, and the α value will be different for different semantic types. For example, for the character type region, α can be set to, but is not limited to, 2.5. β can be a weighting factor determined based on practical experience; for example, β can be set to, but is not limited to, 1.0.

[0087] It's important to note that the quantization parameter increment ΔQP can be controlled within a certain range. For example, ΔQP can be controlled within the range of 0 to -3, where 0 represents no adjustment to the quantization parameter, meaning the original quantization parameter remains unchanged. Furthermore, the smaller the value of ΔQP, the more bitrate resources can be allocated to the corresponding image area, resulting in better encoding quality. For instance, for the same pixel block, the encoding quality is better when ΔQP is -3 than when it is -1.

[0088] In step 303, the quantization parameter values ​​of the pixel block can be adjusted based on the quantization parameter increment. In this disclosure, the quantization parameter can also be referred to as the encoding control parameter.

[0089] It should be noted that, as mentioned earlier, the quantization parameter increment ΔQP can be controlled within a certain range, but in some scenarios, the value of ΔQP can exceed this range. For example, considering that salient regions and human figures in a video frame may overlap, this overlapping region is likely the area of ​​the image that the user is truly focused on. Therefore, this overlapping region should receive more bitrate resources, i.e., it should receive better encoding quality. Thus, in such scenarios, the limitation range of ΔQP can be appropriately relaxed. For example, the original range of ΔQP is 0 to -3, but at locations where salient regions and human figures overlap, the value of ΔQP can exceed this range; for example, it can be a smaller value: -4, to ensure that the overlapping location receives better encoding performance.

[0090] In step 304, the pixel block can be encoded based on the adjusted quantization parameter value. As mentioned earlier, the smaller the QP value, the more bitrate resources are allocated, and the higher the encoding quality.

[0091] It should be noted that the video encoding method provided in this disclosure can be applied to various encoding processes, such as, but not limited to: bitrate allocation, encoding parameter decision-making (e.g., including but not limited to: encoding mode selection, filtering mode selection), etc.

[0092] In this way, since the quantization parameter increment can be calculated separately for different pixel blocks according to the target semantic type and target saliency of the pixel block, and then the corresponding pixel block is encoded based on the calculated quantization parameter increment, this disclosure can achieve differentiated encoding for each pixel block in the video frame. This ensures that the areas of the image that the user is truly interested in receive better encoding quality, thereby guaranteeing a better user viewing experience. At the same time, it also avoids incorrectly allocating high bitrate resources to areas of less user interest, thus avoiding waste of bitrate resources. In other words, the video encoding method provided by this disclosure can achieve a better balance between user viewing experience and bitrate resource allocation.

[0093] Figure 4 This is a flowchart illustrating a video decoding method according to an exemplary embodiment of the present disclosure.

[0094] Reference Figure 4 In step 401, for each pixel block among the multiple pixel blocks contained in the current video frame, the target semantic type and target saliency of the pixel block can be received. The target semantic type can be the semantic type of the content contained in the pixel block identified by the semantic segmentation model, and the target saliency can be the saliency of the content contained in the pixel block detected by the saliency detection model.

[0095] In step 402, the quantization parameter increment can be calculated based on the target semantic type and the target saliency.

[0096] According to an exemplary embodiment of this disclosure, the target subjective value weight corresponding to the pixel block can be determined based on the target semantic type. Each semantic type can be preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight can be. Then, based on the target subjective value weight and the target saliency, a quantization parameter increment can be calculated, wherein the quantization parameter increment can be negatively correlated with the target subjective value weight and negatively correlated with the target saliency.

[0097] According to an exemplary embodiment of this disclosure, a first quantity may also be received, wherein the first quantity may be the number of pixel blocks in a plurality of pixel blocks contained in the current video frame that have the same salience as the target salience. Then, a quantization parameter increment may be calculated based on the target subjective value weight, the target salience, and the first quantity, wherein the quantization parameter increment may be negatively correlated with the first quantity.

[0098] According to an exemplary embodiment of this disclosure, a ratio between a first quantity and a second quantity can be calculated, wherein the second quantity can be the number of multiple pixel blocks. Then, a quantization parameter increment can be calculated based on the target subjective value weight, the target saliency, and the ratio, wherein the quantization parameter increment can be negatively correlated with the ratio.

[0099] According to an exemplary embodiment of this disclosure, the target texture complexity of the content contained in the pixel block can also be received. Then, a quantization parameter increment can be calculated based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment may be negatively correlated with the target texture complexity.

[0100] In step 403, the quantization parameter value of the pixel block can be adjusted based on the quantization parameter increment.

[0101] In step 404, the pixel block can be decoded based on the adjusted quantization parameter values.

[0102] It should be noted that, in this disclosure, in addition to the resources that can be used Figure 4 Besides the decoding method shown, other decoding methods can also be used. For example, the encoding end can send the encoded pixel block and the corresponding adjusted quantization parameter value to the decoding end, and then the decoding end can decode the corresponding pixel block according to the adjusted quantization parameter value. This disclosure does not impose specific restrictions on the specific decoding method used.

[0103] Figure 5 This is a block diagram illustrating a video encoding apparatus 500 according to an exemplary embodiment of the present disclosure.

[0104] Reference Figure 5 The video encoding device 500 may include a semantic type and saliency acquisition module 501, a quantization parameter increment calculation module 502, an adjustment module 503, and an encoding module 504.

[0105] The semantic type and saliency acquisition module 501 can use a semantic segmentation model to identify the target semantic type of the content contained in the pixel block, and can use a saliency detection model to detect the target saliency of the content contained in the pixel block.

[0106] The quantization parameter increment calculation module 502 can calculate the quantization parameter increment ΔQP based on the aforementioned target semantic type and target saliency.

[0107] According to an exemplary embodiment of this disclosure, the quantization parameter increment calculation module 502 can determine the target subjective value weight corresponding to the pixel block based on the target semantic type, wherein each semantic type can be preset with a corresponding subjective value weight. This subjective value weight can be used to characterize the user's preference for the corresponding semantic type, and the higher the user's preference for the corresponding semantic type, the greater the subjective value weight can be. Then, the quantization parameter increment calculation module 502 can calculate the quantization parameter increment ΔQP based on the target subjective value weight and the target saliency, wherein the quantization parameter increment can be negatively correlated with the target subjective value weight, and can also be negatively correlated with the target saliency.

[0108] According to an exemplary embodiment of this disclosure, the video encoding apparatus 500 may further include a quantity determination module.

[0109] The quantity determination module can determine a first number of pixel blocks in the current video frame that have the same saliency as the target saliency. Then, the quantization parameter increment calculation module 502 can calculate the quantization parameter increment based on the aforementioned target subjective value weight, the aforementioned target saliency, the first number, and the second number corresponding to the multiple pixel blocks, wherein the quantization parameter increment can be negatively correlated with the first number.

[0110] According to an exemplary embodiment of this disclosure, the quantization parameter increment calculation module 502 can calculate the ratio between a first quantity and a second quantity, that is, it can also calculate the ratio between the first quantity of pixel blocks with the same salience and the second quantity of multiple pixel blocks contained in the current video frame. Then, the quantization parameter increment calculation module 502 can calculate the quantization parameter increment based on the aforementioned target subjective value weight, the aforementioned target salience, and the ratio, wherein the quantization parameter increment ΔQP can be negatively correlated with the ratio.

[0111] According to an exemplary embodiment of this disclosure, the video encoding apparatus 500 may further include a texture complexity acquisition module.

[0112] The texture complexity acquisition module can obtain the target texture complexity of the content contained in the pixel block. Then, the quantization parameter increment calculation module 502 can calculate the quantization parameter increment ΔQP based on the aforementioned target subjective value weight, the aforementioned target saliency, and the target texture complexity, wherein the quantization parameter increment ΔQP can be negatively correlated with the target texture complexity.

[0113] The adjustment module 503 can adjust the quantization parameter value of the pixel block based on the quantization parameter increment.

[0114] The encoding module 504 can encode the pixel block based on the adjusted quantization parameter values.

[0115] Figure 6 This is a block diagram illustrating a video decoding apparatus 600 according to an exemplary embodiment of the present disclosure.

[0116] Reference Figure 6 The video decoding device 600 may include a semantic type and salience receiving module 601, an incremental calculation module 602, a parameter value adjustment module 603, and a decoding module 604.

[0117] The semantic type and saliency receiving module 601 can receive the target semantic type and target saliency of each pixel block among the multiple pixel blocks contained in the current video frame. The target semantic type can be the semantic type of the content contained in the pixel block identified by the semantic segmentation model, and the target saliency can be the saliency of the content contained in the pixel block detected by the saliency detection model.

[0118] The incremental calculation module 602 can calculate the increment of quantization parameters based on the target semantic type and the target saliency.

[0119] According to an exemplary embodiment of this disclosure, the incremental calculation module 602 can determine the target subjective value weight corresponding to the pixel block based on the target semantic type. Each semantic type can be preset with a corresponding subjective value weight, and the higher the user's preference for the corresponding semantic type, the greater its subjective value weight can be. Then, the incremental calculation module 602 can calculate the quantization parameter increment based on the target subjective value weight and the target saliency. The quantization parameter increment can be negatively correlated with the target subjective value weight and negatively correlated with the target saliency.

[0120] According to an exemplary embodiment of this disclosure, the video decoding apparatus 600 may further include a quantity receiving module.

[0121] The quantity receiving module can receive a first quantity, which can be the number of pixel blocks in the current video frame that have the same salience as the target salience. Then, the incremental calculation module 602 can calculate the quantization parameter increment based on the target subjective value weight, the target salience, and the first quantity, wherein the quantization parameter increment can be negatively correlated with the first quantity.

[0122] According to an exemplary embodiment of this disclosure, the incremental calculation module 602 can calculate the ratio between a first quantity and a second quantity, wherein the second quantity can be the number of multiple pixel blocks. Then, the incremental calculation module 602 can calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the ratio, wherein the quantization parameter increment can be negatively correlated with the ratio.

[0123] According to an exemplary embodiment of this disclosure, the video decoding apparatus 600 may further include a texture complexity receiving module.

[0124] The texture complexity receiving module can receive the target texture complexity of the content contained in the pixel block. Then, the incremental calculation module 602 can calculate the quantization parameter increment based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment can be negatively correlated with the target texture complexity.

[0125] The parameter value adjustment module 603 can adjust the quantization parameter value of the pixel block based on the quantization parameter increment.

[0126] The decoding module 604 can decode the pixel block based on the adjusted quantization parameter values.

[0127] Figure 7 A computing environment 710 coupled to a user interface 750 is shown. The computing environment 710 may be part of a data processing server. The computing environment 710 includes a processor 720, a memory 730, and an input / output (I / O) interface 740.

[0128] Processor 720 typically controls the overall operation of computing environment 710, such as operations associated with display, data acquisition, data communication, and image processing. Processor 720 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 720 may include one or more modules that facilitate interaction between processor 720 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0129] Memory 730 is configured to store various types of data to support the operation of computing environment 710. Memory 730 may include predefined software 732. Examples of such data include instructions for any application or method operating on computing environment 710, video datasets, image data, etc. Memory 730 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0130] I / O interface 740 provides an interface between processor 720 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 740 can be coupled to encoders and decoders.

[0131] In an embodiment, a non-transitory computer-readable storage medium is also provided, including, for example, a memory 730 containing a plurality of programs and / or storing a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method. The plurality of programs can be executed by a processor 720 in a computing environment 710 to perform the above-described methods. In one example, the plurality of programs can be executed by the processor 720 in the computing environment 710 to receive a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 720 in the computing environment 710 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 720 in the computing environment 710 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 720 in the computing environment 710 to transmit the bitstream or data stream. Alternatively, the non-transitory computer-readable storage medium may store a bitstream or data stream, generated by an encoder using, for example, the encoding methods described above, for use by a decoder when decoding video data. This bitstream includes encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.). The non-transitory computer-readable storage medium may be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0132] In one embodiment, a bitstream generated by the above-described encoding method or a bitstream to be decoded by the above-described decoding method is provided. In another embodiment, a bitstream comprising encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method is provided.

[0133] In one embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 720); and a non-transitory computer-readable storage medium or memory 730 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0134] In one embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, the bitstream including encoded video information generated by the encoding method described above or encoded video information to be decoded by the decoding method described above. In another embodiment, a computer program product including, for example, a plurality of programs in a memory 730, the plurality of programs being executable by a processor 720 in a computing environment 710 to perform the methods described above. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0135] In an embodiment, the computing environment 710 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0136] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a digital storage medium, wherein the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0137] In one embodiment, a method for transmitting a bitstream generated by the encoder described above is also provided. In another embodiment, a method for receiving a bitstream to be decoded by the decoder described above is also provided.

[0138] According to the video encoding method, decoding method, bitstream storage method, apparatus, electronic device, and storage medium disclosed herein, when encoding video frames, by fusing information from both saliency (visual attention) and semantic segmentation (semantic understanding), not only can the natural attention of the human visual system to salient regions be considered, but semantic importance scoring is also introduced simultaneously. The factors considered are more comprehensive and richer, thus effectively reducing misjudgment. This allows encoding resources to be allocated as much as possible to areas that better align with human subjective perception, ensuring that areas truly of human attention receive higher encoding quality, thereby improving the user's viewing experience. Simultaneously, it avoids the incorrect allocation of high bitrates to unimportant areas, saving bitrate resources.

[0139] According to exemplary embodiments of this disclosure, more bitrate resources can be allocated to pixel blocks with higher user preference and attention, thereby improving the encoding quality of the corresponding pixel blocks and ensuring a better user viewing experience.

[0140] According to exemplary embodiments of this disclosure, more bitrate resources can be allocated to pixel blocks with the same degree of salience, which can ensure that pixel blocks with the same degree of salience can obtain better encoding quality, and can also ensure that the core content in the video frame can obtain better encoding performance, thereby ensuring a better user viewing experience.

[0141] According to exemplary embodiments of this disclosure, more bitrate resources can be allocated to image regions with higher texture complexity, which can ensure that image regions with higher texture complexity can obtain better encoding quality, and can also ensure that image regions with higher texture complexity in video frames can obtain better encoding performance. This can ensure that the rich ideas contained in the image regions with higher texture complexity can be expressed more completely and accurately, thereby ensuring a better user viewing experience.

[0142] According to exemplary embodiments of this disclosure, differentiated encoding can be implemented for each pixel block in a video frame, thereby ensuring that the areas of the image that the user is truly interested in receive better encoding quality, thus guaranteeing a better user viewing experience. Simultaneously, it also avoids erroneously allocating high bitrate resources to areas of less user interest, thus preventing waste of bitrate resources. In other words, the video encoding method provided by this disclosure achieves a better balance between user viewing experience and bitrate resource allocation.

[0143] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0144] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0145] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A video encoding method, characterized in that, include: For each pixel block in the current video frame, perform the following operations: The semantic segmentation model is used to identify the target semantic type of the content contained in the pixel block, and the saliency detection model is used to detect the target saliency of the content contained in the pixel block. Calculate the quantization parameter increment based on the target semantic type and the target saliency; The quantization parameter value of the pixel block is adjusted based on the quantization parameter increment; The pixel block is encoded based on the adjusted quantization parameter values.

2. The video encoding method as described in claim 1, characterized in that, The step of calculating the quantization parameter increment based on the target semantic type and the target saliency includes: The target subjective value weight corresponding to the pixel block is determined based on the target semantic type. Each semantic type is preset with a corresponding subjective value weight. The higher the user's preference for the corresponding semantic type, the greater its subjective value weight. Based on the target subjective value weight and the target significance, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the target subjective value weight and the increment of the quantification parameter is negatively correlated with the target significance.

3. The video encoding method as described in claim 2, characterized in that, Also includes: Determine a first number of pixel blocks in the current video frame that have the same salience as the target salience. The calculation of the quantitative parameter increment based on the target's subjective value weight and the target's significance includes: Based on the target subjective value weight, the target significance, and the first quantity, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the first quantity.

4. The video encoding method as described in claim 3, characterized in that, The step of calculating the increment of the quantification parameter based on the target subjective value weight, the target significance, and the first quantity includes: Calculate the ratio between the first quantity and the second quantity, where the second quantity is the number of the plurality of pixel blocks; Based on the target subjective value weight, the target significance, and the ratio, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the ratio.

5. The video encoding method as described in claim 2, characterized in that, Also includes: Obtain the target texture complexity of the content contained in the pixel block; The calculation of the quantitative parameter increment based on the target's subjective value weight and the target's significance includes: The quantization parameter increment is calculated based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

6. A video decoding method, characterized in that, include: For each pixel block in the current video frame, perform the following operations: The target semantic type and target saliency of the pixel block are received, wherein the target semantic type is the semantic type of the content contained in the pixel block as identified by the semantic segmentation model, and the target saliency is the saliency of the content contained in the pixel block as detected by the saliency detection model. Calculate the quantization parameter increment based on the target semantic type and the target saliency; The quantization parameter value of the pixel block is adjusted based on the quantization parameter increment; The pixel block is decoded based on the adjusted quantization parameter values.

7. The video decoding method as described in claim 6, characterized in that, The step of calculating the quantization parameter increment based on the target semantic type and the target saliency includes: The target subjective value weight corresponding to the pixel block is determined based on the target semantic type. Each semantic type is preset with a corresponding subjective value weight. The higher the user's preference for the corresponding semantic type, the greater its subjective value weight. Based on the target subjective value weight and the target significance, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the target subjective value weight and the increment of the quantification parameter is negatively correlated with the target significance.

8. The video decoding method as described in claim 7, characterized in that, Also includes: Receive a first quantity, wherein the first quantity is the number of pixel blocks in the plurality of pixel blocks contained in the current video frame that have the same salience as the target salience; The calculation of the quantitative parameter increment based on the target's subjective value weight and the target's significance includes: Based on the target subjective value weight, the target significance, and the first quantity, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the first quantity.

9. The video decoding method as described in claim 8, characterized in that, The step of calculating the increment of the quantification parameter based on the target subjective value weight, the target significance, and the first quantity includes: Calculate the ratio between the first quantity and the second quantity, where the second quantity is the number of the plurality of pixel blocks; Based on the target subjective value weight, the target significance, and the ratio, the increment of the quantification parameter is calculated, wherein the increment of the quantification parameter is negatively correlated with the ratio.

10. The video decoding method as described in claim 7, characterized in that, Also includes: The complexity of the target texture containing the content of the pixel block; The calculation of the quantitative parameter increment based on the target's subjective value weight and the target's significance includes: The quantization parameter increment is calculated based on the target subjective value weight, the target saliency, and the target texture complexity, wherein the quantization parameter increment is negatively correlated with the target texture complexity.

11. A video encoding device, characterized in that, include: The semantic type and saliency acquisition module is configured to, for each of the multiple pixel blocks contained in the current video frame, use a semantic segmentation model to identify the target semantic type of the content contained in the pixel block, and use a saliency detection model to detect the target saliency of the content contained in the pixel block. The quantization parameter increment calculation module is configured to calculate the quantization parameter increment based on the target semantic type and the target saliency. The adjustment module is configured to adjust the quantization parameter value of the pixel block based on the quantization parameter increment; The encoding module is configured to encode the pixel block based on the adjusted quantization parameter values.

12. A video decoding device, characterized in that, include: The semantic type and saliency receiving module is configured to receive the target semantic type and target saliency of each pixel block among multiple pixel blocks contained in the current video frame, wherein the target semantic type is the semantic type of the content contained in the pixel block identified by the semantic segmentation model, and the target saliency is the saliency of the content contained in the pixel block detected by the saliency detection model. The incremental calculation module is configured to calculate the increment of the quantization parameter based on the target semantic type and the target saliency. The parameter value adjustment module is configured to adjust the quantization parameter value of the pixel block based on the quantization parameter increment; The decoding module is configured to decode the pixel block based on the adjusted quantization parameter values.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video encoding method as described in any one of claims 1 to 5, or to implement the video decoding method as described in any one of claims 6 to 10.

14. A non-transitory computer-readable storage medium for storing computer-executable instructions, characterized in that, When executed by one or more computer processors, the computer-executable instructions cause the one or more computer processors to perform the video encoding method as described in any one of claims 1 to 5, or to perform the video decoding method as described in any one of claims 6 to 10.

15. A method for storing a bit stream, characterized in that, include: The video encoding method according to any one of claims 1 to 5 generates a bitstream; Store the bit stream.

Images