Data encoding method, data decoding method, server and terminal equipment

Abstract

The invention provides a data encoding method, a data decoding method, terminal equipment and a server, relates to the technical field of electronic information, and can solve the problem of low image encoding processing efficiency. According to the specific technical scheme, the coding distortion rate is dynamically adjusted according to the visual sensitive value of the image content, so that the coding duration and the code stream size are reduced and the transmission speed is improved while the visual quality is maintained or improved by more effectively utilizing the characteristics of a human visual system.

Description

Technical Field

[0001] The present disclosure relates to the field of electronic information technology, and in particular to data encoding, data decoding methods, servers and terminal devices. Background Art

[0002] With the rapid development of image processing technology, image coding is a key link in image processing. Its goal is to reduce the amount of data storage and transmission as much as possible while ensuring image quality. Traditional image coding methods often use a unified quantization parameter to process the entire image, ignoring the differences in human visual sensitivity in different areas of the image, resulting in low coding efficiency and quality degradation in areas with rich details. Summary of the invention

[0003] The disclosed embodiment provides a data encoding and decoding method, a server and a terminal device, which can solve the problem of large data volume in the existing data encoding process. The technical solution is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a data encoding method is provided, the method comprising:

[0005] Acquire a target frame image, where the target frame image includes M*N macroblocks;

[0006] Traversing the M*N macroblocks, and determining the visual sensitivity value of each macroblock one by one;

[0007] According to the visual sensitivity value of each macroblock and the preset adjustment factor, a target adjustment factor of the rate-distortion optimization processing of each macroblock is set to minimize the cost of the rate-distortion optimization processing;

[0008] According to the adjusted target factor of each macroblock, the rate-distortion optimization processing cost during the encoding process of each macroblock is calculated to obtain the optimal encoding mode of the current macroblock.

[0009] Wherein, the updating of the target adjustment factor of the rate-distortion optimization processing of each macroblock includes:

[0010] If the visual sensitivity value of the current macroblock is greater than the threshold, the value of the target adjustment factor is increased according to the preset adjustment factor;

[0011] If the visual sensitivity value of the current macroblock is less than or equal to the threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

[0012] According to a second aspect of an embodiment of the present disclosure, a data decoding method is provided, the method comprising:

[0013] Acquire target data, where the target data is coded data after rate-distortion optimization processing is performed according to a target adjustment factor, where the target adjustment factor is set according to a visual sensitivity value of each macroblock;

[0014] Decoding and processing the target data to obtain a target frame image;

[0015] The target adjustment factor is set according to the visual sensitivity value of each macroblock, including:

[0016] If the visual sensitivity value of the current macroblock is greater than the threshold, the value of the target adjustment factor is increased according to the preset adjustment factor, then;

[0017] If the visual sensitivity value of the current macroblock is less than or equal to the threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

[0018] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data encoding method as described in the above example when executing the computer program.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a terminal device is provided, comprising: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data decoding method as described in the above example when executing the computer program.

[0020] The encoding and decoding method provided in the embodiment of the present application can determine the human eye perception degree matched by each macroblock in different frame images, and perform regional processing according to the human eye perception degree of different macroblocks. The higher the human eye perception degree of the current macroblock, the greater the allowed rate-distortion optimization processing cost, and more detailed encoding is performed. The lower the human eye perception degree of the current macroblock, the smaller the allowed rate-distortion optimization processing cost, and the higher the compression rate. Different macroblocks have different encoding strategies, so that more effective information can be retained without affecting the size of the code stream, so that the visual effect of the image reconstructed at the receiving end is better.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1A is a framework diagram 1 of a data processing system provided by an embodiment of the present disclosure;

[0024] Figure 1B It is a framework of a data processing system provided by the embodiment of the present disclosure Figure 2 ;

[0025] Figure 2 is a flow chart of a data encoding method provided by an embodiment of the present disclosure;

[0026] Figure 3 It is a flowchart of a data decoding method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements.

[0028] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] Existing coding methods usually use fixed-size macroblocks for coding, ignoring the differences in visual importance of image content. The human eye has different visual sensitivity values ​​for different areas. It is more sensitive to distortion in high-sensitivity areas (such as edges and areas with rich textures), and has a higher tolerance for distortion in low-sensitivity areas (such as smooth areas). Therefore, adaptively adjusting the macroblock size and coding parameters according to the visual sensitivity value can more effectively utilize coding resources and improve coding efficiency and visual effects.

[0030] With the advancement of technology, the terminal side will develop in an increasingly lightweight direction, that is, the configuration and structure of the terminal will become simpler and the size will become smaller.

[0031] At present, various zero-terminal solutions have been proposed. The so-called zero-terminal means that the terminal only needs to have a screen, a simple CPU and memory, an image receiving and decoding module for processing images, and a reverse control processing module. This solution can greatly simplify the size and configuration of the mobile terminal. All programs of the zero-terminal run in the cloud server. The zero-terminal uses the corresponding application by accessing the cloud server and controls the corresponding application through reverse control. The server will send the real-time processing screen to the zero-terminal. In this way, for the zero-terminal, it is just like operating locally.

[0032] In the prior art, when using a zero terminal to display some videos, such as playing movies, animations or games, the server encodes the image and sends it to the zero terminal through the network for decoding and display. The existing video encoding scheme effectively removes spatial / temporal redundancy through intra-frame and inter-frame prediction methods, but does not consider the visual redundancy of the video. In real scenes, the human visual system (Human Visual System) HVS can perceive visual information such as brightness, color, texture, direction, spatial frequency, etc. in the video scene. It is selective in sensitivity to different scenes, and different areas or objects have different visual importance. Therefore, it is of great significance to introduce HVS into video encoding. It can compress videos more efficiently and achieve better visual perception effects.

[0033] In order to solve the transmission pressure problem of video between the cloud and the local end, the present application provides a data encoding method, which is based on an adaptive macroblock division and encoding method of visual sensitivity values. It aims to achieve refined encoding of video content by dynamically adjusting the macroblock size and encoding parameters, thereby improving the visual quality of the reconstructed image while maintaining efficient encoding.

[0034] The method provided in the embodiment of the present application is based on the human visual system (Human Visual System). In real scenes, the human visual system (HVS) can perceive a variety of visual information in the video, such as brightness, color, texture, direction, spatial frequency, etc. The sensitivity of these visual information to the human eye is selective, that is, different scenes, regions or objects have different visual importance. For example, the human eye is sensitive to edges, textures and color changes, but less sensitive to smooth areas or noise.

[0035] Therefore, it is of great significance to introduce HVS into video coding. By simulating the perceptual characteristics of HVS, video encoders can allocate encoding bitrates more intelligently, thereby reducing bitrates while maintaining subjective visual quality. This can not only improve video compression efficiency, but also optimize video transmission and storage performance.

[0036] In the process of encoding and processing video data, the embodiments of the present application can be implemented through compression technology, transmission optimization, and visual perception algorithms. By performing difference encoding, depth information compression, or compression methods based on stereoscopic visual characteristics on stereoscopic video images, the amount of data can be reduced and the image quality can be maintained. At the same time, the bit rate adaptive technology can be used to dynamically adjust the transmission bit rate according to the network bandwidth to ensure smooth transmission under different network conditions.

[0037] The following is a brief description of the application environment of the data encoding and decoding method provided in the embodiment of the present application:

[0038] See also Figure 1A The system shown in the figure, a data encoding and decoding method provided in the embodiment of the present application is applied to a data processing system 10, the system 10 includes: a server 101 and a terminal device 102; the server 101 and the terminal device 102 can be connected by wire or wirelessly;

[0039] Server 101, which has an image encoding function, and the encoding part in this solution is implemented here;

[0040] The terminal device 102 has an image decoding function and can also be used for the interface and display of images. The decoding part of this solution is implemented here. The terminal device 101 can be a receiving device such as a computer, a mobile phone or a PAD.

[0041] Please refer to Figure 1B The data stream encoding and decoding process between the server 101 and the terminal device 102 is shown as follows:

[0042] The encoding end (server 101) collects video frame data through the image acquisition module, and then pre-processes the frame data through the encoding module, and calculates the visual sensitivity value and adjustment factor, so as to determine the image encoding strategy of each macroblock, and transmit it to the encoding module. The encoding module encodes and processes according to different encoding strategies, generates an encoded code stream, and sends the encoded code stream to the terminal device.

[0043] The decoding end (terminal device 102) obtains the bitstream data sent by the encoding end through the bitstream receiving module, and caches the video bitstream pushed by the encoding end as the data source for decoding; the bitstream cached by the decoding processing module is decoded by the decoding module, and finally the image frame decoded by the module is rendered and displayed.

[0044] The system 10 provided by the embodiment of the present application determines the visual sensitivity value of each macroblock and determines the encoding strategy of each macroblock, so that more effective information can be retained without affecting the size of the code stream, so that the image reconstructed at the receiving end has a better visual effect.

[0045] See also Figure 2 The following embodiments take the above-mentioned server or data encoding device as the execution subject, and apply the method provided in the embodiment of the present application to data encoding processing as an example for specific description. The data encoding method provided in the embodiment of the present application includes the following steps 201-203:

[0046] Step 201: Acquire a target frame image.

[0047] The server may collect video images within a continuous time period to obtain a target frame image, or may collect multiple images to obtain a target frame image.

[0048] The target frame image includes several macroblocks. For example, the entire frame image can be initially divided into several macroblocks of equal size (such as 16x16, 32x32, etc.). The size of the macroblock can be adjusted according to the specific application scenario and encoding requirements.

[0049] Step 202: traverse the plurality of macroblocks and determine the visual sensitivity value of each macroblock one by one.

[0050] The visual sensitivity value of each macroblock is determined one by one according to the visual sensitivity detection algorithm of the target user. The algorithm can be based on a variety of visual characteristics, such as contrast sensitivity, brightness adaptability, texture masking effect, etc. The output of the algorithm is a numerical value that measures the visual sensitivity value of the macroblock.

[0051] Step 203: According to the visual sensitivity value of each macroblock and the preset adjustment factor, a target adjustment factor of the rate-distortion optimization processing of each macroblock is set to minimize the cost of the rate-distortion optimization processing.

[0052] The goal of rate-distortion optimization is to obtain the least possible image distortion at the smallest possible encoding bit rate, thereby improving the encoding efficiency of the encoder and obtaining better image quality under limited transmission or storage resources.

[0053] The above distortion level refers to the difference or error between the encoded image and the original image. This difference may manifest as a decrease in image quality, loss of details or change in color, etc.

[0054] The above-mentioned adjustment factors may include quantization parameters, prediction modes, encoding parameters, etc. Their selection and setting will directly affect the encoding bit rate and distortion.

[0055] The quantization parameter is dynamically adjusted according to the visual sensitivity value of the macroblock, image content and encoding requirements: in areas with complex image content and rich details, a smaller quantization parameter is selected to maintain high image quality; in areas with simple image content and less details, a larger quantization parameter may be selected to reduce the bit rate.

[0056] The target adjustment factor for rate-distortion optimization processing of each macroblock is set, including:

[0057] If the visual sensitivity value of the current macroblock is greater than a preset threshold, the value of the target adjustment factor is increased according to the preset adjustment factor;

[0058] If the visual sensitivity value of the current macroblock is less than or equal to the preset threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

[0059] Step 204: Calculate the rate-distortion optimization processing cost during the encoding process of each macroblock according to the target adjustment factor of each macroblock to obtain the optimal encoding mode of the current macroblock.

[0060] By obtaining the rate-distortion optimization cost allowed by the current macroblock, a balance can be achieved between the image distortion rate and the encoding bit rate:

[0061] According to the visual sensitivity value of each macroblock, the rate-distortion optimization processing cost allowed for each macroblock is set. For macroblocks with higher visual sensitivity values ​​(i.e., areas with higher user perception), smaller coding distortion is allowed to ensure the image quality of these areas. On the contrary, for macroblocks with lower visual sensitivity values, larger coding distortion can be accepted to save coding bitstream.

[0062] In order to achieve this balance, encoders usually use a series of complex algorithms and technologies, such as content-based coding, rate-distortion optimization, visual perception models, etc. These technologies can help encoders more accurately identify visually sensitive areas of images and intelligently adjust encoding parameters based on the importance of these areas, thereby achieving the best balance between distortion and encoding bit rate.

[0063] Determine a target coding strategy according to the rate-distortion optimization processing cost allowed for each macroblock, and process the current macroblock according to the target coding strategy to generate target data;

[0064] Wherein, the target coding strategy includes:

[0065] If the allowed rate-distortion optimization processing cost of the current macroblock is greater than a preset threshold, the current macroblock is identified as a first encoding strategy; otherwise, the current macroblock is identified as a second encoding strategy, wherein the encoding quality of the first encoding strategy is higher than the encoding quality of the second encoding strategy.

[0066] Dynamically adjusting the degree of coding distortion according to the visual sensitivity value of the macroblock can allocate coding resources more flexibly. For important visual areas, higher coding quality is adopted, while for less important areas, more efficient coding methods are adopted, which significantly improves coding efficiency and reduces unnecessary coding overhead; at the same time, it can further reduce the coding bit rate while maintaining a certain image quality. This balance makes the encoded image more in line with the user's visual needs. The final encoding result can better reflect the user's visual perception characteristics, making the encoded image more visually natural and clear.

[0067] The way of coding macroblocks can be achieved at least through compression technology, transmission optimization and visual perception. By performing differential coding, depth information compression or compression methods based on stereoscopic visual characteristics on stereoscopic video images, the amount of data can be reduced and the image quality can be maintained. At the same time, the bit rate adaptive technology can dynamically adjust the transmission bit rate according to the network bandwidth to ensure smooth transmission under different network conditions.

[0068] The data encoding method provided in this embodiment combines the visual characteristics of the human eye to perform adaptive macroblock encoding, achieving the goal of improving video quality while ensuring the size of the bitstream. The core of the algorithm provided in this application is not to simply use the encoding strategy to process each macroblock, but to dynamically adjust the size of the encoding distortion rate optimization cost according to the visual sensitivity value of the image content, so that the characteristics of the human visual system can be more effectively utilized to reduce the encoding time and bitstream size while maintaining or improving the visual quality.

[0069] In an optional embodiment, in the encoding method provided by the present application, traversing the plurality of macroblocks in step 202 and determining the visual sensitivity value of each macroblock one by one includes:

[0070] Calculating brightness information of a pixel at a position in a current macroblock, and acquiring a first visual sensitivity value of the brightness information according to the brightness information;

[0071] Calculating texture information of a pixel position in a current macroblock, and acquiring a second visual sensitivity value of the brightness information according to the texture information;

[0072] The visual sensitivity value of the current pixel is obtained according to the first visual sensitivity value and the second visual sensitivity value.

[0073] Each macroblock includes multiple pixels, and the visual sensitivity value of each pixel can be calculated one by one, and then the visual sensitivity value of the macroblock can be obtained after averaging processing.

[0074] The sensitivity of the human visual system to visual information is not evenly distributed, but depends on multiple factors, including: brightness adaptation, brightness adaptation, texture masking effect, motion masking and visual attention. When judging the perception of the current frame image, multiple factors need to be considered comprehensively.

[0075] When calculating the visual sensitivity value of the current macroblock, you can refer to the following calculation formula:

[0076] This paper uses JND (just noticeable difference) as a measure of the sensitivity of HVS to distortion. The JND value of a single pixel is calculated as follows.

[0077] P(i,j)=T L (i,j)+T t(i,j)

[0078] Where i and j represent the coordinate position information of the pixel in the current macroblock, P(i, j) is the JND value of a pixel in an image; L(i, j) is used to indicate the brightness information of the current macroblock; T t (i, j) is used to indicate the texture information of the current macroblock

[0079] In an optional embodiment, the steps of calculating the visual sensitivity value of the current macroblock in the encoding method provided by the present application include:

[0080] The visual sensitivity value of the current macroblock is calculated according to the first calculation formula and the first factor (brightness value) of the pixels in the macroblock, wherein the first calculation formula includes:

[0081] In the first brightness interval, the first factor of the pixel is in increasing relationship with the visual sensitivity value;

[0082] In the second brightness range, the first factor of the pixel is in a decreasing relationship with the visual sensitivity value;

[0083] In the area where the first brightness interval and the second brightness interval overlap, the value of the first factor of the pixel, the visual sensitivity value, is the largest among all brightness intervals.

[0084] Here is a specific example, the first calculation formula includes:

[0085]

[0086] The functional relationship between the background brightness L(i, j) and is described as follows: in the low brightness area L(i, j) < 127, the sensitivity of the human eye and the background brightness are in an increasing relationship; in the high brightness area L(i, j) ≥ 127, the sensitivity of the human eye and the background brightness are in a linear decreasing relationship; when L(i, j) = 127, the sensitivity of the human eye reaches its maximum value.

[0087] The embodiment of the present application can more accurately determine the visual sensitivity value of a pixel by calculating the brightness information of a macroblock, thereby improving the efficiency of subsequent encoding processing.

[0088] In an optional embodiment, the steps of calculating the visual sensitivity value of the current macroblock in the encoding method provided by the present application include:

[0089] According to the second calculation formula and the second factor (texture information) of the pixels in the macroblock, the visual sensitivity value of the current macroblock is calculated, wherein the second calculation formula includes:

[0090] Obtain the average gradient weight within a preset range of pixels and the strength value of the pixel edge;

[0091] The visual sensitivity value of the current macroblock is calculated based on the average gradient weight within a preset range of pixels and the strength value of the pixel edge.

[0092] Here is a specific example, the second calculation formula includes a texture shielding model:

[0093] T t (i,j)=β×g(i,j)×e(i,j)

[0094] Where β is the control parameter, g(i,j) represents the average weight of the gradient near the pixel, and e(i,j) is the strength value of the pixel edge. The model is as follows:

[0095]

[0096] H is a mxm Gaussian low-pass filter with standard deviation δ.

[0097] According to the above calculation results, it can be seen that the detail-rich area can hide more errors than the flat area, so the visual sensitivity value of the pixel in the detail-rich area is larger than the visual sensitivity value of the pixel in the flat area.

[0098] The embodiment of the present application can more accurately determine the visual sensitivity value of a pixel by calculating the texture information of a macroblock, thereby improving the efficiency of subsequent encoding processing.

[0099] In an optional embodiment, in the encoding method provided by the present application, the rate-distortion optimization processing cost allowed for each macroblock may be determined by an RD cost function, wherein the RD cost function includes:

[0100] J RD =D+λ·R

[0101] The distortion D is usually obtained by calculating the difference between the original image and the encoded image. The difference can be the difference between pixel values, the difference between blocks, or the difference between more complex image features. Common distortion calculation methods include mean square error (MSE), peak signal-to-noise ratio (PSNR), or sum of square differences (SSE).

[0102] λ is used to indicate an adjustment factor, which is an adjustable parameter. The adjustment factor is set according to network transmission resources and visual sensitivity values ​​to achieve a balance between distortion and encoding bit rate. In practical applications, the value of λ is usually determined according to factors such as encoding requirements, image content, and target encoding bit rate.

[0103] The coding rate R refers to the code rate value occupied by the encoded image data, which reflects the compression efficiency of the coding.

[0104] Each macroblock is encoded according to the adjusted λ value and the visual sensitivity value of each macroblock. During the encoding process, appropriate encoding parameters (such as quantization parameters, encoding modes, etc.) are determined to minimize the encoding bitstream while meeting the visual quality requirements.

[0105] The above RD cost function is a criterion for measuring the quality of coding schemes, which comprehensively considers the bit rate and distortion after coding. The RD cost function is usually calculated based on SSE (Sum of Squared Errors) and R (the bit rate after coding), and an adjustable parameter λ is introduced to balance the relationship between bit rate and distortion.

[0106] HVS-based rate-distortion optimization (RDO) is an optimization algorithm in video coding that takes into account the perceptual characteristics of HVS to optimize coding parameters. During the RDO process, the encoder calculates the RD cost function under different coding parameters and selects the parameter combination that minimizes the RD cost for encoding.

[0107] In order to strike a balance between distortion and encoding bitrate, the encoder needs to make intelligent adjustments based on the visually sensitive areas of the image. Visually sensitive areas usually refer to those parts of the image that have the greatest impact on the user's visual experience, such as faces, text, important objects, etc. These areas usually contain more details and textures, so users have higher visual requirements for them.

[0108] During the encoding process, the encoder analyzes the visually sensitive areas of the image and allocates encoding resources based on the importance of these areas. For visually sensitive areas, the encoder uses lower distortion and higher encoding bitrate to ensure the image quality of these areas. Although this will increase the overall amount of encoded data, it can significantly improve the user's visual experience.

[0109] On the contrary, for non-visually sensitive areas in the image, the encoder can use higher distortion and lower encoding bit rate to encode. These areas usually contain less details and textures, and users have relatively low visual requirements for them. Therefore, by reducing the image quality and the amount of encoded data in these areas, the encoding efficiency can be further improved without affecting the overall visual experience.

[0110] In an optional embodiment, determining a target adjustment factor for each macroblock in the encoding method provided by the present application includes:

[0111] Get the visual sensitivity value of each pixel in the current macroblock;

[0112] According to the visual sensitivity value of each pixel in the current macroblock, obtain the mean visual sensitivity value of the current macroblock;

[0113] According to the mean value of the visual sensitivity value of the current macroblock and the adjustment factor calculation formula, the target adjustment factor of the current macroblock is calculated, wherein the adjustment factor calculation formula includes:

[0114] λ=λ org C·d

[0115] Among them, λ is used to indicate the target adjustment factor of the current macroblock; λ org Used to indicate the initial quantization parameter;

[0116] d is used to indicate the visual sensitivity value of the current macroblock.

[0117] In an optional embodiment, the target adjustment factor in step 302 of the encoding method provided by the present application includes: Lagrangian factor λ:

[0118] According to the rate-distortion optimization processing cost allowed for each macroblock, adjust the value of the Lagrangian factor λ. The higher the λ value, the greater the RD cost allowed for encoding, that is, the coding distortion can be relatively large in exchange for a smaller coding bitstream. Conversely, the lower the λ value, the smaller the RD cost allowed for encoding, and more sophisticated coding parameters need to be selected to maintain image quality.

[0119] The λ parameter plays a crucial role in the RDO process. It balances the relationship between encoding quality and bitrate. When the user perception is higher, the λ value is higher, which means that the encoder pays more attention to maintaining visual quality during the optimization process, so it allows a larger RD cost, thereby selecting clearer encoding parameters and a larger encoding bitrate. On the contrary, when the user perception is lower, the λ value is lower, and the encoder pays more attention to reducing the bitrate, so it chooses a smaller RD cost and a more blurred encoding parameter.

[0120] Introducing the visual sensitivity value of the target user into image coding and performing rate-distortion optimization based on HVS can significantly improve video compression efficiency and subjective visual quality. By simulating the perceptual characteristics of HVS, the encoder can allocate the encoding bitrate more intelligently, thereby reducing the bitrate while maintaining subjective quality. In addition, by adjusting the λ parameter, the encoder can balance the relationship between encoding quality and bitrate based on user perception and bandwidth requirements.

[0121] The encoding method provided in the embodiment of the present application dynamically adjusts encoding parameters (such as quantization parameters, transform size, etc.) according to parameters such as the visual sensitivity value and size of the macroblock, so as to reduce the bitstream size and encoding time while maintaining the visual quality. Finally, a video file or data stream containing the encoded image data is generated.

[0122] The macroblocks in the cache area are encoded by HEVC (High Efficiency Video Coding), AVC (Advanced Video Coding) and other coding algorithms, and the quantization parameter QP and the coding mode are dynamically adjusted according to the size of the block. Finally, the encoded macroblocks are combined into a complete coded frame and output to the video transmission system. The experimental results show that the method of the present invention can significantly improve the visual quality of the reconstructed image while keeping the code stream size similar.

[0123] The encoded macroblocks are combined into complete encoded frames, and necessary post-processing (such as deblocking filtering, intra-frame prediction optimization, etc.) is performed, and finally the encoded video stream is output for transmission or storage.

[0124] The visual sensitivity value coding method provided in the embodiment of the present application realizes the refined processing of image content by dynamically adjusting the coding optimization strategy and coding parameters. The method can significantly improve the coding efficiency while ensuring the visual quality of the reconstructed image, and is particularly suitable for application scenarios where bandwidth resources are limited but high-quality video transmission is required.

[0125] See also Figure 3 The following embodiments take the above-mentioned terminal device or data decoding device as the execution subject, and apply the method provided in the embodiment of the present application to data decoding processing as an example for specific description. The data decoding method provided in the embodiment of the present application includes the following steps 301-303:

[0126] Step 301: Obtain target data;

[0127] The target data is the coded data after rate-distortion optimization processing according to the target adjustment factor, and the target adjustment factor is set according to the visual sensitivity value of each macroblock;

[0128] The target adjustment factor is set according to the visual sensitivity value of each macroblock, including:

[0129] If the visual sensitivity value of the current macroblock is greater than a preset threshold, the value of the target adjustment factor is increased according to the preset adjustment factor;

[0130] If the visual sensitivity value of the current macroblock is less than or equal to the threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

[0131] As described in the above steps 201 to 205, both the first coded data and the second coded data are from the target frame data, which will not be described in detail here.

[0132] Step 302: Decode and process the target data to obtain a target frame image.

[0133] By using the rate-distortion optimization algorithm, the optimal sampling rate and its corresponding downsampling mode and quantization parameters are selected;

[0134] According to the obtained optimal sampling mode and quantization parameters, the target data is decoded and processed to obtain the target frame image.

[0135] It is also possible to: perform super-resolution reconstruction on small blocks of the decoded image by using multiple models trained using super-resolution reconstruction technology based on deep learning; and then combine the reconstructed image blocks in a corresponding manner to form a final decoded image.

[0136] The decoding method provided in the embodiment of the present application realizes the refined processing of the image content by obtaining different encoding data of different types of macroblocks in the same frame image. The encoding data of the macroblock is generated by dynamically adjusting the encoding optimization and encoding parameters according to the visual sensitivity value of the macroblock. The method can significantly improve the decoding efficiency while ensuring the visual quality of the reconstructed image, and is particularly suitable for application scenarios with limited bandwidth resources but requiring high-quality video transmission.

[0137] An embodiment of the present disclosure provides a server, comprising: a processor and a memory storing a computer program that can be run on the processor, wherein the processor implements the steps of the data encoding method in the above embodiment when executing the computer program.

[0138] For the specific definition of the above-mentioned server, please refer to the definition of the data encoding method above, which will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0139] In an embodiment of the present disclosure, a terminal device is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above data decoding method can be implemented.

[0140] For the specific definition of the above-mentioned terminal device, please refer to the definition of the data encoding method above, which will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0142] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0143] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0144] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data encoding method, characterized in that: The method comprises: Acquire a target frame image, where the target frame image includes M*N macroblocks; Traversing the M*N macroblocks, and determining the visual sensitivity value of each macroblock one by one; According to the visual sensitivity value of each macroblock and the preset adjustment factor, a target adjustment factor of the rate-distortion optimization processing of each macroblock is set to minimize the cost of the rate-distortion optimization processing; According to the target adjustment factor of each macroblock, the rate-distortion optimization processing cost during the encoding process of each macroblock is calculated to obtain the optimal encoding mode of the current macroblock. The step of setting a target adjustment factor for rate-distortion optimization processing of each macroblock includes: If the visual sensitivity value of the current macroblock is greater than a preset threshold, the value of the target adjustment factor is increased according to the preset adjustment factor; If the visual sensitivity value of the current macroblock is less than or equal to the preset threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

2. The data encoding method according to claim 1, characterized in that: The calculating of the rate-distortion optimization processing cost during the encoding process of each macroblock includes: The RD cost function is used to calculate the cost of rate-distortion optimization during the encoding process. Wherein, the RD cost function includes: J RD =D+λ·R Among them, D is the difference between the original macroblock and the encoded macroblock, R is the bit rate value of the current macroblock after encoding, and λ is used to indicate the adjustment factor, which is set according to network transmission resources and visual sensitivity values.

3. The data encoding method according to claim 2, characterized in that: The step of setting a target adjustment factor for rate-distortion optimization processing of each macroblock includes: Get the visual sensitivity value of each pixel in the current macroblock; According to the visual sensitivity value of each pixel in the current macroblock, obtain the mean visual sensitivity value of the current macroblock; Calculate the target adjustment factor of the current macroblock according to the mean value of the visual sensitivity value of the current macroblock and the adjustment factor calculation formula, The adjustment factor calculation formula includes: λ=λ org CD Among them, λ is used to indicate the target adjustment factor of the current macroblock; λ org Used to indicate the initial quantization parameter; d is used to indicate the visual sensitivity value of the current macroblock.

4. The data encoding method according to claim 1, characterized in that: The determining of the visual sensitivity value of each macroblock comprises: Obtain the brightness information and texture information of each pixel in each macroblock; Processing the brightness information and texture information of each pixel in each macroblock through a target calculation formula to calculate the visual sensitivity value of each macroblock; Wherein, the target calculation formula includes: P(i,j)=T L (i,j)+T t (i,j) Among them, P(i,j) is used to indicate the visual sensitivity value of the current pixel; T L (i, j) is used to indicate the brightness information of the current pixel position. T t (i, j) is used to indicate the texture information of the current pixel location.

5. The data encoding method according to claim 4, characterized in that: Calculate the visual sensitivity value of each macroblock, including: The visual sensitivity value of each macroblock is calculated according to the first calculation formula and the brightness information of the pixel position in the macroblock, wherein the first calculation formula is used to indicate: In the first brightness interval, the brightness value of the pixel position is in an increasing relationship with the visual sensitivity value; In the second brightness interval, the brightness value of the pixel position is in a decreasing relationship with the visual sensitivity value; In the area where the first brightness interval and the second brightness interval overlap, the brightness value at the pixel location is the largest among all brightness intervals of the visual sensitivity value.

6. The data encoding method according to claim 4, characterized in that: Calculate the visual sensitivity value of each macroblock, including: The texture information of the pixel position in the macroblock is processed according to the second calculation formula to calculate the visual sensitivity value of each macroblock, wherein the second calculation formula includes: T t (i,j)=β×g(i,j)×e(i,j) Where β is used to indicate the preset control parameter; g(i,j) is used to indicate the average gradient weight near the current pixel; e(i,j) is used to indicate the strength of the pixel edge.

7. The data encoding method according to claim 1, characterized in that: The method further comprises: If the allowed rate-distortion optimization processing cost of the current macroblock is greater than a preset threshold, the current macroblock is identified as a first encoding strategy; otherwise, the current macroblock is identified as a second encoding strategy, wherein the encoding quality of the first encoding strategy is higher than the encoding quality of the second encoding strategy.

8. A data decoding method, characterized in that: The method comprises: Acquire target data, where the target data is coded data after rate-distortion optimization processing is performed according to a target adjustment factor, where the target adjustment factor is set according to a visual sensitivity value of each macroblock; Decoding and processing the target data to obtain a target frame image; The target adjustment factor is set according to the visual sensitivity value of each macroblock, including: If the visual sensitivity value of the current macroblock is greater than a preset threshold, the value of the target adjustment factor is increased according to the preset adjustment factor; If the visual sensitivity value of the current macroblock is less than or equal to the preset threshold, the value of the target adjustment factor is reduced according to the preset adjustment factor.

9. A server, characterized in that: The server comprises: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data encoding method according to any one of claims 1 to 7 when executing the computer program.

10. A terminal device, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the data decoding method described in claim 8.

Images