Data encoding method, data decoding method, server and terminal equipment

Abstract

The invention provides a data encoding method, a data decoding method, terminal equipment and a server, relates to the technical field of electronic information, and can solve the problem of low image encoding processing efficiency. According to the specific technical scheme, the expansion processing strategy of the macro block is dynamically determined according to the visual sensitive value of the image content, so that the coding time length and the code stream size are reduced and the transmission speed is improved while the visual quality is maintained or improved by more effectively utilizing the characteristics of a human visual system.

Description

Technical Field

[0001] The present disclosure relates to the field of electronic information technology, and in particular to data encoding, decoding methods, servers and terminal devices. Background Art

[0002] With the rapid development of image processing technology, the presentation of images has become more diverse. In the existing technology, the macroblock division method uses fixed-size blocks for encoding, ignoring the differences in visual importance of image content. This method cannot effectively utilize the characteristics of the human visual system to optimize the encoding process. Therefore, a method that can adaptively perform macroblock division and encoding based on visual sensitivity values ​​is needed.

[0003] In the process of video coding, how to balance coding efficiency and reconstructed image quality has always been a hot research topic. Summary of the invention

[0004] The disclosed embodiment provides a data encoding and decoding method, a server and a terminal device, which can solve the problem of large data volume in the existing data encoding process. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a data encoding method is provided, the method comprising:

[0006] Acquire a target frame image, and divide the target frame image to generate M*N initial macroblocks;

[0007] Traversing the M*N initial macroblocks, and determining the visual sensitivity value of each initial macroblock one by one;

[0008] If the visual sensitivity value of the initial macroblock is greater than a preset threshold matched by the initial macroblock, directly encoding the initial macroblock to obtain first encoded data;

[0009] If the visual sensitivity value of the initial macroblock is less than or equal to the preset threshold matched by the initial macroblock, the initial macroblock is extended and processed according to the preset macroblock extension method to obtain an extended macroblock, until the visual sensitivity mean value of the extended macroblock is greater than the preset threshold matched by the extended macroblock, the extension is stopped, and the extended macroblock is encoded to obtain second encoded data;

[0010] The first coded data and the second coded data are merged to generate target data, and the target data is sent to the terminal device.

[0011] According to a second aspect of an embodiment of the present disclosure, a data decoding method is provided, the method comprising:

[0012] Acquire target data, where the target data includes at least first coded data and second coded data;

[0013] Decoding and processing the target data to obtain an extended macroblock and an initial macroblock;

[0014] The extended macroblock and the initial macroblock are fused to obtain the target frame image;

[0015] The first coded data refers to the coded data generated by the initial macroblock in the target frame image;

[0016] The second coded data refers to a coded macroblock generated after the initial macroblock in the target frame image is extended, wherein the extended macroblock processing refers to: a macroblock obtained by merging the initial macroblock with an adjacent macroblock according to the visual sensitivity value of the current macroblock;

[0017] The encoding quality of the first encoded data is higher than the encoding quality of the second encoded data.

[0018] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data encoding method as described in the above example when executing the computer program.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a terminal device is provided, comprising: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data decoding method as described in the above example when executing the computer program.

[0020] The encoding and decoding method provided in the embodiment of the present application can determine the human eye perception degree matched by each macroblock in different frame images, and perform regional processing according to the human eye perception of different macroblocks. If the human eye perception of the initial macroblock is high, the regional slices are smaller and more detailed encoding is performed. If the human eye perception of the initial macroblock is low, the regional slices are larger and the compression rate is also higher. Different macroblocks have different encoding strategies, so that more effective information can be retained without affecting the size of the code stream, so that the visual effect of the image reconstructed at the receiving end is better.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1A is a framework diagram 1 of a data processing system provided by an embodiment of the present disclosure;

[0024] Figure 1BIt is a framework of a data processing system provided by the embodiment of the present disclosure Figure 2 ;

[0025] Figure 2 is a flow chart of a data encoding method provided by an embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram of a threshold value in a data encoding method provided by an embodiment of the present disclosure;

[0027] Figure 4 It is a processing logic diagram in a data encoding method provided by an embodiment of the present disclosure;

[0028] Figure 5 It is a flowchart of a data decoding method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements.

[0030] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0031] Existing coding methods usually use fixed-size macroblocks for coding, ignoring the differences in visual importance of image content. The human eye has different visual sensitivity values ​​for different areas. It is more sensitive to distortion in high-sensitivity areas (such as edges and areas with rich textures), and has a higher tolerance for distortion in low-sensitivity areas (such as smooth areas). Therefore, adaptively adjusting the macroblock size and coding parameters according to the visual sensitivity value can more effectively utilize coding resources and improve coding efficiency and visual effects.

[0032] With the advancement of technology, the terminal side will develop in an increasingly lightweight direction, that is, the configuration and structure of the terminal will become simpler and the size will become smaller.

[0033] At present, various zero-terminal solutions have been proposed. The so-called zero-terminal means that the terminal only needs to have a screen, a simple CPU and memory, an image receiving and decoding module for processing images, and a reverse control processing module. This solution can greatly simplify the size and configuration of the mobile terminal. All programs of the zero-terminal run in the cloud server. The zero-terminal uses the corresponding application by accessing the cloud server and controls the corresponding application through reverse control. The server will send the real-time processing screen to the zero-terminal. In this way, for the zero-terminal, it is just like operating locally.

[0034] In the prior art, when using a zero terminal to display some videos, such as playing movies, animations or games, the server encodes the image and sends it to the zero terminal through the network for decoding and display. The existing video encoding scheme effectively removes spatial / temporal redundancy through intra-frame and inter-frame prediction methods, but does not consider the visual redundancy of the video. In real scenes, the human visual system (Human Visual System) HVS can perceive visual information such as brightness, color, texture, direction, spatial frequency, etc. in the video scene. It is selective in sensitivity to different scenes, and different areas or objects have different visual importance. Therefore, it is of great significance to introduce HVS into video encoding. It can compress videos more efficiently and achieve better visual perception effects.

[0035] In order to solve the transmission pressure problem of video between the cloud and the local end, the present application provides a data encoding method, which is based on an adaptive macroblock division and encoding method of visual sensitivity values. It aims to achieve refined encoding of video content by dynamically adjusting the macroblock size and encoding parameters, thereby improving the visual quality of the reconstructed image while maintaining efficient encoding.

[0036] The method provided in the embodiment of the present application is based on the human visual system (Human Visual System), determines the human eye perception degree matched by each macroblock in different frame images, and performs regional processing according to the human eye perception degree of different macroblocks. If the human eye perception degree of the initial macroblock is high, the regional fragmentation is smaller and more detailed encoding is performed. If the human eye perception degree of the initial macroblock is low, the regional fragmentation is larger and the compression rate is also higher. Different macroblocks have different encoding strategies, so that more effective information can be retained without affecting the size of the code stream, so that the visual effect of the image reconstructed at the receiving end is better.

[0037] In the process of encoding and processing video data, the embodiments of the present application can be implemented through compression technology, transmission optimization, and visual perception algorithms. By performing difference encoding, depth information compression, or compression methods based on stereoscopic visual characteristics on stereoscopic video images, the amount of data can be reduced and the image quality can be maintained. At the same time, the bit rate adaptive technology can be used to dynamically adjust the transmission bit rate according to the network bandwidth to ensure smooth transmission under different network conditions.

[0038] The following is a brief description of the application environment of the data encoding and decoding method provided in the embodiment of the present application:

[0039] See also Figure 1AThe system shown in the figure, a data encoding and decoding method provided in the embodiment of the present application is applied to a data processing system 10, the system 10 includes: a server 101 and a terminal device 102; the server 101 and the terminal device 102 can be connected by wire or wirelessly;

[0040] Server 101, which has an image encoding function, and the encoding part in this solution is implemented here;

[0041] The terminal device 102 has an image decoding function and can also be used for the interface and display of images. The decoding part of this solution is implemented here. The terminal device 101 can be a receiving device such as a computer, a mobile phone or a PAD.

[0042] Please refer to Figure 1B The data stream encoding and decoding process between the server 101 and the terminal device 102 is shown as follows:

[0043] The encoding end (server 101) collects video frame data through the image acquisition module, and then pre-processes the frame data through the encoding module. The visual sensitivity value detection and block marking module identifies the visual sensitivity value of each macroblock, and marks the blocks, and transmits them to the encoding module. The encoding module encodes and processes according to different macroblocks, generates an encoded code stream, and sends the encoded code stream to the terminal device.

[0044] The decoding end (terminal device 102) obtains the bitstream data sent by the encoding end through the bitstream receiving module, and caches the video bitstream pushed by the encoding end as the data source for decoding; the bitstream cached by the decoding processing module is decoded by the decoding module, and finally the image frame decoded by the module is rendered and displayed.

[0045] See also Figure 2 The following embodiments take the above-mentioned server or data encoding device as the execution subject, and apply the method provided in the embodiment of the present application to data encoding processing as an example for specific description. The data encoding method provided in the embodiment of the present application includes the following steps 201-203:

[0046] Step 201: Acquire a target frame image, and divide the target frame image into M*N initial macroblocks.

[0047] The target frame image includes several macroblocks. For example, the entire frame image can be divided into several initial macroblocks of equal size (such as 16x16, 32x32, etc.) according to a preset size. The size of the macroblock can be adjusted according to the specific application scenario and encoding requirements.

[0048] The server receives a video frame image to be encoded, where the video frame image includes several frame images. The target frame image may be a frame in a target video, and the target video may include data in a 2D format or data in a 3D format.

[0049] Step 202: traverse the M*N macroblocks and determine the visual sensitivity value of each initial macroblock one by one.

[0050] The visual sensitivity value of each initial macroblock is determined one by one according to the visual sensitivity detection algorithm of the target user. The algorithm can be based on a variety of visual characteristics, such as contrast sensitivity, brightness adaptability, texture masking effect, etc. The output of the algorithm is a numerical value that measures the visual sensitivity value of the macroblock.

[0051] Step 203: If the visual sensitivity value of the initial macroblock is greater than the preset threshold matched by the initial macroblock, the initial macroblock is directly encoded to obtain first encoded data.

[0052] If the visual sensitivity value of the initial macroblock is less than or equal to the preset threshold, the target macroblocks adjacent to the initial macroblock are merged along the preset direction, and the visual sensitivity value of the merged macroblock is determined until the visual sensitivity value of the initial macroblock is greater than the preset threshold.

[0053] Different visual elements (such as color, brightness, texture, motion, etc.) are quantized to different degrees according to the visual sensitivity values ​​of different macroblocks. A coarser quantization step size is used in areas with lower visual importance, while a finer quantization step size is used in important areas to reduce the impact of quantization noise on visual quality.

[0054] If the visual sensitivity value of the initial macroblock is less than or equal to the threshold, the region is considered to have low visual importance to the human eye, and the macroblock is further merged into N larger macroblocks, and the sensitivity detection of the newly generated macroblocks continues until the sensitivity of all blocks exceeds the threshold.

[0055] Step 204: If the visual sensitivity value of the initial macroblock is less than or equal to the preset threshold matched by the initial macroblock, the initial macroblock is extended and processed according to the preset macroblock extension method to obtain an extended macroblock. When the visual sensitivity mean of the extended macroblock is greater than or equal to the preset threshold matched by the extended macroblock, the extension is stopped, and the extended macroblock is encoded to obtain second encoded data.

[0056] After all macroblocks have been subjected to the above-mentioned visual sensitivity value detection and merging processing, macroblocks of different sizes are stored in the cache area, wherein the high-sensitivity area blocks are smaller and the low-sensitivity area blocks are larger.

[0057] Step 205: Fusion process the first coded data and the second coded data to generate target data, and send the target data to the terminal device.

[0058] The first coded data and the second coded data are integrated into target data through a specific data fusion algorithm (such as bit stream merging, data packaging, etc.). Then, the target data is encapsulated into a data packet using a data transmission protocol (such as RTP, HTTP, etc.) and sent to the terminal device.

[0059] The integrity and consistency of the encoded data are ensured through fusion processing, which is convenient for decoding and reconstruction of terminal devices. At the same time, by selecting the appropriate transmission protocol, efficient and reliable data transmission can be ensured, improving user experience.

[0060] For each initial macroblock, its visual sensitivity value is compared with a preset threshold. If the visual sensitivity value is greater than the threshold, an efficient coding algorithm (such as CABAC, DCT transform, etc.) is directly applied to encode the macroblock to generate first coded data. If the visual sensitivity value is less than or equal to the threshold, the macroblock is expanded according to a preset macroblock expansion method (such as gradually increasing the macroblock size until the adjacent low-sensitivity macroblock is covered) until the visual sensitivity mean of the expanded macroblock reaches or exceeds the threshold, and then the expanded macroblock is encoded to generate second coded data.

[0061] Through the above-mentioned conditional judgment, the encoding strategy can be flexibly adjusted. For visually important areas (such as faces, text, etc.), a smaller macroblock size is used for encoding to more accurately capture details; while for background or visually unimportant areas, a larger macroblock size is used to improve compression efficiency, thereby reducing encoding complexity and bit rate while maintaining visual quality.

[0062] The way of encoding and processing image macroblocks can be achieved at least through compression technology, transmission optimization and visual perception. By performing difference encoding, depth information compression or compression methods based on stereoscopic visual characteristics on stereoscopic video images, the amount of data can be reduced and the image quality can be maintained. At the same time, the bit rate adaptive technology can dynamically adjust the transmission bit rate according to the network bandwidth to ensure smooth transmission under different network conditions.

[0063] The data encoding method provided in this embodiment combines the visual characteristics of the human eye to perform adaptive macroblock expansion processing and classification encoding processing, thereby achieving the goal of improving video quality while ensuring the size of the bitstream. The core of the algorithm provided in this application is not to simply use fixed-size blocks for encoding, but to dynamically adjust the size of the block according to the visual sensitivity value of the image content, so that the characteristics of the human visual system can be more effectively utilized to reduce the encoding time and bitstream size while maintaining or improving the visual quality.

[0064] In an optional embodiment, in the encoding method provided by the present application, step 202, traversing the plurality of macroblocks and determining the visual sensitivity value of each initial macroblock one by one, includes:

[0065] Step 301, obtaining brightness information and texture information of each pixel in the initial macroblock;

[0066] Step 302: Calculate the visual sensitivity value of the initial macroblock according to the brightness information and texture information of each pixel in the initial macroblock and the target calculation formula;

[0067] Each macroblock includes multiple pixels, and the visual sensitivity value of each pixel can be calculated one by one, and then the visual sensitivity value of the macroblock can be obtained after averaging processing.

[0068] The sensitivity of the human visual system to visual information is not evenly distributed, but depends on multiple factors, including: brightness adaptation, brightness adaptation, texture masking effect, motion masking and visual attention. When judging the perception of the current frame image, multiple factors need to be considered comprehensively.

[0069] This paper uses JND (just noticeable difference) as a measure of the sensitivity of HVS to distortion. The JND value of a single pixel is calculated as follows.

[0070] P(i,j)=T L (i,j)+T t (i,j)

[0071] Where i and j represent the coordinate position information of the pixel in the initial macroblock, P(i, j) is the JND value of a pixel in an image; L(i, j) is used to indicate the brightness information of the initial macroblock; T t (i, j) is used to indicate the texture information of the initial macroblock

[0072] In an optional embodiment, the step 302 of calculating the visual sensitivity value of the initial macroblock in the encoding method provided by the present application includes:

[0073] The visual sensitivity value of the initial macroblock is calculated according to the first calculation formula and the first factor of the pixel in the macroblock (the brightness value of the pixel position), wherein the first calculation formula includes:

[0074] In the first brightness interval, the first factor of the pixel is in increasing relationship with the visual sensitivity value;

[0075] In the second brightness range, the first factor of the pixel is in a decreasing relationship with the visual sensitivity value;

[0076] In the area where the first brightness interval and the second brightness interval overlap, the value of the first factor of the pixel, the visual sensitivity value, is the largest among all brightness intervals.

[0077] Here is a specific example, the first calculation formula includes:

[0078]

[0079] The functional relationship between the background brightness L(i, j) and is described as follows: in the low brightness area L(i, j) < 127, the sensitivity of the human eye and the background brightness are in an increasing relationship; in the high brightness area L(i, j) ≥ 127, the sensitivity of the human eye and the background brightness are in a linear decreasing relationship; when L(i, j) = 127, the sensitivity of the human eye reaches its maximum value.

[0080] The embodiment of the present application can more accurately determine the visual sensitivity value of a pixel by calculating the brightness information of a macroblock, thereby improving the efficiency of subsequent encoding processing.

[0081] In an optional embodiment, the step 302 of calculating the visual sensitivity value of the initial macroblock in the encoding method provided by the present application includes:

[0082] The visual sensitivity value of the initial macroblock is calculated according to the second calculation formula and the texture information of the pixels in the macroblock, wherein the second calculation formula includes:

[0083] Obtain the average gradient weight within a preset range of pixels and the strength value of edge pixels;

[0084] The visual sensitivity value of the initial macroblock is calculated based on the average gradient weight within a preset range of pixels and the strength value of edge pixels.

[0085] Here is a specific example, the second calculation formula includes a texture shielding model:

[0086] T t (i,j)=β×g(i,j)×e(i,j)

[0087] Where β is the control parameter, g(i,j) represents the average gradient weight near the pixel, and e(i,j) is the strength value of the edge pixel. The model is as follows:

[0088]

[0089] H is a mxm Gaussian low-pass filter with standard deviation δ.

[0090] According to the above calculation results, it can be seen that the detail-rich area can hide more errors than the flat area, so the visual sensitivity value of the pixel in the detail-rich area is larger than the visual sensitivity value of the pixel in the flat area.

[0091] The embodiment of the present application can more accurately determine the visual sensitivity value of a pixel by calculating the texture information of a macroblock, thereby improving the efficiency of subsequent encoding processing.

[0092] In an optional embodiment, in the encoding method provided by the present application, merging adjacent macroblocks along a preset direction in step 203 includes:

[0093] Step 401: If the visual sensitivity value of the initial macroblock is less than a preset threshold matched by the initial macroblock, search for a target macroblock adjacent to the initial macroblock in a first direction according to a preset direction;

[0094] Step 402: merge the target macroblock and the initial macroblock to generate a merged macroblock, and determine the visual sensitivity mean of the merged macroblock;

[0095] Step 403: determine whether the visual sensitivity mean of the merged macroblock is greater than or equal to a threshold corresponding to the size of the merged macroblock;

[0096] Step 404: If the result is yes, stop the expansion process and output the current merged macroblock as an extended macroblock;

[0097] If the result is no, then continue with the expansion process.

[0098] For example: first divide a frame of image into non-overlapping 4x4 macroblocks, starting from the first macroblock (upper left corner), calculate the JND mean of the macroblock in raster scanning order. If the initial macroblock mean is greater than T1, it is marked as a 4x4 block. If it is less than T1, the initial macroblock is expanded to the surrounding area. The expansion order is along the right side of the initial macroblock. After judging the right macroblock, the macroblock below the initial macroblock is judged.

[0099] The encoding method provided in the embodiment of the present application calculates the sensitivity of each initial block based on a visual sensitivity value detection algorithm based on texture complexity and brightness contrast, and compares it with a preset threshold (such as 50%): for macroblocks with sensitivity lower than the threshold, they are further merged into small blocks of 8x8, and the visual sensitivity value detection process is repeated until the sensitivity of all blocks exceeds the threshold.

[0100] In an optional embodiment, in 2 of the method mentioned in the present application, the preset threshold matched by the initial macroblock or the extended macroblock is determined at least according to a target mapping, wherein the target mapping is used to indicate a relationship between a physical parameter of at least one macroblock and a preset visual sensitivity value of the macroblock;

[0101] If the size of the extended macroblock is larger than the size of the initial macroblock, the preset threshold of the extended macroblock is smaller than the threshold of the initial macroblock.

[0102] Please refer to the attached Figure 3 As shown in the mapping relationship between the two, the sizes of the blocks divided according to the visual sensitivity value algorithm model are 11 sizes in the table below, and then based on a large number of experiments, 10 thresholds T1...T10 of the mean visual sensitivity values ​​corresponding to the above macroblock sizes are obtained, and the threshold values ​​of T1 to T10 are sorted from large to small.

[0103] Therefore, the preset thresholds matched by macroblocks with different physical parameters (macroblock sizes) are different, which are further explained here:

[0104] Obtaining parameters of the initial macroblock image;

[0105] Finding a first preset threshold value matching the size of the initial macroblock according to the parameters of the initial macroblock image;

[0106] Obtaining the visual sensitivity value of the initial macroblock image;

[0107] If the visual sensitivity value of the initial macroblock is greater than a first preset threshold, the expansion process is stopped;

[0108] If the visual sensitivity value of the initial macroblock is less than or equal to a first preset threshold, merging adjacent macroblocks along a preset direction to obtain a first extended macroblock;

[0109] Obtaining a macroblock size of the first extended macroblock, and finding a second preset threshold matching the size of the first extended macroblock;

[0110] Calculating a visual sensitivity value of a first extended macroblock;

[0111] If the visual sensitivity value of the first extended macroblock is greater than the second preset threshold, the extension process is stopped; otherwise, the above process is repeated.

[0112] For example: the initial macroblock is first expanded to the right, and the JND value of the right macroblock is calculated. If it is greater than T1, the right macroblock is marked as a 4x4 macroblock. If it is less than T1, it is merged with the left macroblock into an 8x4 macroblock and then the merged JND average is calculated and compared with T2. If it is greater than T2, it is marked as a 4x8 macroblock. If it is less than T2, it continues to expand downward, and so on.

[0113] The encoding method provided in the embodiment of the present application can more accurately determine the visual sensitivity value of the initial macroblock by matching different preset thresholds for macroblocks of different sizes after merging, thereby facilitating subsequent improvement of encoding efficiency.

[0114] In an optional embodiment, the step of generating target data in the encoding method provided by the present application includes:

[0115] The encoding quality of the first encoded data is higher than the encoding quality of the second encoded data.

[0116] The encoding method provided in the embodiment of the present application dynamically adjusts encoding parameters (such as quantization parameters, transform size, etc.) according to parameters such as the visual sensitivity value and size of the macroblock, so as to reduce the bitstream size and encoding time while maintaining the visual quality. Finally, a video file or data stream containing the encoded image data is generated.

[0117] The macroblocks in the cache area are encoded by HEVC (High Efficiency Video Coding), AVC (Advanced Video Coding) and other coding algorithms, and the quantization parameter QP and the coding mode are dynamically adjusted according to the size of the block. Finally, the encoded macroblocks are combined into a complete coded frame and output to the video transmission system. The experimental results show that the method of the present invention can significantly improve the visual quality of the reconstructed image while keeping the code stream size similar.

[0118] Please refer to the attached Figure 4 The encoding flow chart shown further illustrates how the encoding method provided by the present application processes video frame images:

[0119] Step 1: Initial division of macroblocks:

[0120] An image frame to be encoded is received and preliminarily divided into a number of initial macroblocks of equal size; the size of the initial block can be set according to actual needs, such as 4*4, 8*8, 16*16, 32*32, etc.

[0121] Step 2: Visual sensitivity value detection:

[0122] For each initial macroblock, a preset visual sensitivity value detection algorithm is used to calculate its visual sensitivity value. The evaluation of the visual sensitivity value can be based on multiple features of the image, such as contrast, brightness, texture complexity, etc., to simulate the perception differences of the human visual system to different image contents; the calculated visual sensitivity value is compared with the preset threshold:

[0123] Step 3: Adaptive block division:

[0124] According to the evaluation results of the visual sensitivity value, the image blocks are dynamically adjusted. For blocks with lower visual sensitivity values, that is, areas that are visually smoother or have lower information content, the size of the blocks is increased by merging adjacent blocks to reduce the complexity of encoding and improve encoding efficiency; while for blocks with higher visual sensitivity values, that is, areas containing rich details or key information, they are kept independent or further subdivided to ensure that the encoded image has higher visual quality in these areas.

[0125] If the sensitivity is less than or equal to the threshold, the region is considered to have low visual importance and needs further refinement. The macroblock is further merged into N larger macroblocks (N is greater than the size of the initial block), and the sensitivity test is repeated for each newly generated macroblock until the sensitivity of all macroblocks exceeds the preset threshold.

[0126] If the sensitivity is greater than the threshold, the region is considered to have high visual importance, and further merging processing of the macroblock is stopped, and it is directly stored in the cache area in preparation for subsequent encoding processing.

[0127] Step 4: Cache and Encoding:

[0128] The image blocks after adaptive block division are stored in a cache area, which will contain macroblocks of different sizes, wherein the high-sensitivity area blocks are smaller and the low-sensitivity area blocks are larger.

[0129] For smaller macroblocks (high-sensitivity areas), more sophisticated coding parameters are used, such as higher quantization parameters QP, more coding modes, and more sophisticated motion estimation, to ensure the coding quality of these areas and reduce visual distortion.

[0130] For larger macroblocks (low-sensitivity areas), a higher compression rate is used to reduce the code stream size and improve coding efficiency by reducing coding information, using coarser quantization, skipping some coding steps, etc.

[0131] The encoded macroblocks are combined into complete encoded frames, and necessary post-processing (such as deblocking filtering, intra-frame prediction optimization, etc.) is performed, and finally the encoded video stream is output for transmission or storage.

[0132] The coded macroblocks are combined into complete coded frames and output for transmission or storage.

[0133] The visual sensitivity value coding method of the present invention realizes the refined processing of image content by dynamically adjusting the macroblock size and coding parameters. The method can significantly improve the coding efficiency while ensuring the visual quality of the reconstructed image, and is particularly suitable for application scenarios where bandwidth resources are limited but high-quality video transmission is required.

[0134] See also Figure 5 The following embodiments take the above-mentioned terminal device or data decoding device as the execution subject, and apply the method provided in the embodiment of the present application to data decoding processing as an example for specific description. The data decoding method provided in the embodiment of the present application includes the following steps 501-503:

[0135] Step 501: Obtain target data;

[0136] The first coded data refers to the coded data generated by the initial macroblock in the target frame image;

[0137] The second coded data refers to a coded macroblock generated after the initial macroblock in the target frame image is extended, wherein the extended macroblock processing refers to: an extended macroblock after the initial macroblock is merged with an adjacent macroblock according to the visual sensitivity value of the current macroblock;

[0138] The encoding quality of the first encoded data is higher than the encoding quality of the second encoded data.

[0139] As described in the above steps 201 to 205, both the first coded data and the second coded data are from the target frame data, which will not be described in detail here.

[0140] Step 502: Decode and process the target data to obtain an extended macroblock and an initial macroblock.

[0141] The instructions of the decoding process algorithm correspond to the encoding process quality, and the encoding quality of the first decoded data is higher than the encoding quality of the second decoded data.

[0142] Step 503: The extended macroblock and the initial macroblock are fused to obtain a target frame image.

[0143] According to the position information of different macroblocks, the extended macroblock and the initial macroblock are fused to obtain the target frame image.

[0144] The visual sensitivity value-based decoding method of the present invention obtains different coded data of different types of macroblocks in the same frame image. The coded data of the macroblock is generated by dynamically adjusting the macroblock size and coding parameters according to the visual sensitivity value of the macroblock, thereby realizing refined processing of the image content. The method can significantly improve the decoding efficiency while ensuring the visual quality of the reconstructed image, and is particularly suitable for application scenarios with limited bandwidth resources but requiring high-quality video transmission.

[0145] An embodiment of the present disclosure provides a server, comprising: a processor and a memory storing a computer program that can be run on the processor, wherein the processor implements the steps of the data encoding method in the above embodiment when executing the computer program.

[0146] For the specific definition of the above-mentioned server, please refer to the definition of the data encoding method above, which will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0147] In an embodiment of the present disclosure, a terminal device is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above data decoding method can be implemented.

[0148] For the specific definition of the above-mentioned terminal device, please refer to the definition of the data encoding method above, which will not be repeated here. Each module in the above-mentioned data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0149] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0150] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0151] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0152] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image coding method, characterized in that: The method comprises: Acquire a target frame image, and divide the target frame image to generate M*N initial macroblocks; Traversing the M*N initial macroblocks, and determining the visual sensitivity value of each initial macroblock one by one; If the visual sensitivity value of the initial macroblock is greater than a preset threshold matched by the initial macroblock, directly encoding the initial macroblock to obtain first encoded data; If the visual sensitivity value of the initial macroblock is less than or equal to the preset threshold matched by the initial macroblock, the initial macroblock is extended and processed according to the preset macroblock extension method to obtain an extended macroblock, until the visual sensitivity mean value of the extended macroblock is greater than the preset threshold matched by the extended macroblock, the extension is stopped, and the extended macroblock is encoded to obtain second encoded data; The first coded data and the second coded data are fused to generate target data, and the target data is sent to the terminal device.

2. The data encoding method according to claim 1, characterized in that: The step of obtaining the extended macroblock includes: If the visual sensitivity value of the initial macroblock is less than a preset threshold matched by the initial macroblock, searching for a target macroblock adjacent to the initial macroblock in a first direction according to a preset direction; Merging the target macroblock and the initial macroblock to generate a merged macroblock, and determining a visual sensitivity mean of the merged macroblock; Determining whether the visual sensitivity mean of the merged macroblock is greater than a threshold matched by the merged macroblock; If the result is yes, the expansion process is stopped and the current merged macroblock is output as an extended macroblock; If the result is no, then continue with the expansion process.

3. The data encoding method according to claim 1, characterized in that: The preset threshold matched by the initial macroblock or the extended macroblock is determined at least according to a target mapping, wherein the target mapping is used to indicate a relationship between a physical parameter of at least one macroblock and a preset visual sensitivity value of the macroblock; If the size of the extended macroblock is larger than the size of the initial macroblock, the preset threshold of the extended macroblock is smaller than the threshold of the initial macroblock.

4. The data encoding method according to claim 1, characterized in that: The step of determining the visual sensitivity value of each initial macroblock comprises: Obtaining brightness information and texture information of each pixel in the initial macroblock; Processing the brightness information and texture information of each pixel in the initial macroblock by a target calculation formula to calculate the visual sensitivity value of the initial macroblock; Wherein, the target calculation formula includes: P(i,j)=T L (i,j)+T t (i,j) Among them, P(i,j) is used to indicate the visual sensitivity value of the current pixel; T L (i, j) is used to indicate the brightness information of the current pixel position. T t (i, j) is used to indicate the texture information of the current pixel location.

5. The data encoding method according to claim 4, characterized in that: Calculate the visual sensitivity value of the initial macroblock, including: The visual sensitivity value of the initial macroblock is calculated according to the first calculation formula and the brightness information of the pixel position in the macroblock, wherein the first calculation formula is used to indicate: In the first brightness interval, the brightness value of the pixel position is in an increasing relationship with the visual sensitivity value; In the second brightness interval, the brightness value of the pixel position is in a decreasing relationship with the visual sensitivity value; In the area where the first brightness interval and the second brightness interval overlap, the brightness value at the pixel location is the largest among all brightness intervals of the visual sensitivity value.

6. The data encoding method according to claim 4, characterized in that: Calculate the visual sensitivity value of the initial macroblock, including: The texture information of the pixel position in the macroblock is processed according to the second calculation formula to calculate the visual sensitivity value of the initial macroblock, wherein the second calculation formula includes: T t (i,j)=β×g(i,j)×e(i,j) Where β is used to indicate the preset control parameter; g(i,j) is used to indicate the average gradient weight near the current pixel; e(i,j) is used to indicate the strength value of edge pixels.

7. The data encoding method according to claim 1, characterized in that: The encoding quality of the first encoded data in the method is higher than the encoding quality of the second encoded data.

8. A data decoding method, characterized in that: The method comprises: Acquire target data, where the target data includes at least first coded data and second coded data; Decoding and processing the target data to obtain an extended macroblock and an initial macroblock; The extended macroblock and the initial macroblock are fused to obtain the target frame image; The first coded data refers to the coded data generated by the initial macroblock in the target frame image; The second coded data refers to a coded macroblock generated after the initial macroblock in the target frame image is extended, wherein the extended macroblock processing refers to: a macroblock obtained by merging the initial macroblock with an adjacent macroblock according to the visual sensitivity value of the current macroblock; The encoding quality of the first encoded data is higher than the encoding quality of the second encoded data.

9. A server, characterized in that: The server comprises: a processor and a memory storing a computer program executable on the processor, wherein the processor implements the steps of the data encoding method according to any one of claims 1 to 7 when executing the computer program.

10. A terminal device, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the data decoding method described in claim 8.

Images