Grayscale image compression method in weak network environment

Abstract

The present application relates to the technical field of image processing, discloses a grayscale image compression method in a weak network environment, and aims to solve the problem of suboptimal image compression sizes caused by compression methods focusing only on individual images without considering contextual correlations of the images within actual scenes. Key points of the technical solution are: S1: performing correlation extraction; S2: implementing the process of the grayscale image compression method in a weak network environment: first obtaining a data source, and then performing division of macro blocks according to requirements; S3: performing information conversion: converting two-dimensional pixel information into one-dimension pixel information, and finally obtaining one-dimensional discrete curve sequences lcur and lpre corresponding to mcur and mpre; S4: calculating a Fréchet distance FreDis; and S5: determining a transmission state on the basis of the value of FreDis: calculating the Fréchet distance FreDis after micro blocks having the same pixel are converted into one-dimensional discrete curves, and determining whether to perform transmission. The effects of reducing transmission delay and improving experience of users in weak network environments are achieved.

Description

A grayscale image compression method in weak network environment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 13, 2023, with application number 202311712677.3 and invention name “A grayscale image compression method in a weak network environment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of image processing technology, and in particular to a grayscale image compression method in a weak network environment. Background Art

[0004] A grayscale image is an image that only contains brightness information. Compared with color images, it has a smaller file size and is simpler to process. It is widely used in medical imaging, computer vision, and image processing. Taking image processing as an example, grayscale images can be applied to edge detection, image enhancement, image segmentation, etc. These technologies can help improve image quality, extract effective information, and improve visual effects.

[0005] Common image compression methods include lossless compression and lossy compression. Lossless compression does not lose image information and can ensure that the compressed image is completely consistent with the original image. Common lossless compression methods include Huffman coding and LZW coding. Huffman coding achieves lossless compression by counting the frequency of grayscale values, representing frequently occurring grayscale values ​​with shorter codes and less frequently occurring grayscale values ​​with longer codes. LZW coding is a dictionary-based compression method that achieves compression by establishing a dictionary and dynamically updating it. Lossy compression methods can reduce image size to a certain extent, but will lose some image information. Common lossy compression methods include JPEG, WebP, and Wavelet transform. JPEG is a widely used lossy compression method that converts images into frequency domain representation through discrete cosine transform (DCT) and then achieves compression through techniques such as quantization and entropy coding. The WebP algorithm is a compression algorithm developed by Google based on predictive coding and vector quantization. Wavelet transform decomposes images into wavelet coefficients at multiple scales and directions and then achieves compression through techniques such as quantization and entropy coding. It is a multi-resolution analysis technology.

[0006] The above-mentioned existing technical solutions have the following defects: the compression method discussed above is only for individual images and cannot find the contextual association of the image in combination with the actual scene, which also leads to unsatisfactory image compression size. Under the limited bandwidth of a weak network environment, it undoubtedly increases the transmission delay and affects the user experience. At the same time, compared with the color image mode, the user's eye sensitivity to the image will be reduced in the grayscale image mode, and the perception of burrs will also be reduced. Traditional multi-channel image compression algorithms do not take this feature into account. Summary of the Invention

[0007] The purpose of this application is to provide a grayscale image compression method in a weak network environment that utilizes the commonality between adjacent images to improve the compression rate, greatly reduces the end-to-end network data transmission volume, reduces the transmission delay, and improves the user experience in a weak network environment without almost affecting the user experience.

[0008] To achieve the above objectives, this application provides the following technical solutions:

[0009] A grayscale image compression method in a weak network environment is as follows:

[0010] S1: Correlation extraction: extract the correlation parts of adjacent frames in video conferencing, remote assistance, and remote desktop scenarios, and grayscale process the extracted correlation parts;

[0011] S2: Grayscale image compression method process in a weak network environment, first obtain the data source and divide it into macroblocks as needed;

[0012] S3: Information conversion, starting from the dirty area of ​​the current frame (x cur ,y cur ) to extract macroblock m row by row and column by column cur , the top position of the previous frame full image area (x pre ,y pre ) Extract macro m pre , at this time y pre =0, then use the formula to calculate the offset and convert the two-dimensional pixel information into one dimension, and finally get m cur and m pre The corresponding one-dimensional discrete curve sequence l cur and l pre ;

[0013] S4: Calculate the Fréchet distance FreDis, assuming l cur and l pre It is composed of n trajectory points, and l cur ,l pre Orderly expressed as α(l cur )=(u1,…,u n),β(l pre )=(v1,…,v n ), l cur and l pre The sequence length ‖L‖ between trajectory points is defined as the maximum value of the Euclidean distance between each point pair, and the discrete curve sequence l is calculated by ‖L‖ cur and l pre Freche distance FreDis;

[0014] S5: Determine the transmission status according to the value of FreDis, calculate the Frechet distance FreDis converted into a one-dimensional discrete curve for macroblocks with the same pixels, and determine whether to transmit. If the distance is 0, part of the compression calculation process and the data transmission process are omitted.

[0015] Optionally, the macroblock division in S2 specifically divides the image into multiple NxN macroblocks, where N=8, 16, 32, or 64.

[0016] Optionally, the information conversion in S3 is calculated using the following formula: y=x+y*pic_stride pic_stride=img_width*channels.

[0017] Optionally, x and y in S3 represent the horizontal and vertical coordinates of the pixel in the image, respectively, pic_stride represents the memory offset of an entire row of pixels in the image, img_width is the width of the image, and channels is the number of color channels of the image.

[0018] Optionally, in S4, cur and l pre The sequence length ‖L‖ between trajectory points is calculated using the following formula:

[0019] Optionally, the Fréchet distance FreDis in S4 is calculated using the following formula:

[0020] Optionally, in the S5, if the transmission state is determined according to the value of FreDis, and FreDis is not less than 30, then (x pre ,y pre ) block at this position, at this time y pre =y pre +1, go back to S2 and continue with the above steps.

[0021] Optionally, in the S5, if the transmission status is determined according to the value of FreDis, and FreDis is greater than 30, it is further determined whether FreDis is 0, and if it is 0, the transmission status is determined from the previous frame (x pre ,ypre ) position copies the pixel to the current frame (x cur ,y cur )Location.

[0022] Optionally, in said S5, if the transmission state is determined according to the value of FreDis and is not 0, the pixel residual diff(l cur ,l pre ), transmitted in residual form.

[0023] Optionally, the data source acquisition in S2 is specifically acquiring a grayscale image from a graphics card driver.

[0024] In summary, the beneficial technical effects of this application are:

[0025] 1. This application proposes a solution for converting a two-dimensional macroblock image into a one-dimensional sequence to find macroblock associations in scenarios such as video conferencing, remote assistance, and remote desktop in weak network environments. This method greatly reduces the number of memory accesses through dimensionality reduction, and this conversion method provides a basis for rapidly finding macroblock matches in subsequent steps.

[0026] 2. The one-dimensional discrete curve similarity evaluation decision scheme obtained after dimensionality reduction of the macroblock image in this application uses the curve distance calculation formula to obtain a specific value, and uses the value to evaluate the similarity of the macroblocks. If the value is 0, the surface macroblocks are exactly the same. If the value is greater than 0 and less than 30, they are considered to be similar macroblocks. A transmission residual scheme is adopted. This scheme can effectively evaluate the similarity between macroblocks, make full use of the commonality between adjacent images to improve the compression rate, and greatly reduce the end-to-end network data transmission volume without almost affecting the user experience, reduce transmission delay, and improve the user experience in weak network environments;

[0027] 3. This application combines the Fréchet algorithm to evaluate the similarity of macroblock images after serialization. This method makes full use of the commonality between adjacent images to improve the compression rate. Without affecting the user experience, it greatly reduces the end-to-end network data transmission volume, reduces the transmission delay, and improves the user experience in a weak network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a schematic diagram of the same pixel macroblock matching hit situation of the present application;

[0029] FIG2 is a schematic diagram of the transmission residual of similar macroblock matching in this application;

[0030] FIG3 is a schematic diagram of a portion of the content area that is consistent between the previous and next frames of this application;

[0031] Figure 4 is a schematic diagram of the overall process structure of this application. DETAILED DESCRIPTION

[0032] The present application method is further described in detail below with reference to the accompanying drawings.

[0033] Example 1

[0034] Referring to FIG4 , a grayscale image compression method in a weak network environment is as follows:

[0035] S1: Correlation extraction: extract the correlation parts of adjacent frames in video conferencing, remote assistance, and remote desktop scenarios, and grayscale process the extracted correlation parts;

[0036] S2: A grayscale image compression method in a weak network environment. First, a data source is obtained and macroblocks are divided as needed. Specifically, the image is divided into multiple NxN macroblocks, where N = 8, 16, 32, or 64. The data source is obtained by obtaining a grayscale image from a graphics card driver.

[0037] S3: Information conversion, starting from the dirty area of ​​the current frame (x cur ,y cur ) to extract macroblock m row by row and column by column cur , the top position of the previous frame full image area (x pre ,y pre ) Extract macro m pre , at this time y pre =0, then use the formula to calculate the offset and convert the two-dimensional pixel information into one dimension, and finally get m cur and m pre The corresponding one-dimensional discrete curve sequence l cur and l pre ;

[0038] The following formula is used for calculation: y=x+y*pic_stride pic_stride=img_width*channels.

[0039] Where x and y represent the horizontal and vertical coordinates of the pixel in the image, pic_stride represents the memory offset of a whole row of pixels in the image, img_width is the width of the image, and channels is the number of color channels of the image;

[0040] S4: Calculate the Fréchet distance FreDis, assuming l cur and l pre It is composed of n trajectory points, and l cur ,l pre Orderly expressed as α(l cur )=(u1,…,u n ),β(l pre )=(v1,…,v n ), l cur and lpre The sequence length ‖L‖ between trajectory points is defined as the maximum value of the Euclidean distance between each point pair, and the discrete curve sequence l is calculated by ‖L‖ cur and l pre Freche distance FreDis;

[0041] S5: Determine the transmission status according to the value of FreDis, calculate the Frechet distance FreDis after converting it into a one-dimensional discrete curve for the macroblocks with the same pixels, and determine whether to transmit. If the distance is 0, part of the compression calculation process and data transmission process are omitted. If FreDis is not less than 30, then take (x pre ,y pre ) block at this position, at this time y pre =y pre +1, return to S2 and continue to execute the above steps. If FreDis is greater than 30, further determine whether FreDis is 0. If it is 0, start from the previous frame (x pre ,y pre ) position copies the pixel to the current frame (x cur ,y cur ) position, if it is not 0, calculate the pixel residual diff(l cur ,l pre ), transmitted in residual form.

[0042] Example 2

[0043] Scenarios such as video conferencing, remote assistance, and remote desktop in weak network environments have their own unique characteristics. First, images obtained by graphics card drivers often have a strong temporal correlation. Second, users are much less sensitive to grayscale images than color images. Based on these scenario characteristics, this patent application proposes a grayscale image compression method in weak network environments. This method combines the Fréchet algorithm to evaluate the similarity of macroblock images after serialization. This method fully utilizes the commonality between adjacent images to improve the compression rate. Without affecting the user experience, it greatly reduces the end-to-end network data transmission volume, reduces transmission latency, and improves the user experience in weak network environments.

[0044] First, let's briefly explain the correlation between adjacent frames in the aforementioned scenarios of video conferencing, remote assistance, and remote desktop. As shown in Figure 3, when a user scrolls a web page, the underlying graphics engine transmits the image to the graphics driver. Assuming the image before scrolling is frame a and the image after scrolling is frame b, it can be found that the dirty areas of frame b and frame a overlap, and the part within the red framed area contains the same content.

[0045] The grayscale image compression method in a weak network environment proposed in the application is described step by step according to the method flow chart of FIG4 :

[0046] 1) First, the data source is obtained. The grayscale image is obtained from the graphics card driver. At the same time, the macroblocks are divided according to the needs. The image is divided into multiple NxN macroblocks (N=8, 16, 32, 64).

[0047] 2) From the starting position of the dirty area of ​​the current frame (x cur ,y cur ) to extract macroblock m row by row and column by column cur , the top position of the previous frame full image area (x pre ,y pre ) Extract macro m pre , at this time y pre = 0, then use the following formula to calculate the offset and convert the two-dimensional pixel information into one dimension, where x and y represent the horizontal and vertical coordinates of the pixel in the image, pic_stride represents the memory offset of a whole row of pixels in the image, img_width is the width of the image, and channels is the number of color channels of the image. Finally, we can get m cur and m pre The corresponding one-dimensional discrete curve sequence l cur and l pre . y=x+y*pic_stride pic_stride=img_width*channels

[0048] 3) Assumption l cur and l pre It is composed of n trajectory points, and l cur ,l pre Orderly expressed as α(l cur )=(u1,…,u n ),β(l pre )=(v1,…,v n ), l cur and l pre The sequence length ‖L‖ between trajectory points is defined as the maximum value of the Euclidean distance among each pair of points.

[0049] Finally, the discrete curve sequence l can be calculated using the following formula cur and l pre Fréchet is from FreDis.

[0050] 4) If FreDis is not less than 30, then take (x pre ,y pre ) block at this position, at this time y pre =ypre +1, go back to step 2) and continue with the above steps.

[0051] 5) If FreDis is greater than 30, then further determine whether FreDis is 0. If it is 0, then start from the previous frame (x pre ,y pre ) position copies the pixel to the current frame (x cur ,y cur ) position, if it is not 0, calculate the pixel residual diff(l cur ,l pre ), transmitted in residual form;

[0052] Example 3

[0053] At present, we have implemented a feasible codec based on the above solution, and it has been verified to meet the requirements. As shown in Figure 1, there are identical pixel macroblocks in the QR code part of WeChat. According to the above algorithm, the Fréchet distance between macroblock a and macroblock b after serialization into a one-dimensional discrete curve can be calculated. The distance is 0, which indicates that macroblock a is exactly the same as macroblock b, thus eliminating the compression calculation process and data transmission process of macroblock b.

[0054] Another situation is shown in Figure 2. There is diversity in the front-end drawing components of the application software, and there are often multiple layers stacked. In the areas that appear to be pure colors to the naked eye, there are differences in the actual pixels. Taking the WeChat contact background interface in Figure 2 as an example, there are slight differences in the grayscale pixels of macroblocks c and d. According to the method provided by this patent, FreDist = 3 can be calculated, so the strategy is to transmit the residual. It can also be seen in the figure that the cost of transmitting the residual is much less than transmitting the original data.

[0055] It is important to note that the construction and arrangement of the present application shown in a number of different exemplary embodiments are merely illustrative. Although only a few embodiments are described in detail in this disclosure, it should be readily understood by those reading this disclosure that many modifications are possible (e.g., the size, scale, structure, shape and proportion of various elements, as well as parameter values ​​(e.g., temperature, pressure, etc.), mounting arrangements, use of materials, color, changes in orientation, etc.) without materially departing from the novel teachings and advantages of the subject matter described in this application. For example, elements shown as integrally formed may be composed of multiple parts or elements, the positions of elements may be inverted or otherwise changed, and the nature or number or position of discrete elements may be altered or changed, and therefore, all such modifications are intended to be included within the scope of this application. The order or sequence of any process or method steps may be changed or reordered according to alternative embodiments. In the claims, any "means plus function" clause is intended to cover the structures described herein that perform the function, and not only structural equivalence but also equivalent structures. Without departing from the scope of the present application, other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the exemplary embodiments. Therefore, the present application is not limited to a specific embodiment, but extends to various modifications that still fall within the scope of the appended claims.

[0056] Additionally, in order to provide a concise description of example embodiments, all features of an actual embodiment (ie, those features not relevant to the best mode presently contemplated for carrying out the application or those not relevant to implementing the application) may not be described.

[0057] It will be appreciated that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will be a routine task of design, fabrication, and production for those of ordinary skill having the benefit of this disclosure without undue experimentation.

[0058] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, and all of these should be included in the scope of the claims of the present application.

Claims

1. A grayscale image compression method in a weak network environment, characterized by: The method is as follows: S1: Correlation extraction: extract the correlation parts of adjacent frames in video conferencing, remote assistance and remote desktop scenarios, and grayscale the extracted correlation parts; S2: Grayscale image compression method process in a weak network environment, first obtain the data source and divide the macroblocks as needed; S3: Information conversion, starting from the dirty area of ​​the current frame (x cur ,y cur ) to extract macroblock m row by row and column by column cur , the top position of the previous frame full image area (x pre ,y pre ) Remove the macro m pre , at this time y pre = 0, then use the formula to calculate the offset, convert the two-dimensional pixel information to one dimension, and finally get m cur and m pre The corresponding one-dimensional discrete curve sequence l cur and l pre ; S4: Calculate the Fréchet distance FreDis, assuming l cur and l pre It is composed of n trajectory points, and l cur ,l pre It is expressed in order as α(l cur )=(u1,…,u n ),β(l pre )=(v1,…,v n ), l cur and l pre The sequence length ‖L‖ between trajectory points is defined as the maximum value of the Euclidean distance among the points, and the discrete curve sequence l is calculated by ‖L‖ cur and l pre Freche distance FreDis; S5: judging the transmission status according to the value of FreDis, calculating the Freche distance FreDis converted into a one-dimensional discrete curve for macroblocks with the same pixels, and judging whether to transmit. If the distance is 0, part of the compression calculation process and the data transmission process are omitted.

2. The grayscale image compression method in a weak network environment according to claim 1, characterized in that: The macroblock division in S2 specifically divides the image into a plurality of NxN macroblocks, where N=8, 16, 32, 64.

3. The grayscale image compression method in a weak network environment according to claim 2, characterized in that: The information conversion in S3 is calculated using the following formula: y=x+y*pic_stride pic_stride=img_width*channels.

4. The grayscale image compression method in a weak network environment according to claim 3, characterized in that: In S3, x and y represent the horizontal and vertical coordinates of the pixel in the image, respectively; pic_stride represents the memory offset of a whole row of pixels in the image; img_width is the width of the image; and channels is the number of color channels of the image.

5. The grayscale image compression method in a weak network environment according to claim 4, characterized in that: S4 cur and l pre The sequence length ‖L‖ between trajectory points is calculated using the following formula:

6. The grayscale image compression method in a weak network environment according to claim 5, characterized in that: The Frechet distance FreDis in S4 is calculated using the following formula:

7. The grayscale image compression method in a weak network environment according to claim 6, characterized in that: In the S5, if the transmission state is determined according to the value of FreDis, and FreDis is not less than 30, then (x pre ,y pre ) block, at this time y pre =y pre +1, go back to S2 and continue the above steps.

8. The grayscale image compression method in a weak network environment according to claim 7, characterized in that: In the S5, if the transmission state is determined according to the value of FreDis, and FreDis is greater than 30, it is further determined whether FreDis is 0. If it is 0, the previous frame (x pre ,y pre ) position copies the pixel to the current frame (x cur ,y cur )Location.

9. The grayscale image compression method in a weak network environment according to claim 8, characterized in that: In S5, the transmission state is determined according to the value of FreDis. If it is not 0, the pixel residual diff(l cur ,l pre ) and transmitted in residual form.

10. The grayscale image compression method in a weak network environment according to claim 9, characterized in that: The data source acquisition in S2 is specifically to acquire a grayscale image from a graphics card driver.

Images