Semantic perception adaptive compression coding method and device based on saliency detection

Abstract

The invention discloses a semantic perception adaptive compression coding method and device based on saliency detection. The method comprises the following steps: performing saliency detection on an original image based on a frequency coordination FT algorithm to obtain a saliency detection result; the saliency detection result comprises a saliency value of each pixel in the original image; dividing the original image into a plurality of image blocks, and performing importance grading on the image blocks according to a saliency detection result to obtain an importance level of each image block; performing adaptive compression coding on the image blocks according to the importance levels of the image blocks to obtain coded image blocks; and sending the coded image block to a channel for transmission. According to the method, collaborative optimization of the coding efficiency and the visual quality is realized on the premise of ensuring the semantic integrity of the saliency region of the image.

Description

Technical Field

[0001] The present application belongs to the field of optical communication technology, and specifically relates to a semantically-aware adaptive compression coding method and device based on saliency detection. Background Art

[0002] Multimedia data, including images, audio, and video, has become the mainstream form of information exchange. Due to its richness and intuitive nature, these content formats have been widely used across various sectors of society, significantly enriching communication. However, with the surge in high-resolution and high-data-rate content, the demand for network bandwidth and storage space has also increased dramatically, potentially leading to network congestion and processing delays. Therefore, if we can leverage the inherent properties of multimedia data, explore its semantic features, and design more efficient multimedia data transmission solutions, we can effectively conserve communication resources and alleviate bandwidth pressure.

[0003] Saliency detection (SD) is an image semantic perception method that locates salient regions within an image based on image features and statistical information, allowing for block-based processing of multimedia images. Traditional image processing methods rely on fixed block processing, which underutilizes the characteristics of the human visual system. However, semantic perception based on SD can effectively remove redundancy from image features, avoiding inefficient feature extraction and compressed transmission.

[0004] Compressed sensing (CS) technology can reconstruct signals from a small number of measurements at sampling rates far below the traditional Nyquist rate, making it ideally suited for compressing sparse data images. Therefore, based on image SD, semantically aware block-based adaptive compression coding of multimedia images based on image features effectively balances the important semantic expression of images with transmission efficiency. Summary of the Invention

[0005] Purpose: In view of at least one of the above technical problems, this application aims to solve the technical contradiction in existing multimedia image compression coding technology that it is difficult to balance the loss of semantic information and transmission efficiency. In order to solve the problem that traditional fixed block compression sampling destroys the semantic structure of the image, a semantic-aware adaptive compression coding method and device based on saliency detection is provided. The frequency coordination (FT) algorithm is used to perform image saliency detection SD. On this basis, a dynamic quantization parameter adjustment strategy is adopted to grade the importance of each image block. Different degrees of adaptive CS measurement are performed on pictures with different characteristics. Under the premise of ensuring the semantic integrity of the salient areas of the image, the coordinated optimization of coding efficiency and visual quality is achieved.

[0006] This paper introduces FT saliency detection technology to achieve precise segmentation of image semantic regions, overcoming the limitations of traditional fixed-block segmentation methods. By combining image semantic features with compressed sensing, a dynamic compression ratio adjustment mechanism based on saliency levels is constructed to improve the semantic reconstruction performance of CS. Furthermore, by constructing differentiated CS encoding for salient and non-salient regions, this method achieves a coordinated optimization of coding efficiency and semantic fidelity while maximizing image semantics.

[0007] The technical solutions adopted in this application are:

[0008] In a first aspect, the present application provides a semantically-aware adaptive compression coding method based on saliency detection, which is applied to a sending end and includes:

[0009] Get the original image to be sent;

[0010] Performing saliency detection on the original image based on the frequency coordination FT algorithm to obtain a saliency detection result; wherein the saliency detection result includes a saliency value of each pixel in the original image;

[0011] The original image is divided into multiple image blocks, and the importance of the image blocks is graded according to the saliency detection results to obtain the importance level of each image block;

[0012] Adaptively compressing and encoding the image block according to the importance level of the image block to obtain an encoded image block;

[0013] The coded image blocks are sent to the channel for transmission.

[0014] In a second aspect, the present application provides a semantically aware adaptive compression coding device based on saliency detection, comprising a processor and a storage medium;

[0015] The storage medium is used to store instructions;

[0016] The processor is configured to operate according to the instructions to perform the method according to the first aspect.

[0017] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the computer program is executed by a processor.

[0018] In a fourth aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0019] Beneficial Effects: The semantic-aware adaptive compression coding method and device based on saliency detection provided by this application has the following advantages: by accurately mining the semantic features of multimedia data and performing targeted coding, it can significantly reduce the demand for network bandwidth and storage space, effectively avoid network congestion and processing delay problems, and enable smooth transmission of more high-resolution, high-data-rate multimedia content under existing communication infrastructure conditions, greatly improving the overall utilization of communication resources. On the other hand, in terms of image quality assurance, the differentiated CS coding of salient and non-salient areas, as well as the dynamic compression ratio adjustment mechanism based on saliency level, ensure that key semantic information of the image is not lost and restore the useful information of the image to the greatest extent, while taking into account both visual quality and transmission efficiency, and comprehensively improving the comprehensive performance of multimedia data processing and transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of a semantic-aware adaptive compression coding method based on saliency detection according to an embodiment of the present application;

[0021] Figure 2 Schematic diagram of importance block classification results of saliency detection based on the FT algorithm of "cameraman" according to one embodiment of the present application;

[0022] Figure 3 Schematic diagram of adaptive compression coding based on importance grading results according to an embodiment of the present application;

[0023] Figure 4 This is a signal-to-noise ratio-bit error rate curve obtained by simulating transmission through an AWGN channel using 16QAM-OFDM modulation according to an embodiment of the present application;

[0024] Figure 5 15dB is a receiving end constellation diagram obtained by the system according to an embodiment of the present application under the condition of a signal-to-noise ratio of 15dB;

[0025] Figure 6 The figure is a schematic diagram showing the quality comparison between the decompressed restored image and the original image when the signal-to-noise ratio is fixed at 15dB according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application will be further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present application and are not intended to limit the scope of protection of the present application.

[0027] In the description of this application, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0028] Throughout the description of this application, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0029] The term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.

[0030] Example 1: This example provides a semantic-aware adaptive compression coding method based on saliency detection, which is applied to the sending end, such as Figure 1 Shown, including:

[0031] S1. Obtain the original image to be sent;

[0032] S2. Performing saliency detection on the original image based on a frequency-harmonized FT algorithm to obtain a saliency detection result; wherein the saliency detection result includes a saliency value of each pixel in the original image;

[0033] S3, dividing the original image into multiple image blocks, and performing importance grading on the image blocks according to the saliency detection results to obtain an importance level for each image block;

[0034] S4, performing adaptive compression coding on the image block according to the importance level of the image block to obtain a coded image block;

[0035] S5. Send the encoded image block to the channel for transmission.

[0036] At the receiving end, the original image can be restored using the opposite process of the sending end. This approach not only improves transmission efficiency but also effectively protects important parts of the image.

[0037] In some embodiments, in S2, performing saliency detection on the original image based on the frequency-harmonized FT algorithm to obtain a saliency detection result includes:

[0038] Convert the original image to LAB color space; in LAB color space, each pixel of the original image is represented as a three-dimensional vector , where L represents brightness, a and b represent the red-green and blue-yellow components of the color respectively;

[0039] Calculate the global mean of the original image in the LAB color space ,in 、 and are the means of L, a and b components respectively;

[0040] For each pixel in the original image, the saliency value of the pixel is obtained by calculating the Euclidean distance between the pixel and the global mean. .

[0041] Furthermore, the saliency value of the pixel The calculation formula is:

[0042] ,

[0043] in, Represents pixels The significance value of Represents pixels brightness; 、 Represents pixels The red-green and blue-yellow components of the color.

[0044] The FT algorithm performs saliency detection based on the color contrast of the image. This calculation formula measures the degree of difference between each pixel and the overall color distribution of the image. The greater the difference, the more significant the pixel is and the more likely it is to be the target or region of interest in the image. In this way, the FT algorithm can quickly and effectively detect salient areas in the image, providing important prior information for subsequent image processing tasks. Figure 2 The following shows the results of FT saliency detection importance block classification using the 256×256 grayscale image "cameraman". Figure 2 It can be seen that the FT algorithm can simulate the human visual system very well and detect and grade the relatively important parts that are easy to attract the human visual attention.

[0045] After completing the saliency detection of the image, the importance level of each block can be divided.

[0046] In some embodiments, in S3, the importance of the image blocks is graded according to the saliency detection result to obtain the importance level of each image block, including:

[0047] S31. For each image block, calculate the average saliency value of all pixels in the image block to obtain an average saliency value of the image block;

[0048] Specifically, step S31 includes:

[0049] The saliency value set of pixels in an image block is ,in is the number of pixels in the image block, and the average saliency value of the image block for:

[0050] ;

[0051] in is the pixel in the image block The significance value of .

[0052] S32. Classify the image blocks into different importance levels according to the average saliency values ​​of the image blocks to obtain the importance level of each image block.

[0053] Specifically, step S32 includes:

[0054] set up is the average saliency value of the image block, is the first set value, is the second set value;

[0055] like , then the image block is judged as "unimportant" and the importance level is marked as 1;

[0056] like , then the image block is judged as “generally important” and the importance level is marked as 2;

[0057] like , then the image block is judged as the “most important” and the importance level is marked as 3.

[0058] According to the importance level of the image blocks, different compression ratios can be adopted for different image blocks. As can be seen from the above, the importance levels of the image blocks are divided into 1, 2, and 3. Then, in S4, the image blocks are adaptively compressed and encoded according to the importance levels of the image blocks to obtain encoded image blocks, including:

[0059] For image blocks with importance level 3, a compression ratio of 3 / 4 is used for compressed sensing coding;

[0060] For image blocks with importance level 2, compressed sensing coding is performed with a compression ratio of 1 / 2;

[0061] For image blocks with importance level 1, compressed sensing coding is performed with a compression ratio of 1 / 4.

[0062] like Figure 3 Figure 2 shows a schematic diagram of adaptive compression coding based on importance grading. While higher compression ratios for compressed-sensing (CS) image coding can reduce data usage, the quantization process inevitably results in significant image quality loss. Therefore, differential CS coding with dynamic quantization effectively protects image blocks containing rich semantic information while significantly reducing the data size and transmission burden for blocks with fewer semantic features. This approach achieves the ultimate balance between image semantic information transmission and efficiency.

[0063] Furthermore, in some embodiments, the method further includes: after the receiving end receives the coded image block transmitted via the channel, performing the following steps:

[0064] The SL0 algorithm is used to perform compressed sensing decoding, block reorganization, and semantic feature reconstruction on the coded image blocks to obtain the restored image.

[0065] The encoded image blocks obtained after channel transmission are then decoded using the SL0 algorithm for compressed sensing. The core idea of ​​the SL0 algorithm is to exploit the sparsity of the signal (i.e., the image blocks have few non-zero coefficients in the transform domain) to reconstruct the original signal. This is achieved by minimizing the regularized objective function:

[0066]

[0067] in, represents the original signal vector to be restored, represents the observed signal vector, represents the measurement matrix, represents the reconstruction error, is a regularization parameter used to balance reconstruction error and signal sparsity. The SL0 algorithm gradually approaches the optimal solution through iterative optimization. Each step combines Landweber iteration and soft thresholding to ensure the sparsity of the solution.

[0068] Block reconstruction is the process of reassembling independently decoded image blocks according to their original spatial positions to restore the complete image. At the receiving end, the image blocks decoded by the SL0 algorithm must be accurately reassembled based on their position within the original image. This process requires the receiving end to obtain the image block index information, which is transmitted as a frame header along with the compressed data during transmission. The reconstruction algorithm uses this index information to place the decoded image blocks in the correct positions, forming a preliminary reconstructed image.

[0069] Thanks to a differentiated compression strategy based on semantic importance, semantically important regions of the image—which typically contain the most recognizable visual content—retain more detailed information during transmission. During reconstruction, the precise reconstruction of these high-importance regions ensures the semantic integrity of the image as a whole. Furthermore, even with some quality loss at higher compression ratios for less important regions, this does not significantly impact the viewer's understanding and perception of the image content.

[0070] This approach prioritizes the quality of semantically important parts of an image under limited transmission resources. Compared to traditional uniform compression methods, it achieves higher subjective visual quality at the same average compression ratio. This advantage is particularly pronounced when transmission bandwidth is limited, ensuring that critical image information is preserved even if certain parts of the image (typically the background or less important areas) are blurred or distorted.

[0071] Matlab simulation verification of semantic-aware adaptive compression coding method based on SD: Figure 4 The figure shows the signal-to-noise ratio-bit error rate curve obtained by simulating transmission through an AWGN channel using 16QAM-OFDM modulation in this application solution. Figure 5 The figure shows the receiver constellation diagram obtained when the signal-to-noise ratio is 15dB. It can be seen from the figure that the bit error rate performance of the communication system with 16QAM-OFDM modulation improves with the increase of signal-to-noise ratio. In addition, when the signal-to-noise ratio is greater than 13dB, the system bit error rate is lower than the forward error correction threshold of 3.8×10 -3 , which shows that the communication system using 16QAM-OFDM modulation has good transmission performance.

[0072] like Figure 6 The figure shows a comparison of the quality of the decompressed image and the original image, with the signal-to-noise ratio fixed at 15dB. To avoid randomness, the decompression performance of various images after transmission was tested. The figure shows that the decompressed image quality loss is minimal, and key semantic information is fully preserved, effectively reducing the amount of transmitted data while successfully transmitting image semantic features.

[0073] Example 2: Based on Example 1, this example provides a semantically aware adaptive compression coding device based on saliency detection, including a processor and a storage medium;

[0074] The storage medium is used to store instructions;

[0075] The processor is configured to operate according to the instructions to execute the method according to embodiment 1.

[0076] Example 3: Based on Example 1, this example provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in Example 1 is implemented.

[0077] Example 4: Based on Example 1, this example provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in Example 1 when executing the computer program.

[0078] Example 5: Based on Example 1, this example provides a computer program product, including a computer program. When the computer program is executed by a processor, the method described in Example 1 is implemented.

[0079] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0083] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A semantically aware adaptive compression coding method based on saliency detection, applied to a sending end, characterized in that: include: Get the original image to be sent; The saliency detection result is obtained by performing saliency detection on the original image based on the frequency coordination FT algorithm; The saliency detection result includes a saliency value of each pixel in the original image; The original image is divided into multiple image blocks, and the importance of the image blocks is graded according to the saliency detection results to obtain the importance level of each image block; Adaptively compressing and encoding the image block according to the importance level of the image block to obtain an encoded image block; The coded image blocks are sent to the channel for transmission.

2. The method according to claim 1, characterized in that The saliency detection results of the original image are obtained based on the frequency coordination FT algorithm, including: Convert the original image to LAB color space; in LAB color space, each pixel of the original image is represented as a three-dimensional vector , where L represents brightness, a and b represent the red-green and blue-yellow components of the color respectively; Calculate the global mean of the original image in the LAB color space ,in 、 and are the means of L, a and b components respectively; For each pixel in the original image, the saliency value of the pixel is obtained by calculating the Euclidean distance between the pixel and the global mean. .

3. The method according to claim 2, characterized in that Pixel saliency value The calculation formula is: ， in, Represents pixels The significance value of Represents pixels brightness; 、 Represents pixels The red-green and blue-yellow components of the color.

4. The method according to claim 1, wherein The importance of each image block is graded according to the saliency detection results, including: For each image block, calculate the average saliency value of all pixels in the image block to obtain the average saliency value of the image block; According to the average saliency value of the image blocks, the image blocks are divided into different importance levels to obtain the importance level of each image block.

5. The method according to claim 4, characterized in that For each image block, the average saliency value of all pixels in the image block is calculated to obtain the average saliency value of the image block, including: The saliency value set of pixels in an image block is ,in is the number of pixels in the image block, and the average saliency value of the image block for: ； in is the pixel in the image block The significance value of .

6. The method according to claim 4, characterized in that According to the average saliency value of the image block, the image block is divided into different importance levels, including: set up is the average saliency value of the image block, is the first set value, is the second set value; like , then the importance level of the image block is marked as 1; like , then the importance level of the image block is marked as 2; like , then the importance level of the image block is marked as 3.

7. The method according to claim 1, characterized in that The importance levels of the image blocks are divided into 1, 2, and 3. The image blocks are adaptively compressed and encoded according to the importance levels of the image blocks to obtain encoded image blocks, including: For image blocks with importance level 3, a compression ratio of 3 / 4 is used for compressed sensing coding; For image blocks with importance level 2, compressed sensing coding is performed with a compression ratio of 1 / 2; For image blocks with importance level 1, compressed sensing coding is performed with a compression ratio of 1 / 4.

8. A semantically aware adaptive compression coding device based on saliency detection, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images