High-resolution image transmission method and system based on optical communication

Abstract

The present application provides a high-resolution image transmission method and system based on optical communication, which relates to the field of image transmission. The method includes: dividing the high-resolution image to be transmitted into multiple image blocks; for each image block, determining a target compression strategy that matches the content complexity of the image block, and compressing the image block according to the target compression strategy; generating a corresponding image coding mode according to the target compression strategy of each image block; transmitting each compressed image block and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes each received compressed image block and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; the conditional vector of the conditional generative adversarial network is determined according to the image coding mode. Thus, the compression rate and image quality are effectively balanced during the image transmission process, and high-quality transmission of high-resolution images is achieved.

Description

Technical Field

[0001] The present application relates to the field of image transmission technology, and in particular to a high-resolution image transmission method and system based on optical communication. Background Art

[0002] Traditional image compression techniques, such as JPEG and PNG, compress image data by reducing redundant information and exploiting the characteristics of the human visual system. These methods perform well in compressing lower-resolution images, but their limitations become apparent as image resolution increases.

[0003] First, traditional compression technology has difficulty in balancing compression rate and image quality when facing high-resolution images, especially in terms of lossless compression. In addition, the real-time transmission requirement further exposes the shortcomings of traditional compression technology, because data loss and compression artifacts during the compression process not only affect the visual effect of the image, but may also lead to the loss of key details, reducing the transmission quality and application value of high-resolution images.

[0004] In response to the above problems, the industry has not yet proposed a better technical solution. Summary of the invention

[0005] The present application provides a high-resolution image transmission method, system, storage medium, computer program product and electronic device based on optical communication, which are used to at least solve the problem that the compression rate and image quality cannot be balanced when compressing and transmitting high-resolution images.

[0006] In a first aspect, an embodiment of the present application provides a high-resolution image transmission method based on optical communication, which is applied to an optical communication transmitter, and the method includes: dividing a high-resolution image to be transmitted into multiple image blocks; each of the image blocks has a corresponding content complexity; for each of the image blocks, determining a target compression strategy that matches the content complexity of the image block, and compressing the image block according to the target compression strategy to obtain a corresponding compressed image block; generating a corresponding image coding mode according to the target compression strategy of each of the image blocks; transmitting each of the compressed image blocks and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes each of the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; the conditional vector of the conditional generative adversarial network is determined according to the image coding mode.

[0007] In the second aspect, an embodiment of the present application provides a high-resolution image transmission system based on optical communication, which is deployed in an optical communication transmitter, and the system includes: an image segmentation unit, which is used to segment the high-resolution image to be transmitted into multiple image blocks; each of the image blocks has a corresponding content complexity; a strategy matching unit, which is used to determine, for each of the image blocks, a target compression strategy that matches the content complexity of the image block, and compress the image block according to the target compression strategy to obtain a corresponding compressed image block; a coding mode generation unit, which is used to generate a corresponding image coding mode according to the target compression strategy of each of the image blocks; a data transmission unit, which is used to transmit each of the compressed image blocks and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; the conditional vector of the conditional generative adversarial network is determined according to the image coding mode.

[0008] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the high-resolution image transmission method based on optical communication of any embodiment of the present application.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the high-resolution image transmission method based on optical communication of any embodiment of the present application are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the high-resolution image transmission method based on optical communication of any embodiment of the present application.

[0011] The high-resolution image transmission method based on optical communication provided by this application can produce at least the following technical effects:

[0012] (1) By dividing the high-resolution image into multiple image blocks and determining the target compression strategy according to the content complexity of each image block, the compression strategy is dynamically adjusted according to the complexity of the image block, which can flexibly cope with the complexity differences of different regions in the image, ensuring that the image details of complex areas are preserved during the compression process, while relatively simple areas are compressed more efficiently. In this way, redundant data is further reduced on the basis of ensuring the key content details, and the overall compression rate is improved, thereby effectively reducing the amount of data during the transmission of high-resolution images.

[0013] (2) At the optical communication receiver, the image is reconstructed through the Conditional Generative Adversarial Network (CGAN), and the conditional vector is defined according to the image coding mode to specify the specific conditions of the generated image. By learning the reconstruction strategy of the specific conditional vector, CGAN can intelligently reconstruct the compressed image blocks, more accurately restore the details of the original image, effectively repair the compression artifacts and detail loss, effectively balance the image compression rate and image quality during the transmission process, and achieve high-quality transmission of high-resolution images.

[0014] Through this embodiment, the combination of the compression strategy based on content complexity and the conditional generative adversarial network makes the image compression transmission process highly adaptable and extensible. Regardless of the complexity of the image content, the compression strategy can be dynamically adjusted and the image can be reconstructed according to the specific situation, thereby ensuring the high-resolution image transmission effect in different types and application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A flowchart showing an example of a high-resolution image transmission method based on optical communication according to an embodiment of the present application;

[0017] Figure 2 An operation flow chart of an example of dividing a high-resolution image into a plurality of image blocks according to an embodiment of the present application is shown;

[0018] Figure 3 An operation flow chart of an example of compressing an image block according to a target compression strategy according to an embodiment of the present application is shown;

[0019] Figure 4 An operation flow chart of an example of compressing an image block by a context-adaptive entropy compression strategy according to an embodiment of the present application is shown;

[0020] Figure 5 A schematic diagram showing a structural connection of an example of a generator of a CGAN according to an embodiment of the present application is shown;

[0021] Figure 6 A structural block diagram of an example of a high-resolution image transmission system based on optical communication according to an embodiment of the present application is shown;

[0022] Figure 7 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0025] Figure 1 A flowchart of an example of a high-resolution image transmission method based on optical communication according to an embodiment of the present application is shown.

[0026] Regarding the executor of the method of the embodiment of the present application, it can be any controller or processor with computing or processing capabilities. By introducing a content complexity matching compression strategy and a conditional generative adversarial network during the compression and transmission of high-resolution images, the quality of image compression and reconstruction is significantly improved, image compression artifacts are reduced, real-time transmission performance is enhanced, and the adaptability and robustness of the system are improved.

[0027] In some examples, it can be integrated into an electronic device or terminal by software, hardware, or a combination of software and hardware, and the type of terminal or electronic device can be diverse, such as an optical communication transmitter, a mobile phone, a tablet computer, or a desktop computer, etc.

[0028] like Figure 1 As shown, in step S110, the high-resolution image to be transmitted is divided into a plurality of image blocks, each of which has a corresponding content complexity.

[0029] In some embodiments, according to the system settings, the high-resolution image is divided into multiple image blocks according to a predetermined resolution or block size, for example, the image is divided into 16×16 or 32×32 blocks, so that each block can be processed independently. After the segmentation is completed, the content complexity analysis is performed on each image block. Here, the content complexity can be calculated by various image complexity perception algorithms, such as parameters such as gradient changes, texture complexity, and color differences of image blocks, and then these indicators are used to quantify the complexity of each image block, thereby forming a complexity vector. Exemplarily, the corresponding content complexity of the image block is extracted by means of edge detection algorithms (such as Canny edge detection) and texture analysis methods (such as Gabor filters). In this way, by dividing the image into small blocks and independently analyzing the content complexity of each block of the image, the differences in local features of the image can be understood in more detail.

[0030] In step S120, for each image block, a target compression strategy matching the content complexity of the image block is determined, and the image block is compressed according to the target compression strategy to obtain a corresponding compressed image block.

[0031] In some implementations, the best compression strategy is dynamically selected based on the content complexity of each image block. For example, for image blocks with higher complexity, a lower compression ratio (high-fidelity compression strategy) is selected to retain more details; for image blocks with lower complexity, a higher compression ratio (high compression rate strategy) is selected to reduce the amount of data. Furthermore, after selecting the target compression strategy, the corresponding compression algorithm (such as JPEG, WebP, HEVC, etc.) is applied to compress each image block to obtain the corresponding compressed image block.

[0032] In this way, by selecting different compression strategies according to the complexity of the content, the compression rate of the image data can be maximized while minimizing the loss of important details. The details of the high-complexity areas are retained during the compression process, which improves the reconstruction quality of the final image and ensures that the transmitted image can retain sufficient visual information.

[0033] In step S130, a corresponding image coding mode is generated according to the target compression strategy of each image block.

[0034] In some embodiments, the coding mode may include information such as the algorithm type, compression ratio, block size, quantization parameter, parameter settings, etc. of the compression strategy, and the generated image coding mode is packaged together with the compressed image block, so that the optical communication receiver can correctly identify the compression strategy and apply the corresponding decoding and reconstruction strategy, thereby effectively improving the reconstruction quality of the image.

[0035] In step S140, each compressed image block and the corresponding image coding mode are transmitted to an optical communication receiver, so that the optical communication receiver processes the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image, and the conditional vector of the conditional generative adversarial network is determined according to the image coding mode.

[0036] Here, all compressed image blocks and image coding modes are transmitted at high speed through optical communication channels. The high bandwidth and low latency characteristics of optical communication ensure fast and stable data transmission. In addition, CGAN contains a generator and a discriminator. The generator tries to generate realistic temperature calibration results, while the discriminator is used to distinguish the generated results from the real results. The discriminator generally only plays a role in the training phase and is not used in the application reasoning phase of the network.

[0037] It should be noted that CGAN is a variant of generative adversarial networks, which introduces additional conditional information when generating data. Compared with ordinary adversarial networks, CGAN allows users to specify specific conditions for the data to be generated, so that CGAN can learn to generate more structured and diverse data under given conditions, aiming to achieve more sophisticated and controllable generation tasks.

[0038] Through this embodiment, during the training process of CGAN, it has learned how to restore image details based on different compression strategies. Therefore, when processing compressed image blocks, conditional vectors are defined according to the image coding mode, and the reconstruction strategy of specific conditional vectors is learned, which can effectively eliminate compression artifacts and reconstruct high-quality images close to the original images.

[0039] Figure 2 An operational flowchart of an example of segmenting a high-resolution image into multiple image blocks according to an embodiment of the present application is shown.

[0040] like Figure 2 As shown, in step S210, the high-resolution image to be transmitted is divided into a plurality of image units according to a preset pixel size.

[0041] In some embodiments, a grid-based segmentation algorithm is used to evenly segment the image according to a preset pixel size, for example, each image unit is a rectangular area, to ensure that the segmented image units have a consistent size. In addition, the pixel size of the image unit can be adjusted according to the needs of the application scenario. For example, in medical images, the unit size can be smaller to retain more details, while in general applications, the unit size can be slightly larger to reduce the amount of calculation.

[0042] In step S220, the image complexity corresponding to each image unit is calculated.

[0043] Here, the corresponding image complexity may be calculated for each image unit.

[0044] More specifically, image complexity is calculated using a multi-scale edge density method:

[0045] Perform Sobel edge detection on image units at multiple scales to obtain the corresponding multi-scale edge intensity matrix:

[0046] , formula (1);

[0047] , formula (2);

[0048] In the formula, is the blur scale, which determines the strength of the Gaussian blur; The application scale is The Gaussian blurred image cell at position The pixel value of represents the radius of the Gaussian kernel; is the weight of the Gaussian kernel in the Gaussian kernel function of scale, indicating that in the Gaussian kernel matrix, it is located at position The weight value at ; is the image unit at position The pixel value of is in scale Next, location The edge strength of yes exist The gradient of the direction is calculated using the horizontal convolution kernel of the Sobel operator; yes exist The gradient of the direction is calculated using the vertical convolution kernel of the Sobel operator.

[0049] Here, the image is first Gaussian blurred to smooth out the noise and details in the image, allowing the image to retain different degrees of detail and edge information at different scales. The larger the scale, the stronger the blur and the more details are lost; the smaller the scale, the more details are retained. In multi-scale edge detection, the scale is changed to obtain different versions of the image. Then, the edge strength of each pixel in the image block is calculated using the Sobel edge detection method to identify areas of drastic changes in the image (i.e., edges). In multi-scale edge detection, images of different blur levels are processed using the Sobel operator, and for each pixel, its horizontal ( x direction) and vertical direction ( y direction) to determine the strength of the edge to capture the edge information in the image.

[0050] According to the multi-scale edge intensity matrix, the edge density corresponding to each fuzzy scale is calculated:

[0051] , formula (3);

[0052] In the formula, Representation scale The edge density under and Respectively represent the pixel length and pixel width of the image unit;

[0053] It should be noted that edge density is the average value of the edge strength in an image block, which represents the overall complexity of the image block at a certain scale. By calculating edge density, the complexity of the image block at different scales can be evaluated. If the edge density at a certain scale is higher, it means that the image block contains more details or texture information at that scale.

[0054] The edge densities of all scales are weighted and fused to obtain the multi-scale edge density corresponding to the image unit:

[0055] , formula (4);

[0056] is the final multi-scale edge density, representing the image complexity of the image unit; is the total number of fuzzy scales, is the weight of each fuzzy scale.

[0057] Here, the edge density at different scales is weighted averaged to obtain the final edge complexity evaluation result. Therefore, by fusing information at multiple scales, the complexity of the image block can be evaluated more comprehensively, taking into account both the global structure at large scales and the detailed features at small scales, thereby more accurately guiding the selection of compression strategies.

[0058] Through this embodiment, the image is first processed by Gaussian blur and smoothing, and then the Sobel operator is used to detect edges at different scales, followed by calculating the edge density, and finally the information at different scales is fused to obtain the comprehensive complexity of the image block, thereby enabling high-resolution images to be efficiently compressed and transmitted while balancing the compression rate and image quality.

[0059] In step S230, each image unit is clustered according to the image complexity to obtain a corresponding plurality of image blocks.

[0060] Here, various non-restrictive clustering algorithms can be used to cluster each image unit, such as K-means clustering, hierarchical clustering or DBSCAN clustering, etc. Taking the K-means clustering algorithm as an example, the cluster center can be initialized by the K-means++ algorithm to improve the convergence speed of clustering and the stability of the results. Then, each image unit is assigned to the corresponding cluster according to its complexity value to form multiple image blocks. Each clustering result corresponds to an image block, and the size and shape of the image block are determined according to the result of the clustering algorithm. Therefore, through the clustering algorithm, the image units are classified according to their complexity, and the image units with similar content are effectively aggregated together to form image blocks, which can better adopt different processing strategies for areas with different content complexities in the subsequent compression and transmission process.

[0061] Through the embodiments of the present application, a multi-scale edge density method is adopted. By performing edge detection on the image at multiple fuzzy scales and fusing the edge densities at different scales, not only the global structure of the image block (such as large-scale contours and shapes) can be captured, but also the local fine details (such as textures and edges) can be accurately identified. The complexity of the image block can be comprehensively evaluated, making the evaluation of image complexity more comprehensive and accurate.

[0062] Figure 3 An operational flowchart of an example of compressing an image block according to a target compression strategy according to an embodiment of the present application is shown.

[0063] like Figure 3 As shown, in step S310, it is detected whether the content complexity of each image block exceeds a preset complexity threshold.

[0064] In some implementations, a complexity threshold may be pre-set according to the requirements of the application scenario and the characteristics of the image. It should be noted that this complexity threshold may be a fixed value or may be adjusted according to historical data or dynamic calculation results, and the setting of the complexity threshold should be able to effectively distinguish between high-complexity and low-complexity image blocks. Subsequently, image blocks that exceed the complexity threshold may be marked as high-complexity image blocks, and image blocks that do not exceed the threshold may be marked as low-complexity image blocks.

[0065] In step S321, when it is detected that the content complexity of the first image block exceeds a preset complexity threshold, it is determined that the target compression strategy matching the first image block is a lossless compression strategy, and the first image block is compressed according to the lossless compression strategy to obtain a corresponding first compressed image block.

[0066] In some embodiments, a suitable lossless compression strategy is selected for high-complexity image blocks. Lossless compression strategies generally include methods such as PNG and JPEG2000, which do not lose any image information during the compression process and can completely retain the details of the original image. In addition, the compression parameters can be adjusted according to the specific characteristics of the image block (such as edge density, texture complexity, etc.) to optimize the compression effect. For example, in JPEG2000, the wavelet transform series and quantization step size can be adjusted to achieve higher compression efficiency.

[0067] In step S323, when it is detected that the content complexity of the second image block does not exceed the complexity threshold, the target compression strategy matching the second image block is determined to be a lossy compression strategy, and the second image block is compressed according to the lossy compression strategy to obtain a corresponding second compressed image block.

[0068] In some embodiments, for low-complexity image blocks, an appropriate lossy compression strategy is selected. Lossy compression strategies include JPEG, HEVC, etc. These methods discard unimportant image information to a certain extent to significantly reduce the amount of data. In addition, according to the complexity and content characteristics of the image block, the parameters of the lossy compression strategy are adjusted. For example, in JPEG compression, the compression quality factor can be adjusted to minimize the amount of data while ensuring the visual effect of the image.

[0069] In some examples of the embodiments of the present application, the lossy compression strategy is a discrete cosine transform (DCT) compression strategy, and the image coding mode corresponding to the second image block includes a position identifier of the second image block, a quantization matrix, and a quantized DCT coefficient matrix. Here, the position of the image block in the entire image is uniquely identified by the identifier of the image block, ensuring that each image block can be correctly spliced ​​during decoding to restore the complete image. The quantization matrix can use a standard quantization matrix, which is used to quantize the coefficients after the DCT transformation, and the DCT coefficients can be inversely quantized at the receiving end to help restore the image block close to the original image. The quantized DCT coefficients are the main data after compression, and the receiver uses these coefficients to reconstruct the image block through the inverse DCT transformation to achieve reverse efficient decoding. Thereby, it is ensured that the receiving end can correctly decode and reconstruct the image while keeping the compression process simple and effective.

[0070] Through this embodiment, the application of intelligent and differentiated compression strategies for high-resolution images is realized. A lossless compression strategy is used in areas with higher complexity to ensure the retention of key details; a lossy compression strategy is used in areas with lower complexity to effectively reduce the amount of data. As a result, not only the image quality and transmission efficiency are optimized, but also the needs of different application scenarios can be better adapted, providing strong support for the efficient transmission and storage of high-resolution images.

[0071] In some examples of the embodiments of the present application, the lossless compression strategy is a context-adaptive entropy compression strategy.

[0072] Figure 4 An operational flow chart of an example of compressing an image block using a context-adaptive entropy compression strategy according to an embodiment of the present application is shown.

[0073] like Figure 4 As shown, in step S410, a context range that matches the image complexity of the first image block is determined, and there is a positive correlation between the size of the context range and the image complexity.

[0074] Here, an adaptive context modeling mechanism is adopted. Specifically, the system dynamically adjusts the scope and weight of the context according to the actual content characteristics of the image block (i.e., image complexity). A wider range of context is used for modeling in areas with high complexity, while the context range is narrowed in smooth areas to reduce computational complexity.

[0075] In step S420, for each target pixel in the first image block, neighboring pixels of the target pixel within the context range are extracted, and a weighted average value of the neighboring pixels is calculated as a predicted pixel value of the target pixel.

[0076] More specifically, the predicted value of each pixel is calculated by the following formula:

[0077] , formula (5);

[0078] In the formula, Indicates location The predicted value of the pixel at Represents the first image block Middle position The pixel value at ; Indicates the position in the context scope The context pixel weight of the pixel at reflects the contribution of the pixel to the prediction; Indicates location The set of all pixels in the context range of the pixel at .

[0079] Here, the pixel values ​​within the context range are weighted averaged to calculate the current pixel The predicted value of , each context pixel The influence of The larger the weight, the greater the impact on the current pixel prediction value. It can be adaptively adjusted according to the spatial distance or content correlation of neighboring pixels to improve the accuracy of prediction.

[0080] In step S430, the prediction error between the actual pixel value and the predicted pixel value of each target pixel is calculated.

[0081] More specifically, the prediction error is calculated by the following formula:

[0082] , formula (6);

[0083] In the formula, Indicates location The prediction error of the pixel at Indicates the position in the first image block The actual value of the pixel at ; Indicates location The predicted value of the pixel at .

[0084] Here, the actual pixel value is calculated and predicted values The difference between , which represents the deviation between the predicted value and the actual value when predicting the current pixel through the context.

[0085] In step S440, entropy coding is performed on each target pixel in the first image block based on the probability distribution of the prediction error.

[0086] , formula (7);

[0087] , formula (8);

[0088] In the formula, It's location The entropy encoding value of the pixel at express The probability distribution of Represents an image block The entropy coding value of represents the first compressed image block; Represents the entropy coding length calculated according to the probability distribution.

[0089] Here, the prediction error is calculated The entropy encoding value of , entropy coding is based on the probability distribution of errors , the higher the probability of error occurrence, the shorter the corresponding coding length, thereby reducing data redundancy.

[0090] It should be noted that although For a single pixel location But the pixel information within the context Used to predict the pixel value and indirectly affects the probability distribution of the prediction error Therefore, although entropy coding is performed on a single pixel, the coding efficiency is closely related to the information of the entire context area.

[0091] In the embodiment of the present application, a larger context range is used to capture more neighboring pixel information for a highly complex image block, thereby more accurately predicting the current pixel. Then, the pixel values ​​within the context range are extracted, and the weighted average of these pixels is used as the prediction of the current pixel. By calculating the error between the actual pixel value and the predicted value, entropy coding is finally performed based on the probability distribution of the prediction error. Thus, the shortest code can be generated according to the frequency of occurrence of the error, thereby achieving efficient lossless compression.

[0092] Through the embodiments of the present application, context adaptive entropy coding is used to dynamically adjust the context range, and each pixel is predicted and encoded according to the complexity of the image block. Since the prediction error is small and concentrated, entropy coding can generate shorter codes, thereby significantly improving the compression rate and effectively reducing the amount of data transmitted. Compared with traditional lossless compression methods, context adaptive entropy coding can more effectively capture and utilize the spatial correlation within the image block, thereby reducing data redundancy. In addition, context adaptive entropy coding maintains the characteristics of lossless compression, ensuring that no image information is lost during the compression and decompression process. For highly complex image blocks (such as edges, textures and other areas with rich details), all original details can be retained, the high quality of the image can be guaranteed, and common problems such as compression artifacts can be avoided.

[0093] In some examples of the embodiments of the present application, the image coding mode for transmission along with a high-complexity compressed data packet further includes a position identifier of the first image block, a context range, a probability distribution of a prediction error, and a context pixel weight.

[0094] Here, the position identifier of the image block is used to uniquely identify the position of the image block in the entire image. Since the image is divided into multiple blocks for processing during compression, each image block can be accurately located, and the identifier ensures that each image block can be correctly spliced ​​back to the original image during decoding. The context range information describes the range size of the neighboring pixels used for prediction of each pixel during the compression process, which determines the number and distribution of neighboring pixels referenced during the prediction process, so that the receiving end can accurately reproduce the prediction process during encoding during decoding, thereby accurately calculating the original value of each pixel. The probability distribution of the prediction error describes the statistical distribution of the prediction error in the image block, which is used to guide the entropy coding process, so that common error values ​​are assigned a shorter coding length, and rare error values ​​are assigned a longer coding length, which can improve the decoding efficiency and compression ratio, so that the decoding process can more efficiently restore the original pixel value while maintaining the characteristics of lossless compression. The context pixel weight describes the influence of each context pixel on the current pixel during the prediction process. The larger the weight, the greater the contribution of the context pixel to the prediction value, thereby ensuring that the context prediction during encoding can be accurately reproduced during the decoding process, and ensuring that the decoded image block is completely consistent with the original image block.

[0095] Figure 5 A schematic diagram of the structural connection of an example of a generator of a CGAN according to an embodiment of the present application is shown.

[0096] like Figure 5 As shown, the generator 500 of the conditional generative adversarial network includes an image decoding layer 510, a conditional feature embedding module 520, a conditional feature fusion module 530, a multi-scale feature extraction module 540 and an output layer 550.

[0097] The image decoding layer 510 is used to receive each compressed image block and a corresponding image coding mode, and combine each compressed image block to obtain a corresponding image block feature matrix.

[0098] In some embodiments, the image decoding layer first decodes the compressed image blocks, and then recombine the decoded image blocks according to the image coding mode to form an image block feature matrix with spatial structure information, which retains the compressed image information and spatial structure.

[0099] The conditional feature embedding module 520 is used to embed each image coding mode into a high-dimensional feature space to determine a corresponding conditional feature matrix.

[0100] In some embodiments, the encoding pattern is taken as input and passed through multiple fully connected layers or other embedding operations to ultimately generate a conditional feature matrix having the same spatial dimension as the image block feature matrix, and this matrix is ​​aligned with the image block feature matrix.

[0101] The conditional feature fusion module 530 is used to fuse the image block feature matrix and the conditional feature matrix to obtain a corresponding target fused feature matrix.

[0102] Here, the conditional feature fusion module enables the generator to better utilize the content information and encoding mode of the image by fusing information from different sources. Specifically, through fusion methods such as bilinear pooling, the conditional feature fusion module deeply interacts the two input matrices to generate a more expressive fused feature matrix.

[0103] The multi-scale feature extraction module 540 is used to extract multi-scale convolution features corresponding to the target fusion feature matrix in parallel through multiple convolution kernels, and each convolution kernel has a corresponding convolution scale.

[0104] Here, the generator is able to capture the global structure and local details of the image through the multi-scale feature extraction module. As a result, multiple convolution kernels of different scales are simultaneously applied to the target fusion feature matrix, and each convolution kernel extracts the corresponding scale feature map, capturing the feature information of the image at different resolutions and retaining different levels of information of the image.

[0105] The output layer 550 is used to process the multi-scale convolution features through a deconvolution operation to generate a reconstructed high-resolution image.

[0106] Specifically, through the inverse convolution operation, the previously extracted multi-scale convolution features are gradually restored to the resolution of the original image, and finally a reconstructed high-resolution image is output, ensuring the details and quality of the reconstructed image and restoring the original input image to the greatest extent.

[0107] Through the embodiments of the present application, the generator can better retain the spatial structure information of the image through the preliminary decoding and combination of the image decoding layer. Combined with the conditional feature embedding and fusion modules, the generator can make full use of the content information and encoding mode of the image, making the reconstructed high-resolution image closer to the original image in details and global structure.

[0108] In some examples of the embodiments of the present application, the conditional feature fusion module 530 adopts a multimodal feature fusion mechanism. Multimodal feature fusion can deeply explore the relationship between compressed image blocks and conditional feature vectors. By introducing a multimodal fusion network (such as a bilinear pooling layer), the interaction between different information sources is enriched, which not only improves the accuracy of feature expression, but also significantly improves the generator's ability to restore complex image content during the reconstruction process, especially when processing image data with high diversity and complexity.

[0109] Specifically, the conditional feature fusion module 530 may determine the fusion feature in the following manner:

[0110] Based on the bilinear pooling method, the conditional feature matrix With the image block feature matrix Fusion is performed to obtain the initial fusion feature matrix :

[0111] In some embodiments, the image patch feature matrix is output by the image decoding layer and has the shape ,in and are the height and width of the image block, respectively. is the number of feature channels. Conditional feature matrix is generated by the conditional feature embedding module and has the shape of ,in and Also for the height and width of the image block, is the number of channels of the conditional features.

[0112] Then, the image block feature matrix is ​​fused with the conditional feature matrix by using the bilinear pooling method:

[0113] , formula (9);

[0114] In the formula, Represents the initial fusion feature matrix In Location The eigenvalue at Represents the image block feature matrix at position and Channel The eigenvalue at ; Represents the expanded conditional feature matrix In Location and Channel The eigenvalue at ; yes The total number of channels, yes The total number of channels.

[0115] It should be noted that bilinear pooling can capture the high-order interaction information between the two matrices, thereby generating a more expressive initial fusion feature matrix, enabling the generator to better restore image details and structures when generating high-resolution images.

[0116] For the initial fusion feature matrix Perform standardization to obtain the target fusion feature matrix :

[0117] , formula (10);

[0118] In the formula, Represents the target fusion feature matrix In Location The eigenvalue at Indicates a preset constant; Represents the sum of squares of all elements in the entire initial fusion feature matrix, which is used to calculate the normalization factor; and They respectively represent the horizontal spatial dimension index and the vertical spatial dimension index of the initial fusion feature matrix.

[0119] In this way, by normalizing the initial fusion feature matrix, the values ​​of the fusion features are distributed within an appropriate range, ensuring the stability of the fusion features in subsequent processing, facilitating further feature extraction and image generation, and improving the overall computational efficiency of the generator.

[0120] Through the embodiments of the present application, the multimodal feature fusion mechanism realizes the deep interaction between the compressed image block features and the conditional features through advanced feature fusion methods such as bilinear pooling, captures the complex relationship between the image content and the encoding conditions, and enables the generator to more accurately reproduce the details and structure of the high-resolution original image. In addition, through the feature fusion mechanism, the information in the conditional vector is fully utilized, and the generator can more accurately restore the edges and details of the image during the image reconstruction process, effectively reducing artifacts and detail loss, making the output image clearer and more natural.

[0121] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0122] Figure 6 A structural block diagram of an example of a high-resolution image transmission system based on optical communication according to an embodiment of the present application is shown. The system can be deployed in an optical communication transmitter of an optical communication network.

[0123] like Figure 6 As shown, the high-resolution image transmission system 600 based on optical communication includes an image segmentation unit 610 , a strategy matching unit 620 , a coding mode generation unit 630 and a data transmission unit 640 .

[0124] The image segmentation unit 610 is used to segment the high-resolution image to be transmitted into a plurality of image blocks; each of the image blocks has a corresponding content complexity.

[0125] The strategy matching unit 620 is used to determine, for each of the image blocks, a target compression strategy that matches the content complexity of the image block, and compress the image block according to the target compression strategy to obtain a corresponding compressed image block.

[0126] The coding mode generating unit 630 is used to generate a corresponding image coding mode according to the target compression strategy of each image block.

[0127] The data transmission unit 640 is used to transmit each of the compressed image blocks and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; the conditional vector of the conditional generative adversarial network is determined according to the image coding mode.

[0128] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of any of the above-mentioned high-resolution image transmission methods based on optical communication in the present application.

[0129] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs any step of the above-mentioned high-resolution image transmission method based on optical communication.

[0130] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of a high-resolution image transmission method based on optical communication.

[0131] Figure 7 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for performing a high-resolution image transmission method based on optical communication provided by another embodiment of the present application. Figure 7 As shown, the device includes:

[0132] One or more processors 710 and memory 720, Figure 7 A processor 710 is taken as an example.

[0133] The apparatus for performing the high-resolution image transmission method based on optical communication may further include: an input device 730 and an output device 740 .

[0134] The processor 710, the memory 720, the input device 730 and the output device 740 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0135] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the high-resolution image transmission method based on optical communication in the embodiment of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, realizing the high-resolution image transmission method based on optical communication in the above method embodiment.

[0136] The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The input device 730 may receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.

[0138] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, perform the high-resolution image transmission method based on optical communication in any of the above method embodiments.

[0139] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0140] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0141] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0142] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0143] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0144] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0145] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A high-resolution image transmission method based on optical communication, applied to an optical communication transmitter, the method comprising: Dividing a high-resolution image to be transmitted into a plurality of image blocks; Each of the image blocks has a corresponding content complexity; For each of the image blocks, determining a target compression strategy that matches the content complexity of the image block, and compressing the image block according to the target compression strategy to obtain a corresponding compressed image block; Generating a corresponding image coding mode according to the target compression strategy of each of the image blocks; Transmitting each of the compressed image blocks and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes each of the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; The conditional vector of the conditional generative adversarial network is determined according to the image coding mode; The step of dividing the high-resolution image to be transmitted into a plurality of image blocks comprises: Dividing the high-resolution image to be transmitted into a plurality of image units according to a preset pixel size; Calculating the image complexity corresponding to each of the image units; Clustering each of the image units according to the image complexity to obtain a corresponding plurality of image blocks; Among them, the image complexity is calculated by the multi-scale edge density method: Perform Sobel edge detection on image units at multiple scales to obtain the corresponding multi-scale edge intensity matrix: In the formula, is the blur scale, which determines the strength of the Gaussian blur; The application scale is The Gaussian blurred image cell at position The pixel value of represents the radius of the Gaussian kernel; The scale is The weight of the Gaussian kernel in the Gaussian kernel function indicates that the position in the Gaussian kernel matrix is The weight value at ; is the image unit at position The pixel value of is in scale Next, location The edge strength of yes exist The gradient of the direction is calculated using the horizontal convolution kernel of the Sobel operator; is The gradient of the direction is calculated using the vertical convolution kernel of the Sobel operator; According to the multi-scale edge intensity matrix, the edge density corresponding to each fuzzy scale is calculated: In the formula, Representation scale The edge density under and Respectively represent the pixel length and pixel width of the image unit; The edge densities of all scales are weighted and fused to obtain the multi-scale edge density corresponding to the image unit: is the final multi-scale edge density, representing the image complexity of the image unit; is the total number of fuzzy scales, is the weight of each fuzzy scale; The determining of a target compression strategy that matches the content complexity of the image block, and compressing the image block according to the target compression strategy to obtain a corresponding compressed image block, includes: In the case where it is detected that the content complexity of the first image block exceeds a preset complexity threshold, determining that a target compression strategy matching the first image block is a lossless compression strategy, and compressing the first image block according to the lossless compression strategy to obtain a corresponding first compressed image block; In the case where it is detected that the content complexity of the second image block does not exceed the complexity threshold, determining that the target compression strategy matching the second image block is a lossy compression strategy, and compressing the second image block according to the lossy compression strategy to obtain a corresponding second compressed image block; The lossless compression strategy is a context-adaptive entropy compression strategy; The compressing the first image block according to the lossless compression strategy to obtain a corresponding first compressed image block includes: Determining a context range that matches the image complexity of the first image block; the size of the context range is positively correlated with the image complexity; For each target pixel in the first image block, neighboring pixels of the target pixel within the context range are extracted, and a weighted average value of the neighboring pixels is calculated as a predicted pixel value of the target pixel: In the formula, Indicates location The predicted value of the pixel at Represents the first image block Middle position The pixel value at ; Indicates the position in the context scope The context pixel weight of the pixel at reflects the contribution of the pixel to the prediction; Indicates location The set of all pixels in the context range of the pixel at ; Calculate the prediction error between the actual pixel value and the predicted pixel value for each target pixel: In the formula, Indicates location The prediction error of the pixel at Indicates the position in the first image block The actual value of the pixel at ; Indicates location The predicted value of the pixel at ; Based on the probability distribution of the prediction error, entropy coding is performed on each target pixel in the first image block: In the formula, It's location The entropy encoding value of the pixel at express The probability distribution of Represents an image block The entropy coding value of represents the first compressed image block; Represents the entropy coding length calculated according to the probability distribution.

2. The method according to claim 1, wherein: The image coding mode corresponding to the first image block includes a location identifier, a context range, a probability distribution of a prediction error, and a context pixel weight of the first image block.

3. The method according to claim 1, wherein: The lossy compression strategy is a discrete cosine transform compression strategy, and the image coding mode corresponding to the second image block includes a position identifier of the second image block, a quantization matrix, and a quantized discrete cosine transform coefficient matrix.

4. The method according to any one of claims 1 to 3, wherein: The generator of the conditional generative adversarial network includes an image decoding layer, a conditional feature embedding module, a conditional feature fusion module, a multi-scale feature extraction module and an output layer; The image decoding layer is used to receive each compressed image block and a corresponding image coding mode, and combine each compressed image block to obtain a corresponding image block feature matrix; The conditional feature embedding module is used to embed each of the image coding modes into a high-dimensional feature space to determine a corresponding conditional feature matrix; The conditional feature fusion module is used to fuse the image block feature matrix and the conditional feature matrix to obtain a corresponding target fusion feature matrix; The multi-scale feature extraction module is used to extract the multi-scale convolution features corresponding to the target fusion feature matrix in parallel through multiple convolution kernels; Each of the convolution kernels has a corresponding convolution scale; The output layer is used to process the multi-scale convolution features through a deconvolution operation to generate a reconstructed high-resolution image.

5. The method according to claim 4, wherein: The conditional feature fusion module adopts a multimodal feature fusion mechanism and is used to determine the fusion features in the following manner: Based on the bilinear pooling method, the conditional feature matrix With the image block feature matrix Fusion is performed to obtain the initial fusion feature matrix : In the formula, Represents the initial fusion feature matrix In Location The eigenvalue at Represents the image block feature matrix at position and Channel The eigenvalue at ; Represents the expanded conditional feature matrix In Location and Channel The eigenvalue at ; yes The total number of channels, yes The total number of channels; The initial fusion feature matrix is ​​normalized to obtain the target fusion feature matrix : In the formula, Represents the target fusion feature matrix In Location The eigenvalue at Indicates a preset constant; Represents the sum of squares of all elements in the entire initial fusion feature matrix, which is used to calculate the normalization factor; and They respectively represent the horizontal spatial dimension index and the vertical spatial dimension index of the initial fusion feature matrix.

6. A high-resolution image transmission system based on optical communication, deployed in an optical communication transmitter, the system comprising: An image segmentation unit, used for segmenting a high-resolution image to be transmitted into a plurality of image blocks; Each of the image blocks has a corresponding content complexity; a strategy matching unit, configured to determine, for each of the image blocks, a target compression strategy that matches the content complexity of the image block, and compress the image block according to the target compression strategy to obtain a corresponding compressed image block; A coding mode generating unit, configured to generate a corresponding image coding mode according to the target compression strategy of each of the image blocks; A data transmission unit, configured to transmit each of the compressed image blocks and the corresponding image coding mode to an optical communication receiver, so that the optical communication receiver processes each of the received compressed image blocks and the corresponding image coding mode by calling a conditional generative adversarial network to generate a reconstructed high-resolution image; The conditional vector of the conditional generative adversarial network is determined according to the image coding mode; The step of dividing the high-resolution image to be transmitted into a plurality of image blocks comprises: Dividing the high-resolution image to be transmitted into a plurality of image units according to a preset pixel size; Calculating the image complexity corresponding to each of the image units; Clustering each of the image units according to the image complexity to obtain a corresponding plurality of image blocks; Among them, the image complexity is calculated by the multi-scale edge density method: Perform Sobel edge detection on image units at multiple scales to obtain the corresponding multi-scale edge intensity matrix: In the formula, is the blur scale, which determines the strength of the Gaussian blur; The application scale is The Gaussian blurred image cell at position The pixel value of represents the radius of the Gaussian kernel; The scale is The weight of the Gaussian kernel in the Gaussian kernel function indicates that the position in the Gaussian kernel matrix is The weight value at ; is the image unit at position The pixel value of is in scale Next, location The edge strength of yes exist The gradient of the direction is calculated using the horizontal convolution kernel of the Sobel operator; is The gradient of the direction is calculated using the vertical convolution kernel of the Sobel operator; According to the multi-scale edge intensity matrix, the edge density corresponding to each fuzzy scale is calculated: In the formula, Representation scale The edge density under and Respectively represent the pixel length and pixel width of the image unit; The edge densities of all scales are weighted and fused to obtain the multi-scale edge density corresponding to the image unit: is the final multi-scale edge density, representing the image complexity of the image unit; is the total number of fuzzy scales, is the weight of each fuzzy scale; The determining of a target compression strategy that matches the content complexity of the image block, and compressing the image block according to the target compression strategy to obtain a corresponding compressed image block, includes: In the case where it is detected that the content complexity of the first image block exceeds a preset complexity threshold, determining that a target compression strategy matching the first image block is a lossless compression strategy, and compressing the first image block according to the lossless compression strategy to obtain a corresponding first compressed image block; In the case where it is detected that the content complexity of the second image block does not exceed the complexity threshold, determining that the target compression strategy matching the second image block is a lossy compression strategy, and compressing the second image block according to the lossy compression strategy to obtain a corresponding second compressed image block; The lossless compression strategy is a context-adaptive entropy compression strategy; The compressing the first image block according to the lossless compression strategy to obtain a corresponding first compressed image block includes: Determining a context range that matches the image complexity of the first image block; the size of the context range is positively correlated with the image complexity; For each target pixel in the first image block, neighboring pixels of the target pixel within the context range are extracted, and a weighted average value of the neighboring pixels is calculated as a predicted pixel value of the target pixel: In the formula, Indicates location The predicted value of the pixel at Represents the first image block Middle position The pixel value at ; Indicates the position in the context scope The context pixel weight of the pixel at reflects the contribution of the pixel to the prediction; Indicates location The set of all pixels in the context range of the pixel at ; Calculate the prediction error between the actual pixel value and the predicted pixel value for each target pixel: In the formula, Indicates location The prediction error of the pixel at Indicates the position in the first image block The actual value of the pixel at ; Indicates location The predicted value of the pixel at ; Based on the probability distribution of the prediction error, entropy coding is performed on each target pixel in the first image block: In the formula, It's location The entropy encoding value of the pixel at express The probability distribution of Represents an image block The entropy coding value of represents the first compressed image block; Represents the entropy coding length calculated according to the probability distribution.

Images