Binocular Mars image quality enhancement method, device, equipment, medium and product

Abstract

The invention relates to the technical field of computer vision, in particular to a binocular Mars image quality enhancement method and device, equipment, a medium and a product. Comprising the steps that a binocular Mars image pair is acquired, and the binocular Mars image pair comprises a left-eye compressed Mars image and a right-eye compressed Mars image; inputting the binocular Mars image pair into a pre-training image quality enhancement model to obtain an enhanced image pair output by the pre-training image quality enhancement model, the enhanced image pair comprising a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein the pre-trained image quality enhancement model is obtained by training and optimizing a pre-established binocular Mars image data set, and the pre-trained image quality enhancement model comprises a feature extraction component, an attention-based cross-view network and an image reconstruction component.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular, to a method, device, equipment, medium and product for enhancing the quality of binocular Mars images. Background Art

[0002] Driven by the rapid development of Mars exploration, rovers such as Perseverance and Zhurong have successfully landed on Mars and captured precious images of the Martian surface. These images provide valuable data for scientific research. However, the Mars images transmitted back to Earth usually undergo lossy compression, which inevitably introduces compression artifacts and reduces the image quality.

[0003] Currently, one solution for enhancing the quality of Mars images is to utilize the semantic similarity between Mars images for quality enhancement, and the other is to directly use the method for enhancing the quality of Earth images to enhance Mars images. However, both of these methods are monocular methods that only rely on single-view images. Mars rovers are equipped with stereo cameras to capture binocular images. Enhancing binocular images through monocular image quality enhancement methods has problems such as insufficient information utilization, improper handling of left and right view differences, complex calculations, and low efficiency. Therefore, there is an urgent need for a solution for enhancing the quality of binocular Mars images. Summary of the Invention

[0004] To solve the above technical problems or at least partially solve the above technical problems, this application provides a method, device, equipment, medium and product for enhancing the quality of binocular Mars images, which can improve the performance of the binocular Mars image quality enhancement task and improve the efficiency and effect when enhancing the quality of binocular Mars images.

[0005] To achieve the above object, the technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, this application provides a method for enhancing the quality of binocular Mars images, including: obtaining a pair of binocular Mars images, the pair of binocular Mars images including a left-eye compressed Mars image and a right-eye compressed Mars image; inputting the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, the enhanced image pair including a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0007] As an alternative implementation provided in the embodiments of the present application, inputting a binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model includes: inputting a left-eye compressed Mars image and a right-eye compressed Mars image into a feature extraction component to obtain a left-eye feature and a right-eye feature output by the feature extraction component; inputting the left-eye feature and the right-eye feature into an attention-based cross-view network to obtain a left-eye enhanced feature and a right-eye enhanced feature output by the attention-based cross-view network; and inputting the left-eye enhanced feature and the right-eye enhanced feature into an image reconstruction component to obtain a left-eye Mars enhanced image and a right-eye Mars enhanced image output by the image reconstruction component.

[0008] As an alternative implementation provided in the embodiments of the present application, the attention-based cross-view network includes a first block-level attention module, a pixel-level attention module, and a second block-level attention module; inputting the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain a left-eye enhanced feature and a right-eye enhanced feature output by the attention-based cross-view network includes: inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain a first left-eye enhanced feature and a first right-eye enhanced feature output by the first block-level attention module; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain a left-eye matching feature and a right-eye matching feature output by the pixel-level attention module; and inputting the left-eye matching feature and the right-eye matching feature into the second block-level attention module to obtain a second left-eye enhanced feature and a second right-eye enhanced feature output by the second block-level attention module.

[0009] As an alternative implementation provided in the embodiments of the present application, the first block-level attention module includes a first intra-view block attention unit, an inter-view block attention unit, and a second intra-view block attention unit; inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain a first left-eye enhanced feature and a first right-eye enhanced feature output by the first block-level attention module includes: inputting the left-eye feature and the right-eye feature into the first intra-view block attention unit to obtain an intra-view block enhanced feature output by the first intra-view block attention unit; inputting the intra-view block enhanced feature into the inter-view block attention unit to obtain an inter-view block enhanced feature output by the inter-view block attention unit; and inputting the inter-view block enhanced feature into the second intra-view block attention unit to obtain a first left-eye enhanced feature and a first right-eye enhanced feature output by the second intra-view block attention unit.

[0010] As an alternative implementation provided in the embodiments of the present application, inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module includes: splicing the first left-eye enhanced feature with the left-eye feature to obtain a left-eye spliced feature, and splicing the first right-eye enhanced feature with the right-eye feature to obtain a right-eye spliced feature; respectively passing the left-eye spliced feature and the right-eye spliced feature through a channel attention layer, a convolutional layer, and a logical function layer, and then inputting them into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

[0011] As an alternative implementation provided in the embodiments of the present application, the pixel-level attention module includes an inter-view pixel attention unit and an intra-view pixel attention unit; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module includes: inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the inter-view pixel attention unit to obtain the inter-view pixel attention output by the inter-view pixel attention unit; inputting the inter-view pixel attention into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

[0012] In a second aspect, the present application provides a binocular Mars image quality enhancement device, which includes:

[0013] An acquisition module, configured to acquire a pair of binocular Mars images, where the pair of binocular Mars images includes a left-eye compressed Mars image and a right-eye compressed Mars image;

[0014] A quality enhancement module, configured to input the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, where the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image;

[0015] Wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0016] In a third aspect, the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, where when the computer program is executed by the processor, it implements the binocular Mars image quality enhancement method as described in the first aspect or any of its alternative implementations.

[0017] Fourthly, the present application provides a computer-readable storage medium, including: a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the method for enhancing the quality of binocular Mars images as described in the first aspect or any optional implementation manner thereof.

[0018] Fifthly, the present application provides a computer program product, including: the computer program product includes a computer program, and when the computer program runs on a computer, it enables the computer to implement the method for enhancing the quality of binocular Mars images as described in the first aspect or any optional implementation manner thereof.

[0019] The technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art:

[0020] The embodiments of the present disclosure provide a method, device, equipment, medium and product for enhancing the quality of binocular Mars images. The method first obtains a pair of binocular Mars images, including a left-eye compressed Mars image and a right-eye compressed Mars image; inputs the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network and an image reconstruction component. In this way, the present application improves the performance of the binocular Mars image quality enhancement task by mining and utilizing the information of each of the left and right views in the pair of binocular Mars images and the relationship between them. Make full use of the characteristics of binocular Mars images to improve the efficiency and effect when enhancing the quality of binocular Mars images. Description of the Drawings

[0021] The drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic flowchart of a method for enhancing the quality of binocular Mars images provided by an embodiment of the present application;

[0024] Figure 2 It is a schematic structural diagram of a pre-trained image quality enhancement model provided by an embodiment of the present application;

[0025] Figure 3 Schematic diagrams of four different landforms in the binocular Mars image dataset provided by the embodiments of the present application;

[0026] Figure 4 Schematic diagrams of the structures of the block-level attention module and the pixel-level attention module provided by the embodiments of the present application;

[0027] Figure 5 Schematic diagram of the structure of a quality enhancement device for binocular Mars images provided by the embodiments of the present application;

[0028] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the technical terms used in the description of the embodiments or the prior art:

[0030] The Flickr1024 dataset contains Earth images and is a large-scale dataset for stereoscopic image super-resolution research.

[0031] The correlation coefficient (CC) is a statistic for measuring the linear correlation degree between two variables, and the common one is the Pearson correlation coefficient.

[0032] Mutual Information (MI) is a concept in information theory and is used to measure the mutual dependence degree between two random variables.

[0033] The inventor compared the binocular Mars image dataset with the Flickr1024 dataset, and then calculated the correlation coefficient and mutual information. Specifically, for the blocks of the same Mars image, they were paired to calculate the similarity within the view; for the left-eye image and the right-eye image, they were paired to calculate the similarity between the views. The results showed that the CC results of the Mars images were significantly higher than those of the Earth images, indicating that the within-view and between-view correlations of the Mars images were stronger than those of the Earth images. In addition, the within-view MI results of the Mars images were 21.66% higher than those of the Earth images, and the between-view MI results were 20.61% higher than those of the Earth images, indicating that the information gain of the Mars images was greater in both cases. These results highlight the importance of the binocular quality enhancement paradigm for Mars images, as it uses complementary stereoscopic information to improve the image quality. Therefore, the inventor utilized the within-view and between-view correlations of the Mars images to propose a quality enhancement method for binocular Mars images.

[0034] In order to more clearly understand the above-mentioned objects, features, and advantages of the present application, the solution of the present application will be further described below. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.

[0035] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present application, rather than all the embodiments.

[0036] A method for enhancing the quality of binocular Mars images provided in an embodiment of the present application can be implemented through a binocular Mars image quality enhancement device or an electronic device. The electronic device includes, but is not limited to, a personal computer, a laptop computer, a tablet computer, a smart phone, etc. The operating system of the electronic device may include Android, iOS developed by Apple Inc., Windows developed by Microsoft Corporation in the United States, etc., and the embodiments of the present application do not limit this. The electronic device can run alone to implement the present application, or can be connected to the network and implement the present application through interactive operations with other computer devices in the network. Among them, the network where the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN) network, etc.

[0037] It should be noted that the protection scope of a method for enhancing the quality of binocular Mars images described in an embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. Any solution achieved by adding or subtracting steps of the prior art and replacing steps according to the principle of the present application is included in the protection scope of the present application.

[0038] As Figure 1 shown, Figure 2 is a schematic flow chart of a method for enhancing the quality of binocular Mars images according to an embodiment of the present application. This method can be executed by a binocular Mars image quality enhancement device, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. This method mainly includes the following steps S101 to S102:

[0039] S101. Obtain a pair of binocular Mars images.

[0040] The pair of binocular Mars images includes a left compressed Mars image (Compressed left image, I L ) and a right compressed Mars image (Compressed right image, I R ).

[0041] S102. Input the binocular Mars image pair into the pre-trained image quality enhancement model to obtain the enhanced image pair output by the pre-trained image quality enhancement model.

[0042] The enhanced image pair includes an enhanced left Mars image (Enhanced left image, ) and an enhanced right Mars image (Enhanced right image, ).

[0043] Among them, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset. The pre-trained image quality enhancement model includes a feature extraction component (Feature extraction), a bi-level cross-view attention network, and an image reconstruction component (Image reconstruction).

[0044] As Figure 2 shown, the feature extraction component includes: a convolutional layer (Convolutional Layer, Conv.) and residual dense blocks (Residual Dense Blocks, RDBs). The bi-level cross-view attention network includes: a first patch-level attention module, a pixel-level attention module, and a second patch-level attention module. The image reconstruction component includes residual dense blocks and a convolutional layer.

[0045] Among them, the RDBs include two residual dense blocks (RDB), a concatenation layer (Concat), and Conv..

[0046] In some embodiments, before performing step S101 (obtaining the binocular Mars image pair), it further includes constructing a binocular Mars dataset, and then training an image quality enhancement model based on this binocular Mars image dataset.

[0047] Specifically, construct a binocular Mars image dataset for the quality enhancement task. This dataset includes 1,350 pairs of binocular Mars images taken by the Mars rover imaging system. These image pairs are all binocular, with a resolution of 1152×1600, and the image quality is very high or even lossless, without obvious artifacts. Exemplarily, the Mars rover imaging system can be Mastcam-Z (an imaging system installed on the Perseverance Mars rover, including a pair of RGB cameras).

[0048] The binocular Mars image dataset covers four main Mars landforms: Rock, Soil, Sand, and Sky, as Figure 3as shown

[0049] The binocular Mars image dataset is compressed using JPEG to simulate the actual image transmission process from Mars to Earth, thereby generating compressed binocular Mars images. The compressed binocular Mars images are used to train an image quality enhancement model, and the model parameters are adjusted until convergence to obtain a pre-trained image quality enhancement model.

[0050] Apply the pre-trained image quality enhancement model and perform step S102 (input the binocular Mars image pair into the pre-trained image quality enhancement model to obtain the enhanced image pair output by the pre-trained image quality enhancement model). In some embodiments, inputting the binocular Mars image pair into the pre-trained image quality enhancement model to obtain the enhanced image pair output by the pre-trained image quality enhancement model includes: inputting the left-eye compressed Mars image and the right-eye compressed Mars image into the feature extraction component to obtain the left-eye feature and the right-eye feature output by the feature extraction component; inputting the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network; inputting the left-eye enhanced feature and the right-eye enhanced feature into the image reconstruction component to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image output by the image reconstruction component.

[0051] As Figure 2 shown, input the left-eye compressed Mars image I L into the feature extraction component to obtain the left-eye feature F L output by the feature extraction component. Input the right-eye compressed Mars image I R into the feature extraction component to obtain the right-eye feature F R output by the feature extraction component. Then input the left-eye feature F L into the attention-based cross-view network to obtain the left-eye enhanced feature output by the attention-based cross-view network. Input the right-eye feature F R into the attention-based cross-view network to obtain the right-eye enhanced feature output by the attention-based cross-view network. After that, input the left-eye enhanced feature into the image reconstruction component to obtain the left-eye Mars enhanced image Output the right-eye enhanced feature into the image reconstruction component to obtain the right-eye Mars enhanced image output by the image reconstruction component

[0052] For the feature extraction component, as Figure 2 shown. The left-eye compressed Mars image I L and the right-eye compressed Mars image I RFirst, all enter the convolutional layer (Conv.). The convolutional layer performs convolution operations by sliding the convolution kernel over the image to extract the initial features of the image, such as edges, textures, etc. The data processed by the convolutional layer then enters the residual dense block (RDBs). The residual dense block contains multiple convolutional layers inside, and the layers are densely connected, that is, the input of each layer is the concatenation of the output feature maps of all the previous layers. In this way, each layer can directly obtain the feature information of all the previous layers, making full use of the features at different levels, enhancing the propagation and reuse of features. At the same time, the residual dense block also introduces a residual connection. There is a direct path from the input to the output, adding the input features to the features processed by the dense connection layers. This method can effectively solve the problem of gradient disappearance, enabling the network to be trained more easily and helping to retain the information of the original image, improving the performance of the model. For the left-eye compressed Mars image I L First, it passes through the convolutional layer to become initial features, and then after being processed by the residual dense block, the left-eye feature F is obtained L ; For the right-eye compressed Mars image I R First, it passes through the convolutional layer to become initial features, and then after being processed by the residual dense block, the right-eye feature F is obtained R .

[0053] Considering that the Martian surface is highly unstructured, with irregular gravel, rocks, soil, and sand dunes, this application designs an attention-based cross-view network that combines block-level attention and pixel-level attention to capture more extensive context information between the two views and improve the pixel-level matching accuracy. The attention-based cross-view network is used to enhance, as Figure 2 shown, and includes two block-level attention modules and one pixel-level attention module. In some embodiments, the left-eye feature F L and the right-eye feature F R are input into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network, including: inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module; inputting the left-eye matching feature and the right-eye matching feature into the second block-level attention module to obtain the second left-eye enhanced feature and the second right-eye enhanced feature output by the second block-level attention module.

[0054] Input the left-eye feature F L into the first block-level attention module to obtain the first left-eye enhanced feature output by the first block-level attention module. Input the right-eye feature F RInput the first block-level attention module to obtain the first right-eye enhanced feature output by the first block-level attention module. Then, input the first left-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature output by the pixel-level attention module. Input the first right-eye enhanced feature into the pixel-level attention module to obtain the right-eye matching feature output by the pixel-level attention module. Further, input the left-eye matching feature into the second block-level attention module to obtain the second left-eye enhanced feature output by the second block-level attention module. Input the right-eye matching feature into the second block-level attention module to obtain the second right-eye enhanced feature output by the second block-level attention module.

[0055] As Figure 4 shown, the first block-level attention module includes a first intra-view patch attention unit, a cross-view patch attention unit, and a second intra-view patch attention unit. In some embodiments, input the left-eye feature F L and the right-eye feature F R into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module, including: input the left-eye feature F L and the right-eye feature F R into the first intra-view patch attention unit to obtain the intra-view patch enhanced feature output by the first intra-view patch attention unit; input the intra-view patch enhanced feature into the cross-view patch attention unit to obtain the cross-view patch enhanced feature output by the cross-view patch attention unit; input the cross-view patch enhanced feature into the second intra-view patch attention unit to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the second intra-view patch attention unit.

[0056] The first intra-view patch attention unit includes sub-modules such as partition, attention, layer norm, and multi-layer perceptron (MLP). The first intra-view patch attention unit satisfies the following formula (1):

[0057]

[0058] Input the left-eye feature F L into the first intra-view patch attention unit to obtain the intra-view patch enhanced feature F' L output by the first intra-view patch attention unit. Input the right-eye feature F R into the first intra-view patch attention unit to obtain the intra-view patch enhanced feature F' R output by the first intra-view patch attention unit.

[0059] Taking the enhancement of the left-eye compressed Mars image as an example, assume FL ∈R B×C×H×W , the input feature F L is divided into non-overlapping blocks of size P h ×P w . In the intra-view block attention unit, self-attention is applied to each block respectively to generate

[0060] The inter-view block attention unit includes sub-modules such as Partition, Attention, Layer norm, and MLP. The inter-view block attention unit satisfies the following formula (2):

[0061]

[0062] Input the intra-view block enhanced feature F′ L and the intra-view block enhanced feature F′ R into the inter-view block attention unit to obtain the inter-view block enhanced feature F″ output by the inter-view block attention unit L and the inter-view block enhanced feature F″ R .

[0063] During the calculation of the inter-view block attention, the left eye Q L and the right eye K R , V R perform the operation shown in the following formula (3):

[0064]

[0065] As Figure 4 shown, the intra-view block enhanced feature F′ L and the intra-view block enhanced feature F′ RFirst, all enter the chunking module. This module divides the input data into non-overlapping sub-chunks. The chunked data then enters the layer normalization module, which normalizes the data in the layer dimension so that the input data of each layer has a similar distribution. The layer-normalized data enters the attention mechanism module. The attention mechanism module assigns different attention weights to different parts according to the features of the input data, highlighting important feature information and suppressing less important information. The output of the attention mechanism module is added to the original input (the data after chunking and layer normalization) to form a residual connection, which helps with information transmission and network training. The result of the above residual connection enters the layer normalization module again to further adjust the data distribution. Then it enters the multi-layer perceptron (MLP). The multi-layer perceptron consists of multiple fully connected layers and can perform non-linear transformations on the input data, further extracting and fusing features, and enhancing the model's ability to understand and process data. The output of the multi-layer perceptron is added to the data that has gone through the previous processing steps (chunking, layer normalization, and attention mechanism) to obtain the final output, the inter-view block enhanced feature F″ L and the inter-view block enhanced feature F″ R .

[0066] The above inter-view block attention unit, by performing the above series of processes on the intra-view block enhanced feature F′ L and the intra-view block enhanced feature F′ R respectively, can better mine and utilize the information of the left and right views respectively. At the same time, through a similar processing flow, it ensures the consistency and coordination of the processing of the left and right views, helps improve the accuracy and performance of subsequent tasks, and better enhances the quality and feature expression of binocular images.

[0067] The second intra-view block attention unit satisfies the following formula (4):

[0068]

[0069] Input the inter-view block enhanced feature F″ L into the second intra-view block attention unit to obtain the first left-eye enhanced feature F″′ L output by the second intra-view block attention unit. Input the inter-view block enhanced feature F″ R into the second intra-view block attention unit to obtain the first left-eye enhanced feature F″′ R .

[0070] In some embodiments, after obtaining the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module, the first left-eye enhanced feature and the first right-eye enhanced feature are input into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module, including: concatenating the first left-eye enhanced feature with the left-eye feature to obtain the left-eye concatenated feature, and concatenating the first right-eye enhanced feature with the right-eye feature to obtain the right-eye concatenated feature; after passing the left-eye concatenated feature and the right-eye concatenated feature through the channel attention layer, the convolutional layer, and the sigmoid function layer respectively, inputting them into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

[0071] As Figure 2 shown, the first left-eye enhanced feature is concatenated (Concat) with the left-eye feature to obtain the left-eye concatenated feature, and then the left-eye concatenated feature is input into the pixel-level attention module after passing through the channel attention layer (CALayer, CA), the convolutional layer (Conv.), and the sigmoid function layer (Sigmoid). The first right-eye enhanced feature is concatenated with the right-eye feature to obtain the right-eye concatenated feature, and then the right-eye concatenated feature is input into the pixel-level attention module after passing through the CALayer, the Conv., and the Sigmoid. The left-eye matching feature and the right-eye matching feature output by the pixel-level attention module are obtained respectively.

[0072] Specifically, the left and right view features processed by the first block-level attention module are subjected to a concatenation (Concat) operation. Concatenation can combine the feature information of the left and right views, providing richer information for subsequent processing. The concatenated features enter the channel attention layer (CALayer). The channel attention layer assigns weights according to the importance of different channels, enhances the information of important channels, and suppresses less important channels, thereby further optimizing the feature representation. The features processed by the channel attention layer then enter the 1×1 convolutional layer and the 3×3 convolutional layer. The convolutional layer performs convolutional operations by sliding the convolutional kernel on the feature map, which is used to extract higher-level features and further fuse and transform the feature information. The output of the convolutional layer enters the Sigmoid. Sigmoid can map the output value to between 0 and 1, which is used to generate attention weights, and these weights can be used for subsequent pixel-level attention operations to highlight important pixel information.

[0073] In some embodiments, the pixel-level attention module includes: a cross-view pixel attention unit and an intra-view pixel attention unit. Inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module includes: inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the cross-view pixel attention unit to obtain the cross-view pixel attention output by the cross-view pixel attention unit; inputting the cross-view pixel attention into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

[0074] The cross-view pixel attention unit satisfies the following formula (5):

[0075]

[0076] The intra-view pixel attention unit satisfies the following formula (6):

[0077]

[0078] The pixel-level attention module first applies cross-view pixel attention to the first left-eye enhanced feature and the first right-eye enhanced feature, and then applies intra-view pixel attention to each view, thereby obtaining the enhanced feature F″ L and F″ R .

[0079] As Figure 4 shown, here the first left-eye enhanced feature is the new first left-eye enhanced feature obtained by concatenating with the left-eye feature and passing through CALayer, Conv., and Sigmoid, marked as F L in the figure. Similarly, the first right-eye enhanced feature is the new first right-eye enhanced feature obtained by concatenating with the right-eye feature and passing through CALayer, Conv., and Sigmoid, marked as F R in the figure.

[0080] The cross-view pixel attention unit includes sub-modules such as an attention mechanism module (Attention), Layer norm, Concat, and MLP. The first left-eye enhanced feature F L and the first right-eye enhanced feature F RThe input perspective - to - pixel attention unit calculates the perspective - to - pixel attention. Specifically, first, all data enters the attention mechanism module. The attention mechanism module will assign different attention weights to different pixels or features according to the characteristics of the input data, highlighting important information and suppressing less important information. The data processed by the attention mechanism module then enters the layer normalization module, which normalizes the data in the layer dimension, making the input data of each layer have a similar distribution, helping to stabilize network training and improve the generalization ability of the model. The layer - normalized data undergoes a concatenation operation (Concat). Concatenation can combine different feature information together, enrich the data representation, and provide more information for subsequent processing. The concatenated data enters a multi - layer perceptron (MLP). The multi - layer perceptron consists of multiple fully - connected layers, which can perform non - linear transformations on the input data, further extract and fuse features, and enhance the model's understanding and processing ability of the data. The output of the multi - layer perceptron enters the layer normalization module again to further adjust the data distribution. Then, it is added to the original input to form a residual connection, which helps with information transmission and network training, avoiding the problem of performance degradation in deep networks.

[0081] The above - mentioned pixel - level attention module, through the above - mentioned series of processes on the first left - eye enhanced feature F L and the first right - eye enhanced feature F R can better mine and utilize the pixel - level information of the left and right views respectively, improve the expression ability of features and the model's ability to capture key information, and provide better input for the subsequent in - perspective pixel attention unit. Then, the perspective - to - pixel attention is input into the in - perspective pixel attention unit. The in - perspective pixel attention unit includes: Attention, Layer norm, Concat, MLP. The left - eye matching feature F″ L and the right - eye matching feature F″ R output by the in - perspective pixel attention unit are obtained. Thus, the performance of the entire system in the binocular Mars image quality enhancement task is improved.

[0082] Such as Figure 2As shown, the left-eye matching features and right-eye matching features output by the pixel-level attention module are initially feature-extracted and transformed through a 1×1 convolutional layer (Conv.). The convolutional layer performs a convolution operation by sliding the convolution kernel over the data to extract local features. Then, it enters two sequentially connected residual dense blocks (RDBs). The data processed by the residual dense blocks enters the second block-level attention module, and the processing operation of the second block-level attention module is the same as that of the first block-level attention module. A concatenation (Concat) operation is performed on the output of the second block-level attention module to combine the feature information of different parts. The concatenated features enter the channel attention layer (CALayer). Finally, a convolutional layer (Conv.) is used for feature fusion and transformation. Thus, details that disappear in the current view but are retained in another view of the Mars image are discovered.

[0083] The above-mentioned attention-based cross-view network better mines and utilizes the information of the left and right views in the binocular image and the relationship between them through the intra-view block attention and pixel attention mechanisms, thereby improving the performance of the binocular Mars image quality enhancement task. By respectively focusing on different blocks and pixels within the view, it processes the image information more finely, makes full use of the characteristics of the binocular image, improves the efficiency and effect of the model when processing binocular images, and enhances the adaptability and robustness to complex scenes.

[0084] As Figure 2 shown, the image reconstruction component includes RDBs and Conv. The left-eye enhanced features and right-eye enhanced features are input into the image reconstruction component to obtain the left-eye Mars enhanced image and right-eye Mars enhanced image output by the image reconstruction component. Here, the image reconstruction component shares weights when processing the left-eye enhanced features and right-eye enhanced features (Shared weights) to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image Sharing weights means that the corresponding residual dense blocks and convolutional layers in the upper and lower parts use the same weight parameters. This design can reduce the number of model parameters, reduce the computational amount, and at the same time help ensure the consistency and symmetry in the processing of left and right images, enabling the model to work better together when processing the binocular Mars image quality enhancement task and improving the overall performance.

[0085] Specifically, both the left-eye enhanced features and right-eye enhanced features first enter the residual dense blocks (RDBs). The data processed by the residual dense blocks enters two convolutional layers (Conv.). The features are transformed and adjusted to adapt to the subsequent image reconstruction task. The outputs of the two convolutional layers are combined with the left-eye compressed Mars image I L and the right-eye compressed Mars image I RPerform an addition operation to form a residual connection. This residual connection method helps in information transmission, enabling the network to more easily learn the identity mapping, avoiding performance degradation problems in deep networks, and also helping to accelerate the network's convergence speed. After the above processing, the left-eye Mars enhanced image is output respectively and the right-eye Mars enhanced image

[0086] The above image reconstruction component, through the combination of residual dense blocks and convolutional layers, as well as the design of shared weights, can effectively extract and utilize image features, enhance the quality and details of the image, while reducing the consumption of computing resources and improving the training and inference efficiency of the model.

[0087] In summary, a method for enhancing the quality of binocular Mars images provided by this application first obtains a pair of binocular Mars images, including a left-eye compressed Mars image and a right-eye compressed Mars image; inputs the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component. In this way, this application improves the performance of the binocular Mars image quality enhancement task by mining and utilizing the information of the left and right views in the pair of binocular Mars images and the relationship between them. It makes full use of the characteristics of binocular Mars images to improve the efficiency and effect when enhancing the quality of binocular Mars images.

[0088] As Figure 5 shown Figure 5 is a schematic structural diagram of a device for enhancing the quality of binocular Mars images provided by an embodiment of this application. The device includes:

[0089] An acquisition module 501, configured to acquire a pair of binocular Mars images, and the pair of binocular Mars images includes a left-eye compressed Mars image and a right-eye compressed Mars image;

[0090] A quality enhancement module 502, configured to input the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image;

[0091] Wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0092] As an alternative implementation provided in the embodiments of the present application, the quality enhancement module 502 inputs the binocular Mars image pair into the pre-trained image quality enhancement model to obtain the enhanced image pair output by the pre-trained image quality enhancement model, specifically for: inputting the left-eye compressed Mars image and the right-eye compressed Mars image into the feature extraction component to obtain the left-eye feature and the right-eye feature output by the feature extraction component; inputting the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network; and inputting the left-eye enhanced feature and the right-eye enhanced feature into the image reconstruction component to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image output by the image reconstruction component.

[0093] As an alternative implementation provided in the embodiments of the present application, the attention-based cross-view network includes a first block-level attention module, a pixel-level attention module, and a second block-level attention module; the quality enhancement module 502 inputs the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network, specifically for: inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module; and inputting the left-eye matching feature and the right-eye matching feature into the second block-level attention module to obtain the second left-eye enhanced feature and the second right-eye enhanced feature output by the second block-level attention module.

[0094] As an alternative implementation provided in the embodiments of the present application, the first block-level attention module includes a first intra-view block attention unit, an inter-view block attention unit, and a second intra-view block attention unit; the quality enhancement module 502 inputs the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module, specifically for: inputting the left-eye feature and the right-eye feature into the first intra-view block attention unit to obtain the intra-view block enhanced feature output by the first intra-view block attention unit; inputting the intra-view block enhanced feature into the inter-view block attention unit to obtain the inter-view block enhanced feature output by the inter-view block attention unit; and inputting the inter-view block enhanced feature into the second intra-view block attention unit to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the second intra-view block attention unit.

[0095] As an alternative implementation provided in the embodiments of the present application, the quality enhancement module 502 inputs the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module. Specifically, it is used to: splice the first left-eye enhanced feature with the left-eye feature to obtain the left-eye spliced feature, and splice the first right-eye enhanced feature with the right-eye feature to obtain the right-eye spliced feature; after passing the left-eye spliced feature and the right-eye spliced feature through the channel attention layer, the convolutional layer, and the logical function layer respectively, input them into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

[0096] As an alternative implementation provided in the embodiments of the present application, the pixel-level attention module includes an inter-view pixel attention unit and an intra-view pixel attention unit; the quality enhancement module 502 inputs the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module. Specifically, it is used to: input the first left-eye enhanced feature and the first right-eye enhanced feature into the inter-view pixel attention unit to obtain the inter-view pixel attention output by the inter-view pixel attention unit; input the inter-view pixel attention into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

[0097] For the specific limitations of the binocular Mars image quality enhancement device, reference can be made to the limitations of the binocular Mars image quality enhancement method described above, which will not be elaborated here. Each module in the above binocular Mars image quality enhancement device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0098] In one embodiment, the present application provides an electronic device, which can be a terminal, and its internal structure diagram can be as Figure 6As shown in the figure. The electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, carrier network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements a method for detecting lags. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0099] Those skilled in the art can understand that Figure 6 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0100] In one embodiment, the binocular Mars image quality enhancement device provided by the present application can be implemented in the form of a computer program, and the computer program can run on an electronic device as shown in Figure 6 the figure. Each program module constituting the binocular Mars image quality enhancement device can be stored in the memory of the electronic device. For example, Figure 5 the acquisition module 501 and the quality enhancement module 502 shown in the figure. The computer program constituted by each program module enables the processor to execute the steps in the binocular Mars image quality enhancement method of each embodiment of the present application described in this specification.

[0101] For example, Figure 6 the electronic device shown in the figure can be through as shown in Figure 5The acquisition module 501 in the quality enhancement device for binocular Mars images shown executes to acquire a pair of binocular Mars images, where the pair of binocular Mars images includes a left-eye compressed Mars image and a right-eye compressed Mars image; the electronic device can execute to input the pair of binocular Mars images into a pre-trained image quality enhancement model through the quality enhancement module 502 to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0102] In one embodiment, the present application provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0103] Acquire a pair of binocular Mars images, where the pair of binocular Mars images includes a left-eye compressed Mars image and a right-eye compressed Mars image; input the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0104] In one embodiment, when the processor executes the computer program, the following steps are further implemented: input the pair of binocular Mars images into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, including: input the left-eye compressed Mars image and the right-eye compressed Mars image into the feature extraction component to obtain the left-eye feature and the right-eye feature output by the feature extraction component; input the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network; input the left-eye enhanced feature and the right-eye enhanced feature into the image reconstruction component to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image output by the image reconstruction component.

[0105] In one embodiment, when the processor executes the computer program, the following steps are further implemented: The attention-based cross-view network includes a first block-level attention module, a pixel-level attention module, and a second block-level attention module; inputting the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network, including: inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module; inputting the left-eye matching feature and the right-eye matching feature into the second block-level attention module to obtain the second left-eye enhanced feature and the second right-eye enhanced feature output by the second block-level attention module.

[0106] In one embodiment, when the processor executes the computer program, the following steps are further implemented: The first block-level attention module includes a first intra-view block attention unit, an inter-view block attention unit, and a second intra-view block attention unit; inputting the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module, including: inputting the left-eye feature and the right-eye feature into the first intra-view block attention unit to obtain the intra-view block enhanced feature output by the first intra-view block attention unit; inputting the intra-view block enhanced feature into the inter-view block attention unit to obtain the inter-view block enhanced feature output by the inter-view block attention unit; inputting the inter-view block enhanced feature into the second intra-view block attention unit to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the second intra-view block attention unit.

[0107] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module, including: concatenating the first left-eye enhanced feature with the left-eye feature to obtain a left-eye concatenated feature, and concatenating the first right-eye enhanced feature with the right-eye feature to obtain a right-eye concatenated feature; respectively passing the left-eye concatenated feature and the right-eye concatenated feature through a channel attention layer, a convolutional layer, and a logical function layer, and then inputting them into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

[0108] In one embodiment, when the processor executes the computer program, the following steps are further implemented: The pixel-level attention module includes an inter-view pixel attention unit and an intra-view pixel attention unit; inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module, including: inputting the first left-eye enhanced feature and the first right-eye enhanced feature into the inter-view pixel attention unit to obtain the inter-view pixel attention output by the inter-view pixel attention unit; inputting the inter-view pixel attention into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

[0109] When the processor in the electronic device provided in this application executes the computer program, first, a binocular Mars image pair is obtained, including a left-eye compressed Mars image and a right-eye compressed Mars image; the binocular Mars image pair is input into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image data set, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component. In this way, this application improves the performance of the binocular Mars image quality enhancement task by mining and utilizing the information of each of the left and right views in the binocular Mars image pair and the relationship between them. Make full use of the characteristics of the binocular Mars image to improve the efficiency and effect when enhancing the quality of the binocular Mars image.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer, the following steps are implemented:

[0111] Obtain a binocular Mars image pair, the binocular Mars image pair includes a left-eye compressed Mars image and a right-eye compressed Mars image; input the binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image data set, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

[0112] In one embodiment, when the computer program executes the computer program, the following steps are further implemented: input a binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, including: input the left-eye compressed Mars image and the right-eye compressed Mars image into a feature extraction component to obtain the left-eye feature and the right-eye feature output by the feature extraction component; input the left-eye feature and the right-eye feature into an attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network; input the left-eye enhanced feature and the right-eye enhanced feature into an image reconstruction component to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image output by the image reconstruction component.

[0113] In one embodiment, when the computer program executes the computer program, the following steps are further implemented: the attention-based cross-view network includes a first block-level attention module, a pixel-level attention module, and a second block-level attention module; input the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain the left-eye enhanced feature and the right-eye enhanced feature output by the attention-based cross-view network, including: input the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module; input the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module; input the left-eye matching feature and the right-eye matching feature into the second block-level attention module to obtain the second left-eye enhanced feature and the second right-eye enhanced feature output by the second block-level attention module.

[0114] In one embodiment, when the computer program executes the computer program, the following steps are further implemented: the first block-level attention module includes a first intra-view block attention unit, an inter-view block attention unit, and a second intra-view block attention unit; input the left-eye feature and the right-eye feature into the first block-level attention module to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the first block-level attention module, including: input the left-eye feature and the right-eye feature into the first intra-view block attention unit to obtain the intra-view block enhanced feature output by the first intra-view block attention unit; input the intra-view block enhanced feature into the inter-view block attention unit to obtain the inter-view block enhanced feature output by the inter-view block attention unit; input the inter-view block enhanced feature into the second intra-view block attention unit to obtain the first left-eye enhanced feature and the first right-eye enhanced feature output by the second intra-view block attention unit.

[0115] In one embodiment, when the computer program executes the computer program, the following steps are further implemented: input the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module, including: splicing the first left-eye enhanced feature with the left-eye feature to obtain the left-eye spliced feature, and splicing the first right-eye enhanced feature with the right-eye feature to obtain the right-eye spliced feature; respectively passing the left-eye spliced feature and the right-eye spliced feature through the channel attention layer, the convolutional layer and the logical function layer, and then inputting them into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

[0116] In one embodiment, when the computer program executes the computer program, the following steps are further implemented: the pixel-level attention module includes an inter-view pixel attention unit and an intra-view pixel attention unit; input the first left-eye enhanced feature and the first right-eye enhanced feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module, including: input the first left-eye enhanced feature and the first right-eye enhanced feature into the inter-view pixel attention unit to obtain the inter-view pixel attention output by the inter-view pixel attention unit; input the inter-view pixel attention into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

[0117] When the computer program in the computer-readable storage medium provided by the present application executes the computer program, first obtain a binocular Mars image pair, including a left-eye compressed Mars image and a right-eye compressed Mars image; input the binocular Mars image pair into the pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, and the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; wherein, the pre-trained image quality enhancement model is trained and optimized based on a pre-built binocular Mars image data set, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network and an image reconstruction component. In this way, the present application improves the performance of the binocular Mars image quality enhancement task by mining and utilizing the information of each of the left and right views in the binocular Mars image pair and the relationship between them. Make full use of the characteristics of the binocular Mars image to improve the efficiency and effect when enhancing the quality of the binocular Mars image.

[0118] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0119] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0120] In the present application, the processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0121] In the present application, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0122] In this application, computer-readable media include both permanent and non-permanent, removable and non-removable storage media. The storage media can implement information storage by any method or technology, and the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transitory media such as modulated data signals and carrier waves.

[0123] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0124] The above are only specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to these embodiments herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for enhancing the quality of binocular Mars images, characterized in that: include: Acquire a binocular Mars image pair, wherein the binocular Mars image pair includes a left-eye compressed Mars image and a right-eye compressed Mars image; Inputting the binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, wherein the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; The pre-trained image quality enhancement model is obtained by training and optimization based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

2. The method according to claim 1, characterized in that The step of inputting the binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model comprises: Input the left-eye compressed Mars image and the right-eye compressed Mars image into the feature extraction component to obtain the left-eye feature and the right-eye feature output by the feature extraction component; Inputting the left-eye feature and the right-eye feature into the attention-based cross-view network to obtain a left-eye enhanced feature and a right-eye enhanced feature output by the attention-based cross-view network; The left-eye enhancement feature and the right-eye enhancement feature are input into the image reconstruction component to obtain the left-eye Mars enhanced image and the right-eye Mars enhanced image output by the image reconstruction component.

3. The method according to claim 2, characterized in that The attention-based cross-view network includes a first block-level attention module, a pixel-level attention module, and a second block-level attention module; The step of inputting the left eye feature and the right eye feature into the attention-based cross-view network to obtain the left eye enhanced feature and the right eye enhanced feature output by the attention-based cross-view network includes: Inputting the left eye feature and the right eye feature into the first block-level attention module to obtain a first left eye enhanced feature and a first right eye enhanced feature output by the first block-level attention module; Inputting the first left-eye enhancement feature and the first right-eye enhancement feature into the pixel-level attention module to obtain a left-eye matching feature and a right-eye matching feature output by the pixel-level attention module; The left-eye matching feature and the right-eye matching feature are input into the second block-level attention module to obtain a second left-eye enhanced feature and a second right-eye enhanced feature output by the second block-level attention module.

4. The method according to claim 3, characterized in that The first block-level attention module includes a first intra-view block attention unit, an inter-view block attention unit, and a second intra-view block attention unit; The step of inputting the left eye feature and the right eye feature into the first block-level attention module to obtain a first left eye enhanced feature and a first right eye enhanced feature output by the first block-level attention module includes: Inputting the left-eye feature and the right-eye feature into the first intra-view block attention unit to obtain an intra-view block enhancement feature output by the first intra-view block attention unit; Inputting the intra-view block enhancement feature into the inter-view block attention unit to obtain the inter-view block enhancement feature output by the inter-view block attention unit; The inter-view block enhancement feature is input into the second intra-view block attention unit to obtain the first left-view enhancement feature and the first right-view enhancement feature output by the second intra-view block attention unit.

5. The method according to claim 3, characterized in that: The step of inputting the first left-eye enhancement feature and the first right-eye enhancement feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module includes: splicing the first left-eye enhancement feature with the left-eye feature to obtain a left-eye splicing feature, and splicing the first right-eye enhancement feature with the right-eye feature to obtain a right-eye splicing feature; The left-eye splicing feature and the right-eye splicing feature are respectively passed through the channel attention layer, the convolution layer and the logic function layer, and then input into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module.

6. The method according to claim 3, characterized in that The pixel-level attention module includes an inter-view pixel attention unit and an intra-view pixel attention unit; The step of inputting the first left-eye enhancement feature and the first right-eye enhancement feature into the pixel-level attention module to obtain the left-eye matching feature and the right-eye matching feature output by the pixel-level attention module includes: Inputting the first left-eye enhancement feature and the first right-eye enhancement feature into an inter-view pixel attention unit to obtain an inter-view pixel attention output by the inter-view pixel attention unit; The inter-view pixel attention is input into the intra-view pixel attention unit to obtain the left-eye matching feature and the right-eye matching feature output by the intra-view pixel attention unit.

7. A binocular Mars image quality enhancement device, characterized in that: include: An acquisition module, used for acquiring a binocular Mars image pair, wherein the binocular Mars image pair includes a left-eye compressed Mars image and a right-eye compressed Mars image; A quality enhancement module, used for inputting the binocular Mars image pair into a pre-trained image quality enhancement model to obtain an enhanced image pair output by the pre-trained image quality enhancement model, wherein the enhanced image pair includes a left-eye Mars enhanced image and a right-eye Mars enhanced image; The pre-trained image quality enhancement model is obtained by training and optimization based on a pre-built binocular Mars image dataset, and the pre-trained image quality enhancement model includes a feature extraction component, an attention-based cross-view network, and an image reconstruction component.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the binocular Mars image quality enhancement method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for enhancing the quality of a binocular Mars image according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that include: The computer program product comprises a computer program, and when the computer program is executed on a computer, the computer is enabled to implement the binocular Mars image quality enhancement method according to any one of claims 1 to 6.

Images