A method and system for generating super-resolution binocular images based on geometric consistency

Abstract

This invention provides a method and system for generating super-resolution binocular images based on geometric structure consistency, comprising: extracting deep features from low-resolution binocular images using a convolutional neural network or Transformer model, combining a cross-attention module to realize information interaction between the left and right images, and constructing a pixel correlation matrix between the left and right images; using the pixel correlation matrix to obtain the pixel correspondence between the left and right images and constructing a continuous disparity field; spatially warping the deep features based on the continuous disparity field to obtain warped features, and aligning the deep features of the left and right images; merging the spatially aligned deep features and warped features and inputting them into a feature upsampling module based on implicit two-dimensional representation, and outputting high-resolution binocular images of the same scene. This invention proposes a binocular image super-resolution technology that includes binocular image feature extraction, construction of a continuous disparity field based on implicit two-dimensional representation, spatial alignment of left and right image features, and feature upsampling based on implicit two-dimensional representation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for generating super-resolution binocular images based on geometric consistency. Background Technology

[0002] Existing binocular super-resolution methods typically only interact with information from the binocular images during feature extraction. For the extracted deep features, the left and right images remain independent during upsampling. This paradigm tends to optimize visual details at each viewpoint while neglecting geometric consistency under binocular stereo imaging conditions. Considering the characteristics of binocular images, the disparity between the left and right images, as an explicit constraint, can help align features from different viewpoints, thereby achieving high-resolution detail restoration with consistent geometry.

[0003] Traditional feature upsampling processes often employ transposed convolution or PixelShuffle methods. These methods are limited to fixed-scale upsampling, treating super-resolution at different scales as independent tasks, and are often time-consuming and labor-intensive during training. In recent years, implicit 2D representation techniques have developed rapidly. For discrete images, implicit 2D representation treats the image as a continuous representation, mapping continuous 2D spatial coordinates to pixel values.

[0004] Therefore, a novel method for generating super-resolution binocular images based on geometric consistency is needed. Summary of the Invention

[0005] This invention provides a method and system for generating super-resolution binocular images based on geometric consistency, in order to overcome the deficiencies in the prior art.

[0006] In a first aspect, the present invention provides a method for generating super-resolution binocular images based on geometrical consistency, comprising:

[0007] Acquire low-resolution binocular images of the same scene;

[0008] Deep features of low-resolution binocular images are extracted using convolutional neural networks or Transformer models. A cross-attention module is used to realize the interaction of information between the left and right images and to construct a pixel correlation matrix between the left and right images.

[0009] By using the pixel correlation matrix, the pixel correspondence between the left and right images is obtained, and a continuous disparity field is constructed in an implicit two-dimensional representation.

[0010] Based on the continuous parallax field, the deep features are spatially warped to obtain warped features, which are then spatially aligned with the deep features of the left and right images.

[0011] The spatially aligned deep features and warped features are merged and input into the feature upsampling module based on implicit 2D representation, and the output is a high-resolution binocular image of the same scene.

[0012] According to the present invention, a method for generating super-resolution binocular images based on geometric consistency is provided to acquire low-resolution binocular images of the same scene, comprising:

[0013] Low-resolution images captured by a binocular camera in the same scene, or low-resolution images captured at the left and right viewpoints respectively;

[0014] The two cameras are located on the same plane and have the same parameters.

[0015] According to the present invention, a method for generating super-resolution binocular images based on geometric consistency is provided, which uses a convolutional neural network or a Transformer model to extract deep features from low-resolution binocular images, including:

[0016] Input low-resolution binocular images , and The left and right images are represented respectively, and shallow features are extracted using a shallow convolutional neural network.

[0017] Shallow features are input into a deep feature extractor that stacks multiple convolutional neural networks or Transformer modules to extract deep features. , and These represent the deep features of the left image and the deep features of the right image, respectively.

[0018] According to the present invention, a super-resolution binocular image generation method based on geometric consistency is provided, which combines a cross-attention module to realize the interaction of left and right image information and constructs a pixel correlation matrix of the left and right images, including:

[0019] A cross-attention module is inserted between each convolutional neural network module or Transformer module to enable interaction of left and right image information.

[0020] Obtain the pixel correlation matrix of the left and right images ,in, The left image is represented by the first The first column and right image The relationship between columns.

[0021] According to the present invention, a super-resolution binocular image generation method based on geometric structure consistency is provided. This method utilizes a pixel correlation matrix to obtain the pixel correspondence between the left and right images, and constructs a continuous disparity field in an implicit two-dimensional representation. The method includes:

[0022] coordinates in two-dimensional image space and deep features of images For input, coordinates Corresponding signal value For the output, the implicit two-dimensional representation is determined to treat the image as a continuous representation;

[0023] The implicit two-dimensional representation is mainly composed of a multilayer perceptron (MLP) network, and its expression is:

[0024]

[0025] Latent encoding The coordinates are uniformly distributed in the two-dimensional space of the image, and each code is assigned a two-dimensional coordinate. For the coordinates to be queried... Corresponding signal value Represented as:

[0026]

[0027] in, Indicates the distance from the coordinates to be queried Recent potential encoding, express Two-dimensional coordinates in image space;

[0028] To prevent jagged edges and to aggregate richer information, Extended to latent encoding within the local neighborhood:

[0029]

[0030] in, To query coordinates The four neighboring areas centered on the center Adjacent pixels The weight, which is determined by... and Calculate the area of ​​the rectangular region formed by the vertices;

[0031] Through the pixel correlation matrix of the left and right images The pixel correspondence between the left and right images is obtained, and the Winner-Take-All (WTA) algorithm is used to regress and obtain a low-resolution original disparity map. Disparity latent coding is extracted using ResNet residual network. , continuously two-dimensional coordinates The disparity values ​​corresponding to the continuous disparity field are obtained by regression in the multilayer perceptron (MLP). The process is as follows:

[0032]

[0033]

[0034] .

[0035] According to the present invention, a super-resolution binocular image generation method based on geometric structure consistency is provided, which obtains warped features by spatially warping deep features based on a continuous parallax field, and spatially aligns the deep features of the left and right images, including:

[0036] Based on the continuous parallax field for deep features Spatial warping is performed separately to obtain warping features. ;

[0037] Based on warping characteristics Continuous two-dimensional coordinates and the corresponding disparity value Obtain the spatial coordinates after warping. .

[0038] According to the present invention, a method for generating super-resolution binocular images based on geometric structure consistency is provided, which merges spatially aligned deep features and warping features into a feature upsampling module based on implicit two-dimensional representation, and outputs high-resolution binocular images of the same scene, including:

[0039] Spatial alignment of deep features and warping characteristics After merging, the data is input into a multilayer perceptron (MLP) regression module based on implicit two-dimensional representation for feature upsampling, to obtain the corresponding RGB pixel values. :

[0040]

[0041] From RGB pixel values Render high-resolution binocular images of any scale.

[0042] Secondly, the present invention also provides a super-resolution binocular image generation system based on geometrical consistency, comprising:

[0043] The acquisition module is used to acquire low-resolution binocular images of the same scene;

[0044] The extraction module is used to extract deep features from low-resolution binocular images using convolutional neural networks or Transformer models. It combines the cross-attention module to realize the interaction of information between the left and right images and construct the pixel correlation matrix between the left and right images.

[0045] The module is used to obtain the pixel correspondence between the left and right images using the pixel correlation matrix, and to construct a continuous disparity field in an implicit two-dimensional representation.

[0046] The alignment module is used to spatially warp deep features based on a continuous parallax field to obtain warped features, and spatially align the deep features of the left and right images.

[0047] The output module is used to merge spatially aligned deep features and warped features into the feature upsampling module based on implicit two-dimensional representation, and output high-resolution binocular images of the same scene.

[0048] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the super-resolution binocular image generation method based on geometric consistency as described above.

[0049] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the super-resolution binocular image generation method based on geometric consistency as described above.

[0050] The present invention provides a method and system for generating super-resolution binocular images based on geometric structure consistency. By adopting a method that differs from previous binocular image feature extraction and independent upsampling of left and right viewpoint features, it proposes a binocular image super-resolution technology that includes four steps: binocular image feature extraction, construction of a continuous disparity field based on implicit two-dimensional representation, alignment of left and right image feature spaces, and feature upsampling based on implicit two-dimensional representation. Since implicit two-dimensional representation is used instead of traditional upsampling modules (such as bicubic upsampling), it can be applied to image super-resolution with any upsampling factor. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is one of the flowcharts of the super-resolution binocular image generation method based on geometric structure consistency provided by the present invention;

[0053] Figure 2 This is the second flowchart of the super-resolution binocular image generation method based on geometric structure consistency provided by the present invention;

[0054] Figure 3 This is a schematic diagram of the super-resolution binocular image generation system based on geometric consistency provided by the present invention;

[0055] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] Figure 1 This is one of the flowcharts illustrating the super-resolution binocular image generation method based on geometric consistency provided in this embodiment of the invention, such as... Figure 1 As shown, it includes:

[0058] Step 100: Acquire low-resolution binocular images of the same scene;

[0059] Step 200: Use a convolutional neural network or Transformer model to extract deep features from low-resolution binocular images, combine the cross-attention module to realize the interaction of information between the left and right images, and construct the pixel correlation matrix between the left and right images.

[0060] Step 300: Using the pixel correlation matrix, obtain the pixel correspondence between the left and right images, and construct a continuous disparity field in an implicit two-dimensional representation form;

[0061] Step 400: Spatially warp the deep features based on the continuous parallax field to obtain warped features, and spatially align the deep features of the left and right images.

[0062] Step 500: Merge the spatially aligned deep features and warped features and input them into the feature upsampling module based on implicit 2D representation to output a high-resolution binocular image of the same scene.

[0063] Specifically, such as Figure 2 As shown, this embodiment of the invention first uses convolutional neural networks (CNNs) or Transformers to extract deep features from binocular images. Simultaneously, a cross-attention module is used to achieve information interaction between the left and right image features, obtaining an association matrix of the left and right image features. Based on the association matrix, a continuous disparity field can be constructed between the left and right images. The continuous disparity field is used to spatially warp the deep features, aligning the deep features of the left and right images in space. The aligned deep features are then merged and input into a feature upsampling module based on implicit two-dimensional representation to obtain high-resolution binocular images.

[0064] In this invention, the binocular images are derived from two viewpoint images of the same scene captured by a binocular camera, one on the left and one on the right. The goal of this invention is to achieve synchronous super-resolution of the left and right viewpoint images. This invention differs from previous methods of binocular image feature extraction and independent upsampling of left and right viewpoint features. It proposes a binocular image super-resolution method comprising four steps: binocular image feature extraction, construction of a continuous disparity field based on implicit two-dimensional representation, alignment of left and right image feature spaces, and feature upsampling based on implicit two-dimensional representation. Because it uses implicit two-dimensional representation instead of traditional upsampling modules (such as bicubic upsampling), this invention can be applied to image super-resolution at any upsampling factor.

[0065] In one embodiment, the input data and output data in this invention are defined as follows:

[0066] The input includes low-resolution stereo images of the same scene, which are usually images acquired using a stereo camera or images acquired from the left and right viewpoints of the same scene; the output is a high-resolution stereo image corresponding to the input.

[0067] The first step is deep feature extraction:

[0068] Define the input low-resolution binocular image as First, shallow features are extracted using a shallow convolutional neural network. Then, deep features are extracted by inputting the data into a deep feature extractor consisting of multiple convolutional neural networks or stacked Transformer modules. Simultaneously, a cross-attention module is inserted between each module to enable information interaction between the left and right images and obtain the pixel correlation matrix of the left and right images, defined as... .in, The left image is represented by the first The first column and right image The relationship between columns.

[0069] The second step is to construct a continuous disparity field based on the implicit two-dimensional representation:

[0070] Implicit 2D representation treats the image as a continuous representation: the input is coordinates in a 2D image space. and the latent encoding of images (i.e., deep features) The output is coordinates. Corresponding signal value (Can be color information) (or other information). Due to coordinates It is continuous in the spatial domain, so the implicit two-dimensional representation supports representations at arbitrary resolutions.

[0071] The implicit two-dimensional representation is mainly composed of a multilayer perceptron (MLP) network, and its expression is as follows:

[0072]

[0073] Assuming latent encoding Evenly distributed in the two-dimensional space of the image, each code can be assigned a two-dimensional coordinate, thus for the coordinates to be queried... The corresponding signal value can be represented as:

[0074]

[0075] in Indicates the distance from the coordinates to be queried Recent potential encoding, express Two-dimensional coordinates in image space. Meanwhile, to prevent jagged edges and aggregate richer information, [the following is used:] ... Extended to latent encoding within the local neighborhood:

[0076]

[0077] in, To query coordinates The four neighboring areas centered on the center Adjacent pixels The weight, which is determined by... and The area of ​​the rectangular region formed by the vertices is calculated. The implicit two-dimensional representation establishes a direct relationship between the coordinates in the two-dimensional space and the corresponding signal values. The functional relationship established by equation (3) is applicable to all images.

[0078] Based on the pixel correlation matrix obtained in step one The properties of this property can be used to regress the original disparity map at a low resolution using the Winner Takes All (WTA) algorithm. Then, a residual network is used to extract the disparity latent encoding, and a continuous disparity field based on the implicit two-dimensional representation is trained. The process is as follows:

[0079]

[0080]

[0081]

[0082] The third step is to align the feature spaces of the left and right images:

[0083] Based on the deep features of the left and right images obtained in the aforementioned steps Based on the continuous parallax field, the deep features of the left and right images are spatially warped to obtain the warped features. Taking the left viewpoint as an example, we use parallax to spatially warp the latent encoding of the right viewpoint, aligning it with the space of the left viewpoint. During the spatial warping of the right viewpoint, the space located... The warping features at the location and located The original feature corresponds to the coordinates at that location. The relationship between the two coordinates can be written as:

[0084]

[0085] The fourth step is feature upsampling based on implicit two-dimensional representation:

[0086] The spatial alignment feature is obtained in step three. and Then, these values ​​are merged and input into a feature upsampling module based on implicit two-dimensional representation, ultimately outputting the corresponding RGB values. :

[0087]

[0088] Ultimately, high-resolution binocular images of any scale are rendered using the feature upsampling module.

[0089] The super-resolution binocular image generation system based on geometric structure consistency provided by the present invention will be described below. The super-resolution binocular image generation system based on geometric structure consistency described below can be referred to in correspondence with the super-resolution binocular image generation method based on geometric structure consistency described above.

[0090] Figure 3 This is a schematic diagram of the structure of the super-resolution binocular image generation system based on geometric consistency provided in an embodiment of the present invention, as shown below. Figure 3 As shown, it includes: an acquisition module 31, an extraction module 32, a construction module 33, an alignment module 34, and an output module 35, wherein:

[0091] The acquisition module 31 is used to acquire low-resolution binocular images of the same scene; the extraction module 32 is used to extract deep features of the low-resolution binocular images using a convolutional neural network or Transformer model, and to combine the cross-attention module to realize the interaction of information between the left and right images, and to construct the pixel correlation matrix of the left and right images; the construction module 33 is used to obtain the pixel correspondence between the left and right images using the pixel correlation matrix, and to construct a continuous disparity field in the form of an implicit two-dimensional expression; the alignment module 34 is used to spatially warp the deep features based on the continuous disparity field to obtain warped features, and to spatially align the deep features of the left and right images; the output module 35 is used to merge the spatially aligned deep features and warped features and input them into the feature upsampling module based on the implicit two-dimensional expression, and output a high-resolution binocular image of the same scene.

[0092] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a super-resolution binocular image generation method based on geometric consistency. This method includes: acquiring low-resolution binocular images of the same scene; extracting deep features from the low-resolution binocular images using a convolutional neural network or Transformer model, combining a cross-attention module to achieve information interaction between the left and right images, and constructing a pixel correlation matrix between the left and right images; using the pixel correlation matrix to obtain the pixel correspondence between the left and right images, constructing a continuous disparity field in an implicit two-dimensional representation; spatially warping the deep features based on the continuous disparity field to obtain warped features, spatially aligning the deep features of the left and right images; merging the spatially aligned deep features and warped features and inputting them into a feature upsampling module based on the implicit two-dimensional representation, outputting a high-resolution binocular image of the same scene.

[0093] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the super-resolution binocular image generation method based on geometric consistency provided by the above methods. The method includes: acquiring low-resolution binocular images of the same scene; extracting deep features of the low-resolution binocular images using a convolutional neural network or a Transformer model, combining a cross-attention module to realize the interaction of left and right image information, and constructing a pixel correlation matrix of the left and right images; using the pixel correlation matrix to obtain the pixel correspondence between the left and right images, and constructing a continuous disparity field in an implicit two-dimensional expression form; spatially warping the deep features based on the continuous disparity field to obtain warped features, and spatially aligning the deep features of the left and right images; merging the spatially aligned deep features and warped features and inputting them into a feature upsampling module based on implicit two-dimensional expression, and outputting a high-resolution binocular image of the same scene.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating super-resolution binocular images based on geometric consistency, characterized in that, include: Acquire low-resolution binocular images of the same scene; Deep features of low-resolution binocular images are extracted using convolutional neural networks or Transformer models. A cross-attention module is used to realize the interaction of information between the left and right images and to construct a pixel correlation matrix between the left and right images. By using the pixel correlation matrix, the pixel correspondence between the left and right images is obtained, and a continuous disparity field is constructed in an implicit two-dimensional representation. Based on the continuous parallax field, the deep features are spatially warped to obtain warped features, which are then spatially aligned with the deep features of the left and right images. The spatially aligned deep features and warped features are merged and input into the feature upsampling module based on implicit 2D representation, and the output is a high-resolution binocular image of the same scene. Specifically, the pixel correspondence between the left and right images is obtained using a pixel correlation matrix, and a continuous disparity field is constructed in an implicit two-dimensional representation, including: coordinates in two-dimensional image space and deep features of images For input, coordinates Corresponding signal value For the output, the implicit two-dimensional representation is determined to treat the image as a continuous representation; The implicit two-dimensional representation is mainly composed of a multilayer perceptron (MLP) network, and its expression is: Latent encoding The coordinates are uniformly distributed in the two-dimensional space of the image, and each code is assigned a two-dimensional coordinate. For the coordinates to be queried... Corresponding signal value Represented as: in, Indicates the distance from the coordinates to be queried Recent potential encoding, express Two-dimensional coordinates in image space; To prevent jagged edges and to aggregate richer information, Extended to latent encoding within the local neighborhood: in, To query coordinates The four neighboring areas centered on the center Adjacent pixels The weight, which is determined by... and Calculate the area of ​​the rectangular region formed by the vertices; Through the pixel correlation matrix of the left and right images The pixel correspondence between the left and right images is obtained, and the Winner-Take-All (WTA) algorithm is used to regress and obtain a low-resolution original disparity map. Disparity latent coding is extracted using ResNet residual network. , continuously two-dimensional coordinates The disparity values ​​corresponding to the continuous disparity fields are obtained by regression in the multilayer perceptron (MLP). The process is as follows: 。 2. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Acquire low-resolution binocular images of the same scene, including: Low-resolution images captured by a binocular camera in the same scene, or low-resolution images captured at the left and right viewpoints respectively; The two cameras are located on the same plane and have the same parameters.

3. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Deep features of low-resolution binocular images are extracted using convolutional neural networks or Transformer models, including: Input low-resolution binocular images , and The left and right images are represented respectively, and shallow features are extracted using a shallow convolutional neural network. Shallow features are input into a deep feature extractor that stacks multiple convolutional neural networks or Transformer modules to extract deep features. , and These represent the deep features of the left image and the deep features of the right image, respectively.

4. The method for generating super-resolution binocular images based on geometric consistency according to claim 3, characterized in that, By combining a cross-attention module, information interaction between the left and right images is achieved, and a pixel correlation matrix between the left and right images is constructed, including: A cross-attention module is inserted between each convolutional neural network module or Transformer module to enable interaction of left and right image information. Obtain the pixel correlation matrix of the left and right images ,in, The left image is represented by the first The first column and right image The relationship between columns.

5. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Based on the spatial warping of deep features using a continuous parallax field, warped features are obtained. These warped features are then spatially aligned with the deep features of the left and right images, including: Based on the continuous parallax field for deep features Spatial warping is performed separately to obtain warping features. ; Based on warping characteristics Continuous two-dimensional coordinates and the corresponding disparity value Obtain the spatial coordinates after warping. .

6. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, The spatially aligned deep features and warped features are merged and input into a feature upsampling module based on implicit 2D representation, outputting high-resolution binocular images of the same scene, including: Spatially aligned deep features and warping characteristics After merging, the data is input into a multilayer perceptron (MLP) regression module based on implicit two-dimensional representation for feature upsampling, to obtain the corresponding RGB pixel values. : From RGB pixel values Render high-resolution binocular images of any scale.

7. A super-resolution binocular image generation system based on geometric structure consistency, based on the super-resolution binocular image generation method based on geometric structure consistency according to any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire low-resolution binocular images of the same scene; The extraction module is used to extract deep features from low-resolution binocular images using convolutional neural networks or Transformer models. It combines the cross-attention module to realize the interaction of information between the left and right images and construct the pixel correlation matrix between the left and right images. The module is used to obtain the pixel correspondence between the left and right images using the pixel correlation matrix, and to construct a continuous disparity field in an implicit two-dimensional representation. The alignment module is used to spatially warp deep features based on a continuous parallax field to obtain warped features, and spatially align the deep features of the left and right images. The output module is used to merge spatially aligned deep features and warped features into the feature upsampling module based on implicit two-dimensional representation, and output high-resolution binocular images of the same scene.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the super-resolution binocular image generation method based on geometric consistency as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the super-resolution binocular image generation method based on geometric consistency as described in any one of claims 1 to 6.

Images