Super-resolution binocular image generation method and system based on geometric structure consistency

Abstract

The invention provides a super-resolution binocular image generation method and system based on geometric structure consistency, and the method comprises the steps: extracting the deep features of a low-resolution binocular image through employing a convolutional neural network or a Transform model, achieving the information interaction of a left image and a right image in combination with a cross attention module, and constructing a pixel incidence matrix of the left image and the right image; acquiring a pixel corresponding relation of the left image and the right image by using the pixel incidence matrix, and constructing a continuous parallax field; performing spatial warping on the deep features based on a continuous parallax field to obtain warping features, and aligning the deep features of the left and right images; and merging the deep features and the warping features after spatial alignment, inputting the merged features into a feature up-sampling module based on implicit two-dimensional expression, and outputting a high-resolution binocular image of the same scene. The invention provides a binocular image super-resolution technology comprising binocular image feature extraction, continuous parallax field construction based on implicit two-dimensional expression, left and right image feature space alignment and feature upsampling based on implicit two-dimensional expression.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a super-resolution binocular image generation method and system based on geometric structure consistency. BACKGROUND

[0002] The existing binocular image super-resolution usually only interacts the information of the binocular image in the feature extraction process, and the left and right images are still independent of each other in the up-sampling process of the extracted deep features. This processing paradigm tends to optimize the visual details of each viewpoint, while ignoring the geometric structure consistency under the binocular stereo imaging condition. Considering the characteristics of the binocular image, the disparity of the left and right images as an explicit constraint can help the alignment of the features of different viewpoints, thereby realizing the recovery of the geometric structure consistent high-resolution details.

[0003] The traditional feature up-sampling process is mostly the transpose convolution or PixelShuffle method, which is limited to fixed scale up-sampling, and different scale super-resolution is regarded as an independent task and is time-consuming and laborious in the training process. In recent years, the implicit two-dimensional expression technology has developed rapidly. For discrete images, the implicit two-dimensional expression regards the image as a continuous representation, and maps the continuous two-dimensional space coordinates to the pixel value.

[0004] Therefore, it is necessary to propose a new super-resolution binocular image generation method based on geometric structure consistency. SUMMARY

[0005] The present application provides a super-resolution binocular image generation method and system based on geometric structure consistency to solve the defects existing in the prior art.

[0006] In a first aspect, the present application provides a super-resolution binocular image generation method based on geometric structure consistency, comprising: obtaining low-resolution binocular images of the same scene; extracting deep features of the low-resolution binocular images using a convolutional neural network or a Transformer model, realizing information interaction of the left and right images by combining a cross-attention module, and constructing a pixel correlation matrix of the left and right images; using the pixel correlation matrix to obtain the pixel correspondence relationship of the left and right images, and constructing a continuous disparity field in the form of implicit two-dimensional expression; spatial warping the deep features based on the continuous disparity field to obtain warped features, and aligning the deep features of the left and right images in space; merging and inputting the spatially aligned deep features and the warped features into a feature up-sampling module based on implicit two-dimensional expression, and outputting high-resolution binocular images of the same scene.

[0007] The application provides a super-resolution binocular image generation method based on geometric structure consistency, low-resolution binocular images of the same scene are acquired, and the method comprises the following steps: The low-resolution images of the same scene are collected by a binocular camera or are collected by a left-view camera and a right-view camera respectively. The left camera and the right camera are located on the same plane, and the parameters of the left camera and the right camera are consistent.

[0008] The application provides a super-resolution binocular image generation method based on geometric structure consistency, deep features of low-resolution binocular images are extracted by using a convolutional neural network or a Transformer model, and the method comprises the following steps: The low-resolution binocular images are inputted , and respectively represent left images and right images, and shallow features are extracted by using a shallow convolutional neural network. Deep features are extracted by inputting the shallow features into a deep feature extractor in which a plurality of convolutional neural network or Transformer module stacks are arranged , and respectively represent left image deep features and right image deep features.

[0009] The application provides a super-resolution binocular image generation method based on geometric structure consistency, left image information and right image information are interacted by using a cross-attention module, and a pixel correlation matrix of the left images and the right images is constructed, and the method comprises the following steps: The cross-attention module is inserted between each convolutional neural network module or Transformer module, and left image information and right image information are interacted. The pixel correlation matrix of the left images and the right images is acquired , wherein represents the relationship between the i-th column of the left images and the j-th column of the right images.

[0010] The application provides a super-resolution binocular image generation method based on geometric structure consistency, pixel corresponding relationships of left images and right images are acquired by using the pixel correlation matrix, and a continuous disparity field is constructed in an implicit two-dimensional expression form, and the method comprises the following steps: Coordinates x and y in a two-dimensional image space and deep features f of the images are inputted , and a signal value corresponding to the coordinates x and y is outputted as y(x, y), and the images are regarded as continuous representations in the implicit two-dimensional expression form. The subject of the implicit two-dimensional expression is composed of a multi-layer perceptron (MLP) network, and the expression is as follows: ​​​​​

[0011] Latent encoding The coordinates are uniformly distributed in the two-dimensional space of the image, and each code is assigned a two-dimensional coordinate. For the coordinates to be queried... Corresponding signal value Represented as:

[0012] in, Indicates the distance from the coordinates to be queried Recent potential encoding, express Two-dimensional coordinates in image space; To prevent jagged edges and to aggregate richer information, Extended to latent encoding within the local neighborhood:

[0013] in, To query coordinates The four neighboring areas centered on the center Adjacent pixels The weight, which is determined by... and Calculate the area of ​​the rectangular region formed by the vertices; Through the pixel correlation matrix of the left and right images The pixel correspondence between the left and right images is obtained, and the Winner-Take-All (WTA) algorithm is used to regress and obtain a low-resolution original disparity map. Disparity latent coding is extracted using ResNet residual network. , continuously two-dimensional coordinates The disparity values ​​corresponding to the continuous disparity field are obtained by regression in the multilayer perceptron (MLP). The process is as follows:

[0014]

[0015] .

[0016] According to the present invention, a super-resolution binocular image generation method based on geometric structure consistency is provided, which obtains warped features by spatially warping deep features based on a continuous parallax field, and spatially aligns the deep features of the left and right images, including: Based on the continuous parallax field for deep features Spatial warping is performed separately to obtain warping features. ; Based on warping characteristics , continuous two-dimensional coordinates and the corresponding parallax value , obtain the spatial coordinates after warping .

[0017] According to the super-resolution binocular image generation method based on geometric structure consistency provided by the application, the spatially aligned deep features and the warped features are combined and input into the feature upsampling module based on implicit two-dimensional expression, and the high-resolution binocular image of the same scene is output, which comprises: The spatially aligned deep features and the warped features are combined and input into the multi-layer perception (MLP) in the feature upsampling module based on implicit two-dimensional expression for regression, and the corresponding RGB pixel value is obtained:

[0018] The high-resolution binocular image of any scale is rendered by the RGB pixel value .

[0019] In the second aspect, the application further provides a super-resolution binocular image generation system based on geometric structure consistency, which comprises: An acquisition module is configured to acquire low-resolution binocular images of the same scene. An extraction module is configured to extract deep features of the low-resolution binocular images by using a convolutional neural network or a Transformer model, realize information interaction between left and right images by combining a cross-attention module, and construct a pixel correlation matrix of the left and right images. A construction module is configured to obtain pixel corresponding relationships of the left and right images by using the pixel correlation matrix, and construct a continuous parallax field in the form of implicit two-dimensional expression. An alignment module is configured to perform spatial warping on the deep features based on the continuous parallax field to obtain warped features, and align the deep features of the left and right images in space. An output module is configured to combine the spatially aligned deep features and the warped features and input them into the feature upsampling module based on implicit two-dimensional expression, and output high-resolution binocular images of the same scene.

[0020] In the third aspect, the application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the super-resolution binocular image generation method based on geometric structure consistency as described above when executing the program.

[0021] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the super-resolution binocular image generation method based on geometric structure consistency as described above.

[0022] The super-resolution binocular image generation method and system based on geometric structure consistency provided by the present application adopts a method different from previous feature extraction of binocular images and independent up-sampling of features of left and right viewpoints, and proposes a binocular image super-resolution technology including four steps of feature extraction of binocular images, construction of a continuous disparity field based on implicit two-dimensional expression, alignment of feature spaces of left and right images, and feature up-sampling based on implicit two-dimensional expression. Since the implicit two-dimensional expression is adopted to replace a traditional up-sampling module (such as bicubic up-sampling), the image super-resolution of any up-sampling multiple can be applied. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0024] Figure 1 is one of the flowcharts of the super-resolution binocular image generation method based on geometric structure consistency provided by the present application; Figure 2 is another flowchart of the super-resolution binocular image generation method based on geometric structure consistency provided by the present application; Figure 3 is a structural schematic diagram of the super-resolution binocular image generation system based on geometric structure consistency provided by the present application; Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0026] Figure 1 is one of the flowcharts of the super-resolution binocular image generation method based on geometric structure consistency provided by the present application, as shown in Figure 1 the flowchart includes: Step 100: obtaining low-resolution binocular images of the same scene; Step 200: extracting deep features of the low-resolution binocular images using a convolutional neural network or a Transformer model, realizing information interaction of left and right image features by a cross-attention module, and constructing a pixel correlation matrix of left and right images; Step 300: obtaining pixel correspondence of left and right images by using the pixel correlation matrix, and constructing a continuous disparity field in an implicit two-dimensional expression form; Step 400: spatial warping of deep features based on the continuous disparity field to obtain warped features, and spatial alignment of deep features of left and right images; Step 500: merging and inputting the spatially aligned deep features and the warped features into a feature upsampling module based on implicit two-dimensional expression, and outputting high-resolution binocular images of the same scene.

[0027] Specifically, as shown in the figure, Figure 2 the embodiment of the present application first extracts deep features of binocular images using a convolutional neural network (CNN) or a Transformer, simultaneously realizes information interaction of left and right image features by a cross-attention module, and obtains a correlation matrix of left and right image features. Relying on the correlation matrix, a continuous disparity field of left and right images can be constructed. The deep features are spatially warped by using the continuous disparity field, the deep features of left and right images are spatially aligned, and the aligned deep features are merged and input into a feature upsampling module based on implicit two-dimensional expression to obtain high-resolution binocular images.

[0028] Among them, the binocular images are left and right two-viewpoint images of the same scene captured by a binocular camera, and the target of the present application is to realize the synchronous super-resolution of left and right viewpoint images. Different from the previous methods of binocular image feature extraction and independent upsampling of left and right viewpoint features, the present application proposes a binocular image super-resolution invention including four steps of binocular image feature extraction, continuous disparity field construction based on implicit two-dimensional expression, spatial alignment of left and right image features, and feature upsampling based on implicit two-dimensional expression. Since implicit two-dimensional expression is used instead of the traditional upsampling module (such as bicubic upsampling), the present application can be applied to image super-resolution of any upsampling multiple.

[0029] In one embodiment, the input data and output data in the embodiment of the present application are defined as follows: The input includes low-resolution binocular images of the same scene, and the binocular images are usually images collected by a binocular camera or images collected at left and right viewpoints in the same scene; the output is a high-resolution binocular image corresponding to the input.

[0030] The first step is deep feature extraction: the low-resolution binocular images of the input are First, shallow features are extracted by a shallow convolutional neural network, and then deep features are extracted by a deep feature extractor stacked with multiple convolutional neural networks or Transformer modules Meanwhile, cross-attention modules are inserted between each module to realize the interaction of left and right image information and obtain the pixel correlation matrix of left and right images, defined as wherein, represents the relationship between the column of the left image and the column of the right image.

[0031] The second step is to construct a continuous disparity field based on implicit two-dimensional representation: The implicit two-dimensional representation regards the image as a continuous representation: the input is the coordinate in the two-dimensional image space and the latent encoding (i.e., deep feature ) of the image; the output is the signal value corresponding to the coordinate (which can be color information or other information). Since the coordinate is continuous in the spatial domain, the implicit two-dimensional representation supports arbitrary resolution representation.

[0032] The main body of the implicit two-dimensional representation is composed of a multilayer perceptron (MLP) network, and its expression is:

[0033] Assuming that the latent encodings are uniformly distributed in the two-dimensional space of the image, a two-dimensional coordinate can be assigned to each encoding, so for the query coordinate , the corresponding signal value can be expressed as:

[0034] wherein represents the nearest latent encoding to the query coordinate , represents the two-dimensional coordinate of in the image space. At the same time, in order to prevent jagged generation and aggregate more abundant information, the is extended to the latent encodings within the local neighborhood:

[0035] wherein, is the four-neighborhood centered on the query coordinate , weight of the neighboring pixel, which is calculated by the area of the rectangle region composed of the vertices and The implicit two-dimensional representation establishes a direct relationship between the coordinates of the two-dimensional space and the signal value corresponding to the coordinates. The function relationship established by equation (3) is applicable to all images.

[0036] According to the properties of the pixel correlation matrix obtained in step one , the low-resolution original disparity map can be obtained by using the Winner Takes All (WTA) algorithm for regression , and then the residual network is used to extract the disparity latent code to train the continuous disparity field based on the implicit two-dimensional representation. The process is as follows:

[0037]

[0038]

[0039] The third step is to align the left and right image feature spaces: According to the deep features of the left and right images obtained in the previous steps and the continuous disparity field, the deep features of the left and right images are respectively warped in space according to the continuous disparity field to obtain warped features . Taking the left viewpoint as an example, we warp the right viewpoint latent code in space with the disparity, so that it is aligned with the left viewpoint in space. During the spatial warping process of the right view, the warped feature located at corresponds to the original feature located at . The relationship between the two coordinates can be written as:

[0040] The fourth step is feature upsampling based on the implicit two-dimensional representation: After obtaining the spatially aligned features and in step three, they are combined and input into the feature upsampling module based on the implicit two-dimensional representation, and finally the corresponding RGB value is output:

[0041] Finally, the feature upsampling module is used to render high-resolution binocular images of any scale.

[0042] ​The super-resolution binocular image generation system based on geometric structure consistency provided by the present invention will be described below. The super-resolution binocular image generation system based on geometric structure consistency described below can be referred to in correspondence with the super-resolution binocular image generation method based on geometric structure consistency described above.

[0043] Figure 3 This is a schematic diagram of the structure of the super-resolution binocular image generation system based on geometric consistency provided in an embodiment of the present invention, as shown below. Figure 3 As shown, it includes: an acquisition module 31, an extraction module 32, a construction module 33, an alignment module 34, and an output module 35, wherein: The acquisition module 31 is used to acquire low-resolution binocular images of the same scene; the extraction module 32 is used to extract deep features of the low-resolution binocular images using a convolutional neural network or Transformer model, and to combine the cross-attention module to realize the interaction of information between the left and right images, and to construct the pixel correlation matrix of the left and right images; the construction module 33 is used to obtain the pixel correspondence between the left and right images using the pixel correlation matrix, and to construct a continuous disparity field in the form of an implicit two-dimensional expression; the alignment module 34 is used to spatially warp the deep features based on the continuous disparity field to obtain warped features, and to spatially align the deep features of the left and right images; the output module 35 is used to merge the spatially aligned deep features and warped features and input them into the feature upsampling module based on the implicit two-dimensional expression, and output a high-resolution binocular image of the same scene.

[0044] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a super-resolution binocular image generation method based on geometric consistency. This method includes: acquiring low-resolution binocular images of the same scene; extracting deep features from the low-resolution binocular images using a convolutional neural network or Transformer model, combining a cross-attention module to achieve information interaction between the left and right images, and constructing a pixel correlation matrix between the left and right images; using the pixel correlation matrix to obtain the pixel correspondence between the left and right images, constructing a continuous disparity field in an implicit two-dimensional representation; spatially warping the deep features based on the continuous disparity field to obtain warped features, spatially aligning the deep features of the left and right images; merging the spatially aligned deep features and warped features and inputting them into a feature upsampling module based on the implicit two-dimensional representation, outputting a high-resolution binocular image of the same scene.

[0045] Moreover, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0046] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the super-resolution binocular image generation method based on geometric consistency provided by the above-mentioned methods, the method comprising: acquiring low-resolution binocular images of the same scene; extracting deep features of the low-resolution binocular images using a convolutional neural network or a Transformer model, realizing information interaction between left and right images in combination with a cross-attention module, and constructing a pixel correlation matrix of the left and right images; using the pixel correlation matrix, obtaining the pixel correspondence relationship of the left and right images, and constructing a continuous disparity field in an implicit two-dimensional expression form; based on the continuous disparity field, performing spatial warping on the deep features to obtain warped features, and aligning the deep features of the left and right images in space; inputting the spatially aligned deep features and the warped features into a feature upsampling module based on the implicit two-dimensional expression, and outputting high-resolution binocular images of the same scene.

[0047] The device embodiments described above are only schematic, wherein the units illustrated as separate components can or can not be physically separate, and the components illustrated as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0048] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0049] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating super-resolution binocular images based on geometrical consistency, characterized in that, include: Acquire low-resolution binocular images of the same scene; Deep features of low-resolution binocular images are extracted using convolutional neural networks or Transformer models. A cross-attention module is used to realize the interaction of information between the left and right images and to construct the pixel correlation matrix between the left and right images. By using the pixel correlation matrix, the pixel correspondence between the left and right images is obtained, and a continuous disparity field is constructed in an implicit two-dimensional representation. Based on the continuous parallax field, the deep features are spatially warped to obtain warped features, which are then spatially aligned with the deep features of the left and right images. The spatially aligned deep features and warped features are merged and input into the feature upsampling module based on implicit 2D representation, and the output is a high-resolution binocular image of the same scene.

2. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Acquire low-resolution binocular images of the same scene, including: Low-resolution images captured by a binocular camera in the same scene, or low-resolution images captured at the left and right viewpoints respectively; The two cameras are located on the same plane and have the same parameters.

3. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Deep features of low-resolution binocular images are extracted using convolutional neural networks or Transformer models, including: Input low-resolution binocular images , and The left and right images are represented respectively, and shallow features are extracted using a shallow convolutional neural network. Shallow features are input into a deep feature extractor that stacks multiple convolutional neural networks or Transformer modules to extract deep features. , and These represent the deep features of the left image and the deep features of the right image, respectively.

4. The method for generating super-resolution binocular images based on geometric consistency according to claim 3, characterized in that, By combining a cross-attention module, information interaction between the left and right images is achieved, and a pixel correlation matrix between the left and right images is constructed, including: A cross-attention module is inserted between each convolutional neural network module or Transformer module to enable interaction of left and right image information. Obtain the pixel correlation matrix of the left and right images ,in, The left image is represented by the first The first column and right image The relationship between columns.

5. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Using a pixel correlation matrix, the pixel correspondence between the left and right images is obtained, and a continuous disparity field is constructed in an implicit two-dimensional representation, including: coordinates in two-dimensional image space and deep features of images For input, coordinates Corresponding signal value For the output, the implicit two-dimensional representation is determined to treat the image as a continuous representation; The implicit two-dimensional representation is mainly composed of a multilayer perceptron (MLP) network, and its expression is: Latent encoding The coordinates are uniformly distributed in the two-dimensional space of the image, and each code is assigned a two-dimensional coordinate. For the coordinates to be queried... Corresponding signal value Represented as: in, Indicates the distance from the coordinates to be queried Recent potential encoding, express Two-dimensional coordinates in image space; To prevent jagged edges and to aggregate richer information, Extended to latent encoding within the local neighborhood: in, To query coordinates The four neighboring areas centered on the center Adjacent pixels The weight, which is determined by... and Calculate the area of ​​the rectangular region formed by the vertices; Through the pixel correlation matrix of the left and right images The pixel correspondence between the left and right images is obtained, and the Winner-Take-All (WTA) algorithm is used to regress and obtain a low-resolution original disparity map. Disparity latent coding is extracted using ResNet residual network. , continuously two-dimensional coordinates The disparity values ​​corresponding to the continuous disparity field are obtained by regression in the multilayer perceptron (MLP). The process is as follows: 。 6. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, Based on the spatial warping of deep features using a continuous parallax field, warped features are obtained. These warped features are then spatially aligned with the deep features of the left and right images, including: Based on the continuous parallax field for deep features Spatial warping is performed separately to obtain warping features. ; Based on warping characteristics Continuous two-dimensional coordinates and the corresponding disparity value Obtain the spatial coordinates after warping. .

7. The method for generating super-resolution binocular images based on geometric consistency according to claim 1, characterized in that, The spatially aligned deep features and warped features are merged and input into a feature upsampling module based on implicit 2D representation, outputting high-resolution binocular images of the same scene, including: Spatially aligned deep features and warping characteristics After merging, the data is input into a multilayer perceptron (MLP) regression module based on implicit two-dimensional representation for feature upsampling, to obtain the corresponding RGB pixel values. : From RGB pixel values Render high-resolution binocular images of any scale.

8. A super-resolution binocular image generation system based on geometrical consistency, characterized in that, include: The acquisition module is used to acquire low-resolution binocular images of the same scene; The extraction module is used to extract deep features of low-resolution binocular images using convolutional neural networks or Transformer models. It combines the cross-attention module to realize the interaction of information between the left and right images and construct the pixel correlation matrix between the left and right images. The module is used to obtain the pixel correspondence between the left and right images using the pixel correlation matrix, and to construct a continuous disparity field in an implicit two-dimensional representation. The alignment module is used to spatially warp deep features based on a continuous parallax field to obtain warped features, and spatially align the deep features of the left and right images. The output module is used to merge spatially aligned deep features and warped features into the feature upsampling module based on implicit two-dimensional representation, and output high-resolution binocular images of the same scene.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the super-resolution binocular image generation method based on geometric consistency as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the super-resolution binocular image generation method based on geometric consistency as described in any one of claims 1 to 7.

Images