Sparse camera array configuration design method and device applied to super-resolution imaging, equipment, medium and product

Abstract

The invention discloses a sparse camera array configuration design method and device applied to super-resolution imaging, equipment, a medium and a product, and relates to the field of sparse camera arrays. The method comprises the following steps: according to the frequency spectrum of a high-resolution image corresponding to each object distance and the continuation frequency spectrum corresponding to each preset frequency domain coordinate, obtaining the weight of the image corresponding to each object distance under the continuation frequency spectrum corresponding to each preset frequency domain coordinate; according to each preset frequency domain coordinate, obtaining an intensity ratio component of the continuation spectrum corresponding to each preset frequency domain coordinate of the array of each configuration under each object distance; obtaining a weighted continuation spectrum intensity ratio corresponding to the array of each configuration under each object distance according to the intensity ratio component and the weight of the continuation spectrum, and obtaining a preset quantile of a curve corresponding to the array of each configuration based on the weighted continuation spectrum intensity ratio; according to the method, the design period of the sparse camera array configuration can be shortened, and the array configuration with the super-resolution imaging performance more stable along with the influence of the object distance is obtained.

Description

[0001] This application claims priority to a Chinese patent application filed with the Patent Office of China on February 13, 2025, with application number 202510161921.4 and invention name “Sparse camera array configuration design method, device, equipment, medium and product for super-resolution imaging”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of sparse camera arrays, and in particular to a sparse camera array configuration design method, apparatus, equipment, medium and product for super-resolution imaging. Background Art

[0003] Because the accuracy of geometric feature positioning directly impacts visual measurement accuracy, high-resolution imaging is crucial for high-precision visual measurement. Imaging system resolution is typically affected by lens aperture and pixel size, the latter of which is the primary factor limiting shortwave infrared imaging resolution. Low spatial sampling frequency of the sensor leads to image aliasing and loss of high-frequency information. Camera array super-resolution imaging systems can capture low-resolution images with sub-pixel displacement and reconstruct high-resolution images. Their intuitive hardware design makes them an effective solution to imaging resolution issues.

[0004] A camera array typically consists of multiple cameras of the same model arranged in a grid pattern, spaced evenly apart, on a plane with a large overlap in their fields of view. The camera array creates sub-pixel displacements between sub-images, enabling super-resolution reconstruction to produce images that exceed the spatial resolution of a single camera. However, existing camera arrays typically consist of dozens or even hundreds of cameras, resulting in high costs and severely limiting their flexibility. To address this, it has been proposed that a grid-arranged camera array, with multiple repetitions in the horizontal and vertical directions and a high level of redundancy in the complementary information, could be designed to achieve equivalent imaging results with fewer cameras, thus creating a sparse camera array. Sparse camera array systems are significantly smaller and lighter than traditional arrays and can be mounted on motion platforms such as gimbals. Therefore, research on sparse camera array configuration design is of great scientific and engineering significance for achieving lightweight, high-performance super-resolution imaging. There is an urgent need for methods that can shorten the design cycle of sparse camera array configurations and achieve array configurations in which super-resolution imaging performance is more stable with respect to object distance. Summary of the Invention

[0005] The purpose of this application is to provide a sparse camera array configuration design method, device, equipment, medium and product for super-resolution imaging, which can shorten the sparse camera array configuration design cycle and obtain an array configuration in which the super-resolution imaging performance is more stable with the influence of object distance.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for designing a sparse camera array configuration for super-resolution imaging, comprising:

[0008] Constructing a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same;

[0009] A three-dimensional scene constructed in a three-dimensional rendering software is rendered using rendering cameras at different object distances to obtain high-resolution images corresponding to the respective object distances; the focal length of the lens of the rendering camera is the same as the focal length of the lens of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the camera in the sparse camera array set; and the resolution of the rendering camera is a preset resolution;

[0010] For a high-resolution image corresponding to any object distance, a frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to each preset frequency domain coordinate is obtained based on the frequency spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate; the extended spectrum corresponding to the preset frequency domain coordinate is each extended spectrum obtained by moving the center of the frequency spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinate;

[0011] For any sparse camera array of any configuration in the sparse camera array set, for any object distance of the rendering camera, obtaining, based on the sub-pixel displacement of each camera in the sparse camera array of the configuration from the reference coordinates at the object distance and each preset frequency domain coordinate, an intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance;

[0012] Obtaining a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance;

[0013] Obtaining a preset quantile of an intensity ratio-object-distance curve corresponding to the sparse camera array of the configuration according to the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio-object-distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio;

[0014] The configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set is determined as the optimal configuration.

[0015] In a second aspect, the present application provides a sparse camera array configuration design device for super-resolution imaging, comprising:

[0016] A sparse camera array set construction module is used to construct a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same;

[0017] A rendering module, configured to render a three-dimensional scene constructed in the three-dimensional rendering software using rendering cameras at different object distances to obtain high-resolution images corresponding to the respective object distances; the focal length of the lens of the rendering camera is the same as the focal length of the lens of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the camera in the sparse camera array set; and the resolution of the rendering camera is a preset resolution;

[0018] a weight calculation module for obtaining, for a high-resolution image corresponding to any object distance, a frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectra corresponding to the preset frequency domain coordinates based on the frequency spectrum of the high-resolution image corresponding to the object distance and the extended spectra corresponding to the preset frequency domain coordinates; the extended spectra corresponding to the preset frequency domain coordinates being the extended spectra obtained by shifting the center of the frequency spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinates;

[0019] a module for calculating an extended spectrum intensity ratio component, configured to obtain, for any sparse camera array of any configuration in the sparse camera array set and for any object distance of the rendering camera, an intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance based on the sub-pixel displacement of each camera in the sparse camera array of the configuration from the reference coordinate at the object distance and each preset frequency domain coordinate;

[0020] a weighted extended spectrum intensity ratio calculation module, configured to obtain a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance;

[0021] a preset quantile calculation module for obtaining a preset quantile of an intensity ratio-object-distance curve corresponding to the sparse camera array of the configuration according to a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio-object-distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio;

[0022] The optimal configuration determination module is used to determine the configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set as the optimal configuration.

[0023] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned sparse camera array configuration design method for super-resolution imaging.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned sparse camera array configuration design method for super-resolution imaging.

[0025] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned sparse camera array configuration design method for super-resolution imaging.

[0026] According to the specific embodiments provided in this application, this application has the following technical effects:

[0027] The present application provides a sparse camera array configuration design method, apparatus, device, medium, and product for super-resolution imaging. Existing camera array configuration research is mostly qualitative. After manually designing the array configuration, an actual imaging system must be built to conduct imaging experiments to verify the configuration's effectiveness, which prolongs the configuration design cycle. The present application uses rendering cameras at different object distances to render a three-dimensional scene constructed in three-dimensional rendering software, obtaining high-resolution images corresponding to each object distance. Based on the high-resolution images corresponding to each object distance and a sparse camera array of each configuration, the weighted extended spectrum intensity ratio corresponding to each object distance for the sparse camera array of each configuration is obtained. The optimal configuration is determined based on the intensity ratio-object-distance curve. This allows the design of the sparse camera array configuration to be completed without building an actual imaging system, shortening the design cycle. Furthermore, the super-resolution imaging performance of a sparse camera array is significantly affected by object distance. The present application considers the overall performance at multiple object distances, thereby achieving relatively stable performance at different object distances and an array configuration whose super-resolution imaging performance is more stable with the influence of object distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A flow chart of a method for designing a sparse camera array configuration for super-resolution imaging provided in an embodiment of the present application;

[0030] Figure 2 Schematic diagram of a sparse camera array configuration design method for super-resolution imaging provided in an embodiment of the present application;

[0031] Figure 3 Schematic diagram of aliasing suppression for multi-camera sampling;

[0032] Figure 4 High-resolution images of the aircraft scene corresponding to different object distances;

[0033] Figure 5 Schematic diagram of the optimal configuration obtained by using the sparse camera array configuration design method for super-resolution imaging provided by this application;

[0034] Figure 6 for Figure 5 Intensity versus object distance plot for the configuration shown;

[0035] Figure 7 A box plot of a variable period fringe scene and a super-resolution image obtained by imaging and reconstructing the variable period fringe scene;

[0036] Figure 8 is a box plot of the resolution plate scene graph and the super-resolution image imaged and reconstructed under the resolution plate scene;

[0037] Figure 9 The imaging scenario and array configuration used in the experiment;

[0038] Figure 10 This is the curve of super-resolution quality-object distance change;

[0039] Figure 11 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] In an exemplary embodiment, a method for designing a sparse camera array configuration for super-resolution imaging is provided. The method is executed by a computer device, specifically, a computer device such as a terminal or a server, or a terminal and a server. The method for designing a sparse camera array configuration for super-resolution imaging is as follows: Figure 1 and Figure 2 As shown, the following steps are included, wherein:

[0043] Step 201: constructing a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same.

[0044] Step 202: Rendering a 3D scene constructed in the 3D rendering software using rendering cameras at different object distances to obtain high-resolution images corresponding to each object distance; the focal length of the rendering camera is the same as the focal length of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the camera in the sparse camera array set; and the resolution of the rendering camera is a preset resolution. The preset resolution is determined based on the super-resolution factor of the multi-image super-resolution reconstruction algorithm in super-resolution imaging. For example, when the super-resolution factor is 2 and the image resolution of the input to the multi-image super-resolution reconstruction algorithm is 640x512, the preset resolution, i.e., the resolution of the image output by the multi-image super-resolution reconstruction algorithm, is (640x2)x(512x2)=1280x1024.

[0045] Step 203: For a high-resolution image corresponding to any object distance, obtain the frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to each preset frequency domain coordinate based on the spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate; the extended spectrum corresponding to the preset frequency domain coordinate is each extended spectrum obtained by moving the center of the spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinate.

[0046] Step 204: For any configuration of a sparse camera array in the sparse camera array set and any object distance of the rendering camera, obtain, based on the sub-pixel displacement of each camera in the sparse camera array of the configuration to a reference coordinate (which is a preset coordinate) at the object distance and each preset frequency domain coordinate, an intensity ratio component of an extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance.

[0047] Step 205: Obtain a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on the intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and the frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance.

[0048] Step 206: Obtain a preset quantile of an intensity ratio-object-distance curve corresponding to the sparse camera array of the configuration according to the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio-object-distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio.

[0049] Step 207: Determine the configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set as the optimal configuration.

[0050] By implementing the above steps 201 to 207 , the design cycle of the sparse camera array configuration can be shortened, and an array configuration with a super-resolution imaging performance that is more stable with respect to the effect of object distance can be obtained.

[0051] In another exemplary embodiment of the present application, before constructing a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set, the method further includes:

[0052] Step 1.1: Determine the performance parameters of the sparse camera array and the array configuration design constraints.

[0053] (1) Determine the camera type, field of view, pixel width, and resolution used in the sparse camera array based on the performance of similar devices and the working environment. At a given imaging distance, determine the lens focal length with the goal of ensuring that the target is within the camera field of view for the entire imaging process. Determine the super-resolution multiple of the multi-image super-resolution reconstruction algorithm used in super-resolution imaging, and derive the preset resolution based on the super-resolution multiple.

[0054] (2) Determine the size of cameras, lenses, and other equipment in the process of building a sparse camera array, as well as the mechanical structure (the spacing between cameras) and other limitations as constraints for the sparse camera array configuration optimization design, with the principle of no collision between cameras. This is generally a well-known method.

[0055] In another exemplary embodiment of the present application, rendering a 3D scene constructed in 3D rendering software using rendering cameras at different object distances to obtain high-resolution images corresponding to each object distance specifically includes:

[0056] Step 2: High-resolution image rendering of the imaging scene.

[0057] (1) Obtain a 3D model of the target and build a simulated imaging scene (3D scene) in 3D rendering software based on the target 3D model. The scene contains the imaging target and its motion trajectory. By adjusting parameters such as the frame rate and time axis, the target motion speed is made consistent with the actual imaging scene. Adjust the rendering camera parameters to make them consistent with the lens focal length and camera field of view determined in step 1, and move the rendering camera to the reference coordinate position.

[0058] (2) The rendering resolution is set to be consistent with the preset resolution, the object distance of the rendering camera is adjusted each time rendering is performed, the rendered image is set to a grayscale image, and the high-resolution image corresponding to each object distance is output.

[0059] In another exemplary embodiment of the present application, the frequency domain energy distribution weight of the high-resolution image corresponding to the object distance in the extended spectrum corresponding to each preset frequency domain coordinate is obtained based on the spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate, specifically:

[0060] According to the formula Calculate the frequency domain energy distribution weight w of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to the kth preset frequency domain coordinate k , where W represents the correction coefficient matrix, which can be a two-dimensional Dirichlet kernel. This is achieved by creating a two-dimensional all-0 array with the same dimension as the high-resolution image and setting an n×n pixel 1 element in the center of the array. The two-dimensional FFT transform of the array is performed and its modulus value is taken, where n is the multiple of the high-resolution image and the actual camera's one-dimensional resolution. represents the Hadamard product operation, F k (u i ,v i ) represents the frequency domain coordinate (u i ,v i ) is the extended spectrum generated by the scene under low-frequency sampling, F g (u i ,v i ) represents the frequency domain coordinate (u i ,v i ) corresponds to the frequency component, G represents the total number of preset frequency domain coordinates, F0(u i ,v i ) represents the frequency domain coordinate (u i ,v i ) corresponding to the frequency component, || 2Indicates the calculation of the square of the absolute value. The specific value of G can be 4, that is, the preset frequency domain coordinates include 4, namely (0, 2π), (2π, 0), (2π, 2π), (2π, -2π). Under this G value, the frequency domain normalized coordinates of the spectrum of the high-resolution image corresponding to the object distance are (u i ,v i )∈[-2π,2π)×[-2π,2π), which is the visible range in the frequency domain at 2x super-resolution. The weight calculation method is a region-by-region statistical analysis of the frequency domain energy distribution of the high-resolution image. This method is stable against spectral changes caused by zooming and detail changes within a certain range of the target.

[0061] In another exemplary embodiment of the present application, before obtaining, based on the sub-pixel displacement of each camera in the sparse camera array of the configuration at the object distance from the reference coordinates and each preset frequency domain coordinate, the intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance is obtained, the method further includes:

[0062] According to the distance from each camera in the sparse camera array of the configuration to the reference coordinate, the pixel width of the camera, the focal length of the lens and the object distance, the sub-pixel displacement from each camera in the sparse camera array of the configuration to the reference coordinate at the object distance is obtained. Specifically, according to the parallax formula Calculates the sub-pixel displacement from the camera to the reference coordinates. B is the distance from the camera to the reference coordinates. When calculating the horizontal sub-pixel displacement from the camera to the reference coordinates, B is the horizontal distance from the camera to the reference coordinates. When calculating the vertical sub-pixel displacement from the camera to the reference coordinates, B is the vertical distance from the camera to the reference coordinates. pix is the pixel width of the camera in the sparse camera array, f is the focal length of the lens in the sparse camera array, and D is the object distance of the rendering camera.

[0063] In another exemplary embodiment of the present application, an intensity ratio component of an extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance is obtained based on the sub-pixel displacement of each camera in the sparse camera array of the configuration at the object distance to the reference coordinate and each preset frequency domain coordinate, specifically:

[0064] According to the formula Calculate the intensity ratio component r of the extended spectrum corresponding to the kth preset frequency domain coordinate of the sparse camera array of the configuration at the object distance k , where u k Indicates the kth preset frequency domain horizontal coordinate, v k Indicates the kth preset frequency domain ordinate, x m represents the horizontal sub-pixel displacement of the mth camera in the sparse camera array of the configuration to the reference coordinate at the object distance, y mrepresents the vertical sub-pixel displacement of the mth camera in the sparse camera array of the configuration to the reference coordinate at the object distance, n represents the total number of cameras in the sparse camera array of the configuration, j represents an imaginary number, and e represents a natural constant.

[0065] The effect of multi-camera sampling on the extended spectrum intensity is as follows Figure 3 Compared with single-camera imaging, multi-camera imaging adds a coordinate-related phase term to the frequency domain sampling function, which causes the equal-intensity periodic repetitive spectrum generated by discretization to become unequal-intensity repetitive, and the extended spectrum intensity is always lower than the original spectrum. Therefore, the ratio of the extended spectrum intensity to the original spectrum intensity is used The degree of aliasing of the signal obtained by camera array sampling can be evaluated.

[0066] In another exemplary embodiment of the present application, a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance is obtained based on the intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and the frequency domain energy distribution weight of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance. Specifically, the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance is obtained:

[0067] According to the formula Calculate the weighted extended spectrum intensity ratio WRRS corresponding to the sparse camera array of the configuration at the object distance, where w k represents the frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to the kth preset frequency domain coordinate, r k represents the intensity ratio component of the extended spectrum corresponding to the kth preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, n represents the total number of cameras in the sparse camera array of the configuration, and G represents the total number of preset frequency domain coordinates.

[0068] In another exemplary embodiment of the present application, the preset quantile is the 95% quantile.

[0069] In another exemplary embodiment of the present application, a preset quantile of an intensity ratio-object distance curve corresponding to the sparse camera array of the configuration is obtained based on the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance, specifically:

[0070] (1) With the object distance as the abscissa and the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance as the ordinate, an intensity ratio object distance curve corresponding to the sparse camera array of the configuration is plotted.

[0071] (2) Calculate the 95% quantile of the intensity-to-object distance curve corresponding to the sparse camera array of the configuration.

[0072] This application also provides a super-resolution imaging device for aircraft scenes. The device consists of three parts: a rigid support module, an image acquisition module, and an image processing module. The overall process of the super-resolution imaging device generally includes: A: Designing a sparse camera array. B: Using the sparse camera array to image the target (aircraft scene), obtaining six images per imaging. C: Processing the six images using a multi-image super-resolution reconstruction algorithm to obtain a single super-resolution image.

[0073] The image acquisition module is a sparse camera array with the optimal configuration determined by the above-mentioned sparse camera array configuration design method for super-resolution imaging. The rigid support module is used to support the image acquisition module; the image acquisition module is used to acquire multiple low-resolution images; and the image processing module is used to achieve image super-resolution reconstruction.

[0074] Specifically, the rigid support module is a metal bracket with a camera mounting structure. The mounting structure ensures that the installation position of each camera meets the optimal configuration requirements of the design. When the super-resolution imaging device is applied to an aircraft scene, the optimal configuration is a sparse camera array suitable for super-resolution imaging of the aircraft scene. The rendered image of the aircraft scene is as follows: Figure 4 The image acquisition module contains 6 short-wave infrared cameras with a pixel size of 15 μ m , the focal length of the lens is 150mm The image processing module is a high-performance computer equipped with an acquisition card. It receives the camera images from the image acquisition module in real time and reconstructs the super-resolution image through the multi-image super-resolution algorithm. The specific configuration of the sparse camera array is as follows: Figure 5 As shown, the sparse camera array intensity versus object distance curve is as follows Figure 6 As shown in the figure, the overall performance curve remains basically stable in the aircraft scenario.

[0075] This application provides an embodiment to illustrate the effectiveness of the above method proposed in this application using the weighted extended spectrum intensity ratio as an indicator:

[0076] Commonly used synthetic datasets for multi-image super-resolution usually include multiple groups of image pairs consisting of a high-resolution image and multiple low-resolution images, where multiple low-resolution images with sub-pixel relative displacement are obtained by downsampling a single high-resolution image, and the downsampling method is usually bicubic interpolation or kernel function interpolation. However, a large number of studies have shown that there is a difference between the images in such synthetic datasets and the real high-low resolution image mapping relationship. The burst super-resolution dataset with real sub-pixel displacement is difficult to provide the specified sub-pixel image displacement required by this application. Therefore, the images used in this embodiment are taken by moving a short-wave infrared camera equipped with a two-axis translation stage. The high-resolution true value uses the real-shot image located at the initial moving point of the translation stage, and the low-resolution image is obtained by downsampling the corresponding displacement image by mean, simulating the large pixel integration effect.

[0077] The super-resolution algorithm uses the MAP (Maximum A-Posteriori) framework combined with the classic non-deep learning multi-image super-resolution algorithm of the BTV (Bilateral Total Variation) regularization term to achieve 2x super-resolution. The PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index Measure) are used to evaluate the quality of the super-resolution image. PSNR and SSIM are common evaluation indicators in the field of super-resolution research. PSNR evaluates the ratio of the maximum possible signal power to the noise power, while SSIM comprehensively evaluates the similarity of the image in brightness, contrast, and image structure. The higher the PSNR and SSIM of the super-resolution image (SSIM does not exceed 1), the closer the super-resolution image is to the true high-resolution image.

[0078] The validity test results of the Weighted Ratio of Repeated Spectra (WRRS) are as follows: Figure 7 and Figure 8 shown.

[0079] Figure 7 The first picture from left to right in part (a) is a 0° rotation variable period stripe scene. Figure 7 The second picture from left to right in part (a) is a 30° rotation variable period stripe scene. Figure 7 The third picture from left to right in part (a) is a 45° rotated variable period stripe scene. Figure 7 The first figure from top to bottom in part (b) is the box-line plot of the PSNR of the 0° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (0, 2π). Figure 7The second figure from top to bottom in part (b) is the box-line plot of the 0° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (0, 2π). Figure 7 The third figure from top to bottom in part (b) is a box plot of the PSNR of the 30° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (0, 2π). Figure 7 The fourth figure from top to bottom in part (b) is a box plot of the 30° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (0, 2π). Figure 7 The fifth figure from top to bottom in part (b) is a box plot of the PSNR of the 45° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (0, 2π). Figure 7 The sixth figure from top to bottom in part (b) is a box plot of the 45° stripe super-resolution image SSIM and the extended spectrum intensity ratio component at the coordinate (0, 2π). Figure 7 The first figure from top to bottom in part (c) is the box-line plot of the PSNR of the 0° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, 0). Figure 7 The second figure from top to bottom in part (c) is the box-line diagram of the 0° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, 0). Figure 7 The third figure from top to bottom in part (c) is a box-line plot of the PSNR of the 30° stripe super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, 0). Figure 7 The fourth figure from top to bottom in part (c) is a box-line plot of the 30° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, 0). Figure 7 The fifth figure from top to bottom in part (c) is a box-line plot of the PSNR of the 45° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, 0). Figure 7 The sixth figure from top to bottom in part (c) is a box plot of the 45° stripe super-resolution image SSIM and the extended spectrum intensity ratio component at the coordinate (2π, 0). Figure 7 The first figure from top to bottom in part (d) is the box-line plot of the PSNR of the 0° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, 2π). Figure 7 The second figure from top to bottom in part (d) is the box-line plot of the 0° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, 2π). Figure 7 The third figure from top to bottom in part (d) is a box-line plot of the PSNR of the 30° stripe super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, 2π). Figure 7 The fourth figure from top to bottom in part (d) is a box plot of the 30° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, 2π). Figure 7The fifth figure from top to bottom in part (d) is a box-line plot of the PSNR of the 45° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, 2π). Figure 7 The sixth figure from top to bottom in part (d) is a box plot of the 45° stripe super-resolution image SSIM and the extended spectrum intensity ratio component of the coordinate (2π, 2π). Figure 7 The first figure from top to bottom in part (e) is the box-line plot of the PSNR of the 0° stripe super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, -2π). Figure 7 The second figure from top to bottom in part (e) is the box-line plot of the 0° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, -2π). Figure 7 The third figure from top to bottom in part (e) is a box plot of the PSNR of the 30° stripe super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, -2π). Figure 7 The fourth figure from top to bottom in the middle (e) is a box-line plot of the 30° stripe super-resolution image SSIM and the extended spectrum intensity ratio of the coordinate (2π, -2π). Figure 7 The fifth figure from top to bottom in part (e) is a box-and-whisker plot of the PSNR of the 45° stripe super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, -2π). Figure 7 The sixth figure from top to bottom in part (e) is a box plot of the 45° stripe super-resolution image SSIM and the extended spectrum intensity ratio component at the coordinate (2π, -2π). Figure 7 The first figure from top to bottom in part (f) is the boxplot of PSNR and fixed weight WWRS of 0° stripe super-resolution image. Figure 7 The second figure from top to bottom in part (f) is the boxplot of SSIM and fixed weight WWRS of 0° stripe super-resolution image. Figure 7 The third figure from top to bottom in part (f) is the boxplot of PSNR and fixed weight WWRS of 30° stripe super-resolution image. Figure 7 The fourth figure from top to bottom in part (f) is the boxplot of SSIM and fixed weight WWRS of 30° stripe super-resolution image. Figure 7 The fifth figure from top to bottom in part (f) is the boxplot of PSNR and fixed weight WWRS of 45° stripe super-resolution image. Figure 7 The sixth figure from top to bottom in part (f) is the boxplot of SSIM and fixed-weight WWRS of 45° stripe super-resolution image. Figure 7 The first figure from top to bottom in part (g) is the PSNR and WWRS boxplot of the 0° stripe super-resolution image. Figure 7 The second figure from top to bottom in part (g) is the SSIM and WWRS boxplot of the 0° stripe super-resolution image. Figure 7 The third figure from top to bottom in part (g) is the PSNR and WWRS boxplot of the 30° stripe super-resolution image. Figure 7The fourth figure from top to bottom in part (g) is the SSIM and WWRS boxplot of the 30° stripe super-resolution image. Figure 7 The fifth figure from top to bottom in part (g) is the PSNR and WWRS boxplot of the 45° stripe super-resolution image. Figure 7 The sixth figure from top to bottom in part (g) is the SSIM and WWRS boxplot of the 45° stripe super-resolution image.

[0080] Figure 8 The first picture from left to right in part (a) is the full-resolution plate imaging scene picture. Figure 8 The second picture from left to right in part (a) is a partial resolution plate imaging scene picture. Figure 8 The first figure from top to bottom in part (b) is the box-line plot of the PSNR of the full-resolution super-resolution image and the extended spectrum intensity ratio component at the coordinate (0, 2π). Figure 8 The second figure from top to bottom in part (b) is the box-line plot of the SSIM component of the super-resolution image of the full-resolution plate and the extended spectrum intensity ratio of the coordinate (0, 2π). Figure 8 The third figure from top to bottom in part (b) is a box-line plot of the PSNR of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (0, 2π). Figure 8 The fourth figure from top to bottom in part (b) is a box-and-whisker plot of the SSIM component of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (0, 2π). Figure 8 The first figure from top to bottom in part (c) is the box-line plot of the PSNR of the full-resolution super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, 0). Figure 8 The second figure from top to bottom in part (c) is the box-line plot of the ratio of the SSIM of the full-resolution plate super-resolution image to the extended spectrum intensity of the coordinate (2π, 0). Figure 8 The third figure from top to bottom in part (c) is a box-line plot of the PSNR of the partial resolution plate super-resolution image and the extended spectrum intensity ratio component of the coordinate (2π, 0). Figure 8 The fourth figure from top to bottom in part (c) is a box-and-whisker plot of the SSIM component of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component at the coordinate (2π, 0). Figure 8 The first figure from top to bottom in part (d) is the box-line plot of the PSNR of the full-resolution plate super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, 2π). Figure 8 The second figure from top to bottom in part (d) is the box-line plot of the SSIM component of the super-resolution image of the full-resolution plate and the extended spectrum intensity ratio of the coordinate (2π, 2π). Figure 8 The third figure from top to bottom in part (d) is a box-line plot of the PSNR of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (2π, 2π). Figure 8The fourth figure from top to bottom in part (d) is a box-and-whisker plot of the SSIM component of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (2π, 2π). Figure 8 The first figure from top to bottom in part (e) is the box-line plot of the PSNR of the full-resolution super-resolution image and the extended spectrum intensity ratio component at the coordinate (2π, -2π). Figure 8 The second figure from top to bottom in part (e) is the box-line plot of the ratio of the SSIM of the full-resolution plate super-resolution image to the extended spectrum intensity of the coordinate (2π, -2π). Figure 8 The third figure from top to bottom in part (e) is a box-line plot of the PSNR of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (2π, -2π). Figure 8 The fourth figure from top to bottom in part (e) is a box-and-whisker plot of the SSIM component of the super-resolution image of the partial resolution plate and the extended spectrum intensity ratio component of the coordinate (2π, -2π). Figure 8 The first figure from top to bottom in the middle (f) is the PSNR and WWRS box-whisker plot of the full-resolution plate super-resolution image. Figure 8 The second figure from top to bottom in part (f) is the SSIM and WWRS box-whisker plot of the full-resolution plate super-resolution image. Figure 8 The third figure from top to bottom in part (f) is the PSNR and WWRS box-whisker plot of the super-resolution image of some resolution plates. Figure 8 The fourth figure from top to bottom in part (f) is the SSIM and WWRS box plot of the partial resolution plate super-resolution image.

[0081] The two figures show the results of super-resolution reconstruction by taking 200 random low-resolution images with known displacements in the variable period stripe and resolution plate scenes, and randomly selecting 2000 sets of non-repeated image combinations. The images use box plots to statistically analyze the statistical correlation between super-resolution quality and WRRS and its components. The outliers represented by the cross in the figure do not exceed 10% of the corresponding samples. It can be seen that there is a significant correlation between WRRS and the median of super-resolution quality (the center horizontal line of the box in the box plot) in all scenes. Further, Figure 7 Part (b) to Figure 7 Part (e) shows the statistical correlation between super-resolution quality and each WRRS component. It can be seen that the variable-period fringe image and the WRRS components in the corresponding direction show a strong statistical correlation, while the correlation in other directions is relatively weak. This indicates that super-resolution quality is the result of the combined effect of the target scene and the camera array configuration. This application proposes a weighted extended spectrum intensity ratio metric that can effectively integrate these two effects to predict super-resolution quality.

[0082] Select Figure 9 The imaging scene shown in part (a) is the optimal configuration ( Figure 9The proposed configuration in part (b)) and other configurations such as Figure 9 Comparing the first four configurations in part (b), the curve of super-resolution mass-object distance variation is shown as Figure 10 As shown, Figure 10 The first figure from left to right in part (a) is the WRRS-object distance curve of the full-resolution plate and each array configuration. Figure 10 The second figure from left to right in part (a) is the PSNR-object distance curve of the super-resolution image of the full-resolution plate. Figure 10 The third figure from left to right in part (a) is the super-resolution image SSIM-object distance curve of the full-resolution plate. Figure 10 The first figure from left to right in part (b) is the WRRS-object distance curve of some resolution plates and various array configurations. Figure 10 The second figure from left to right in part (b) is the PSNR-object distance curve of the super-resolution image of some resolution plates. Figure 10 The third figure from left to right in part (b) is the super-resolution image SSIM-object distance curve of some resolution plates. Figure 10 The first figure from left to right in part (c) is the WRRS-object distance curve of the aircraft model and each array configuration. Figure 10 The second figure from left to right in part (c) is the PSNR-object distance curve of the super-resolution image of the aircraft model. Figure 10 The third figure from left to right in section (c) shows the SSIM vs. object distance curve for a super-resolution image of an aircraft model. Because it's difficult to capture moving objects at object distances between 200m and 1000m, the images captured here use a translation stage to obtain the equivalent object distance and sub-pixel displacement for the configuration. Compared to the previous three configurations, the optimal configuration obtained using the sparse camera array configuration design method for super-resolution imaging provided in this application avoids the sharp performance drops caused by periodic integer sub-pixel displacements in the horizontal and vertical directions, such as the local minima of the curves at distances of 300m, 600m, and 900m. The intensity-to-object distance curves for the 90-degree variant and the proposed configuration show consistent performance in general scenarios, but diverge in the aircraft scenario. The actual super-resolution quality curves are consistent with the predictions. In the aircraft scenario, the proposed configuration consistently outperforms its 90-degree rotation variant at distances between 700m and 1000m, demonstrating that the sparse camera array configuration design method for super-resolution imaging provided in this application can effectively design array configurations based on the imaging scenario.

[0083] The high super-resolution imaging performance of the sparse camera array provided in this application is derived primarily from the following two aspects: 1) Using the weighted extended spectrum intensity ratio as a quantitative indicator, it is possible to quantitatively predict the super-resolution performance of a camera array placed at any position in a specific scenario. 2) By combining object distance variations with aircraft scenes in the configuration design, an optimized 6-camera sparse camera array configuration is obtained that maintains good performance throughout the entire imaging process. The proposed configuration achieves a relatively uniform distribution of vertical sampling points under varying object distances, which is consistent with the characteristics of scenes with relatively rich vertical frequency components of aircraft, thereby achieving uniform and good super-resolution imaging performance.

[0084] The weighted extended spectrum intensity ratio uses the statistical distribution of frequency domain energy in each region of the high-resolution image as a weight, and performs a weighted synthesis of the image signal sampling capabilities of the sparse camera array in each direction reflected by the extended spectrum intensity ratio. The weighted extended spectrum intensity ratio can effectively predict the super-resolution image performance of the camera array. The sparse camera array configuration design method for super-resolution imaging provided in this application uses the weighted extended spectrum intensity ratio as an evaluation index, combined with imaging parameters such as camera focal length, pixel size, and imaging distance, to design a sparse camera array configuration for a specific scene, thereby achieving the design of the sparse camera array configuration without the need to build an actual imaging system, which can shorten the design cycle of such imaging systems.

[0085] Starting from signal sampling, this application proposes a sparse camera array configuration design method for super-resolution imaging with the goal of reducing image signal aliasing and improving super-resolution imaging performance. This application can predict the super-resolution performance of sparse camera arrays of various configurations in a given scenario through the weighted extended spectrum intensity ratio.

[0086] Related techniques use the MAP super-resolution framework to derive a Gaussian process with array configuration as a variable, obtain a configuration optimization function, and design a sparse camera array for isotropic scenes. This is a quantitative design method. However, this method does not consider changes in the imaging scene caused by target motion, limiting its application in dynamic scenes. The present application addresses this shortcoming through steps 206 and 207.

[0087] A related art study proposes a 5-camera sparse camera array. This array is a simplified version of a 5×5 dense camera array. Its design strategy is to avoid horizontal and vertical overlap while increasing the baseline distance between cameras. This method is qualitative and lacks quantitative evaluation. Furthermore, it cannot horizontally compare the performance of arbitrary sparse camera array configurations with unrestricted camera positions. This application addresses this shortcoming by using the weighted extended spectrum intensity ratio as an indicator.

[0088] Related art proposes a four-camera array that uses an additional mechanical structure to control the cameras to capture multiple images with random jitter for super-resolution reconstruction. The starting point is to obtain complementary information beneficial for super-resolution through a sufficient number of random samples. This method does not control the camera array configuration and is also a qualitative design. The present application addresses this shortcoming by using the weighted extended spectrum intensity ratio as an indicator.

[0089] Related art also proposes a design method for proportionally simplifying a dense grid-shaped camera array. This method only retains cameras in the dense array whose image displacement distribution is nearly uniform. The array designed using this method exhibits significant imaging distance dependence. As the imaging distance changes, the uniform distribution of image displacement is disrupted, resulting in a decrease in the array's super-resolution performance. Furthermore, this method is a qualitative design approach. The present application addresses this shortcoming through steps 206 and 207.

[0090] Related technologies have proposed a camera array super-resolution imaging system that combines low-resolution cameras with SLR cameras. This system uses the SLR camera's high-resolution image as a priori information to aid in super-resolution reconstruction. However, for non-visible light cameras like short-wave infrared, the resolution is limited by manufacturing processes, and the lack of high-resolution models comparable to SLR cameras makes constructing such an array difficult. This application addresses this shortcoming by rendering high-resolution images and using the weighted extended spectrum intensity ratio as a metric.

[0091] Related technologies have proposed a method for achieving super-resolution imaging using random displacement of a single camera. This method has good imaging results on mobile devices, but requires additional mechanical structures to generate random displacement on fixed imaging devices. The multiple exposures required by this method also somewhat reduce the dynamic imaging capabilities of the imaging system. This application addresses this shortcoming by eliminating the need to construct an actual configuration, only a virtual one and performing calculations.

[0092] Based on the same inventive concept, embodiments of the present application also provide a sparse camera array configuration design device for super-resolution imaging, which is used to implement the aforementioned sparse camera array configuration design method for super-resolution imaging. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the sparse camera array configuration design device for super-resolution imaging provided below can be found in the above-mentioned limitations of the sparse camera array configuration design method for super-resolution imaging, and will not be further elaborated here.

[0093] In an exemplary embodiment, a sparse camera array configuration design apparatus for super-resolution imaging is provided, comprising:

[0094] A sparse camera array set construction module is used to construct multiple sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same.

[0095] A rendering module is configured to render a three-dimensional scene constructed in three-dimensional rendering software using rendering cameras at different object distances to obtain high-resolution images corresponding to each object distance; the focal length of the rendering camera's lens is the same as the focal length of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the cameras in the sparse camera array set; and the resolution of the rendering camera is a preset resolution.

[0096] A weight calculation module is used to obtain, for a high-resolution image corresponding to any object distance, a frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to each preset frequency domain coordinate based on the spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate; the extended spectrum corresponding to the preset frequency domain coordinate is each extended spectrum obtained by moving the center of the spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinate.

[0097] The extended spectrum intensity ratio component calculation module is used to obtain, for any sparse camera array configuration in the sparse camera array set and for any object distance of the rendering camera, the intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array configuration at the object distance based on the sub-pixel displacement of each camera in the sparse camera array configuration to the reference coordinate at the object distance and each preset frequency domain coordinate.

[0098] A weighted extended spectrum intensity ratio calculation module is configured to obtain a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance.

[0099] A preset quantile calculation module is used to obtain a preset quantile of an intensity ratio object distance curve corresponding to the sparse camera array of the configuration according to the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio object distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio.

[0100] The optimal configuration determination module is used to determine the configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set as the optimal configuration.

[0101] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sparse camera array configuration design data for super-resolution imaging. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sparse camera array configuration design method for super-resolution imaging is implemented.

[0102] Those skilled in the art will understand that Figure 11 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0103] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above-mentioned method embodiments when executed by a processor.

[0104] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above method embodiments are implemented.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0106] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0107] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0108] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A sparse camera array configuration design method for super-resolution imaging, characterized in that: The sparse camera array configuration design method for super-resolution imaging includes: Constructing a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same; A three-dimensional scene constructed in a three-dimensional rendering software is rendered using rendering cameras at different object distances to obtain high-resolution images corresponding to the respective object distances; the focal length of the lens of the rendering camera is the same as the focal length of the lens of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the camera in the sparse camera array set; and the resolution of the rendering camera is a preset resolution; For a high-resolution image corresponding to any object distance, a frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to each preset frequency domain coordinate is obtained based on the frequency spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate; the extended spectrum corresponding to the preset frequency domain coordinate is each extended spectrum obtained by moving the center of the frequency spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinate; For any sparse camera array of any configuration in the sparse camera array set, for any object distance of the rendering camera, obtaining, based on the sub-pixel displacement of each camera in the sparse camera array of the configuration from the reference coordinates at the object distance and each preset frequency domain coordinate, an intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance; Obtaining a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance; Obtaining a preset quantile of an intensity ratio-object-distance curve corresponding to the sparse camera array of the configuration according to the weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio-object-distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio; The configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set is determined as the optimal configuration.

2. The method for designing a sparse camera array configuration for super-resolution imaging according to claim 1, wherein: The frequency domain energy distribution weight of the high-resolution image corresponding to the object distance in the extended spectrum corresponding to each preset frequency domain coordinate is obtained according to the frequency spectrum of the high-resolution image corresponding to the object distance and the extended spectrum corresponding to each preset frequency domain coordinate, specifically: According to the formula Calculate the frequency domain energy distribution weight w of the high-resolution image corresponding to the object distance under the extended spectrum corresponding to the kth preset frequency domain coordinate k , where W represents the correction coefficient matrix, represents the Hadamard product operation, F k (u i ,v i ) represents the frequency domain coordinate (u i ,v i ) corresponding frequency component, F g (u i ,v i ) represents the frequency domain coordinate (u i ,v i ) corresponding to the frequency component, G represents the total number of preset frequency domain coordinates, || 2 Calculates the square of the absolute value.

3. The method for designing a sparse camera array configuration for super-resolution imaging according to claim 1, wherein: Before obtaining, based on the sub-pixel displacement of each camera in the sparse camera array of the configuration at the object distance from the reference coordinate and each preset frequency domain coordinate, the intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance is obtained, the method further includes: The sub-pixel displacement of each camera in the sparse camera array of the configuration from the reference coordinate at the object distance is obtained according to the distance from each camera in the sparse camera array of the configuration to the reference coordinate, the pixel width of the camera, the focal length of the lens, and the object distance.

4. The method for designing a sparse camera array configuration for super-resolution imaging according to claim 1, wherein: According to the sub-pixel displacement of each camera in the sparse camera array of the configuration at the object distance to the reference coordinate and each preset frequency domain coordinate, the intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance is obtained, specifically: According to the formula Calculate the intensity ratio component r of the extended spectrum corresponding to the kth preset frequency domain coordinate of the sparse camera array of the configuration at the object distance k , where u k Indicates the kth preset frequency domain horizontal coordinate, v k Indicates the kth preset frequency domain ordinate, x m represents the horizontal sub-pixel displacement of the mth camera in the sparse camera array of the configuration to the reference coordinate at the object distance, y m represents the vertical sub-pixel displacement of the mth camera in the sparse camera array of the configuration to the reference coordinate at the object distance, n represents the total number of cameras in the sparse camera array of the configuration, j represents an imaginary number, and e represents a natural constant.

5. The method for designing a sparse camera array configuration for super-resolution imaging according to claim 1, wherein: According to the intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and the frequency domain energy distribution weights of the high-resolution image corresponding to the object distance in the extended spectrum corresponding to each preset frequency domain coordinate, a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance is obtained, specifically: According to the formula Calculate the weighted extended spectrum intensity ratio WRRS corresponding to the sparse camera array of the configuration at the object distance, where wk represents the frequency domain energy distribution weight of the extended spectrum corresponding to the kth preset frequency domain coordinate of the high-resolution image corresponding to the object distance, and r k represents the intensity ratio component of the extended spectrum corresponding to the kth preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, n represents the total number of cameras in the sparse camera array of the configuration, and G represents the total number of preset frequency domain coordinates.

6. The method for designing a sparse camera array configuration for super-resolution imaging according to claim 1, wherein: The preset quantile is the 95% quantile.

7. A sparse camera array configuration design device for super-resolution imaging, characterized in that: The sparse camera array configuration design device for super-resolution imaging includes: A sparse camera array set construction module is used to construct a plurality of sparse camera arrays of different configurations to obtain a sparse camera array set; the pixel width of each camera in the sparse camera array set is the same, the field of view of each camera in the sparse camera array set is the same, the resolution of each camera in the sparse camera array set is the same, and the focal length of each lens in the sparse camera array set is the same; A rendering module, configured to render a three-dimensional scene constructed in the three-dimensional rendering software using rendering cameras at different object distances to obtain high-resolution images corresponding to the respective object distances; the focal length of the lens of the rendering camera is the same as the focal length of the lens of the lens in the sparse camera array set; the field of view of the rendering camera is the same as the field of view of the camera in the sparse camera array set; and the resolution of the rendering camera is a preset resolution; a weight calculation module for obtaining, for a high-resolution image corresponding to any object distance, a frequency domain energy distribution weight of the high-resolution image corresponding to the object distance under the extended spectra corresponding to the preset frequency domain coordinates based on the frequency spectrum of the high-resolution image corresponding to the object distance and the extended spectra corresponding to the preset frequency domain coordinates; the extended spectra corresponding to the preset frequency domain coordinates being the extended spectra obtained by shifting the center of the frequency spectrum of the high-resolution image corresponding to the object distance to the preset frequency domain coordinates; a module for calculating an extended spectrum intensity ratio component, configured to obtain, for any sparse camera array of any configuration in the sparse camera array set and for any object distance of the rendering camera, an intensity ratio component of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance based on the sub-pixel displacement of each camera in the sparse camera array of the configuration from the reference coordinate at the object distance and each preset frequency domain coordinate; a weighted extended spectrum intensity ratio calculation module, configured to obtain a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at the object distance based on intensity ratio components of the extended spectrum corresponding to each preset frequency domain coordinate of the sparse camera array of the configuration at the object distance, and frequency domain energy distribution weights of the extended spectrum corresponding to each preset frequency domain coordinate of the high-resolution image corresponding to the object distance; a preset quantile calculation module for obtaining a preset quantile of an intensity ratio-object-distance curve corresponding to the sparse camera array of the configuration according to a weighted extended spectrum intensity ratio corresponding to the sparse camera array of the configuration at each object distance; the abscissa of the intensity ratio-object-distance curve is the object distance, and the ordinate is the weighted extended spectrum intensity ratio; The optimal configuration determination module is used to determine the configuration of the sparse camera array with the smallest preset quantile of the intensity-to-object-distance curve in the sparse camera array set as the optimal configuration.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the sparse camera array configuration design method for super-resolution imaging according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for designing a sparse camera array configuration for super-resolution imaging according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for designing a sparse camera array configuration for super-resolution imaging according to any one of claims 1 to 6 is implemented.

Images