Imaging system evaluation method and electronic device

Abstract

This application provides an imaging system evaluation method and electronic device. The method mainly uses an imaging system to capture images of real scenes. While using the real scene images as test images, a high-quality image is also generated based on the real scene images as a reference image. Finally, the test image and the reference image are converted to the frequency domain to calculate frequency information and obtain a modulation transfer function curve that can reflect the imaging capability of the imaging system. In this way, the degree of reproduction of the real scene by the images captured by the imaging system can be effectively evaluated, making the evaluation results more reliable.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to an imaging system evaluation method and electronic equipment. Background Technology

[0002] With the continuous development of electronic devices, people have become accustomed to using electronic devices to take photos and record videos to document their lives. The imaging capabilities of electronic devices can be reflected by parameters such as image resolution and contrast. From the user's perspective, this can be expressed as the degree to which the image reproduces the real object being photographed and the level of texture detail in the image.

[0003] It is evident that the imaging system capabilities of electronic devices can impact the user experience. Therefore, manufacturers evaluate the imaging system of electronic devices during the research and development process or before the devices leave the factory to ensure that the imaging system capabilities of the electronic devices meet user needs as much as possible.

[0004] However, existing imaging system evaluation methods have large errors, resulting in low accuracy of evaluation results. Summary of the Invention

[0005] This application provides an imaging system evaluation method and electronic device, which can improve the accuracy of imaging system evaluation results.

[0006] In a first aspect, embodiments of this application provide an imaging system evaluation method, which includes: acquiring a test image and generating a reference image based on the test image, wherein the test image is obtained by the imaging system capturing a real scene, the reference image contains the same image content as the test image, and the image quality of the reference image is higher than that of the test image; converting the test image and the reference image to the frequency domain to perform frequency information calculation, generating a modulation transfer function curve corresponding to the test image in the frequency domain, the modulation transfer function curve being used to indicate the imaging capability of the imaging system.

[0007] In the technical solution of this application, real scene images are mainly obtained by imaging system. While using the real scene image as a test image, a high-quality image is also generated based on the real scene image as a reference image. Finally, the test image and the reference image are converted to the frequency domain to calculate frequency information and obtain the modulation transfer function curve that can reflect the imaging capability of the imaging system. In this way, the degree of restoration of the real scene by the image captured by the imaging system can be effectively evaluated, making the evaluation results more reliable.

[0008] Optionally, in one possible implementation of the first aspect, the above-mentioned conversion of the test image and reference image to the frequency domain for frequency information calculation to generate the modulation transfer function curve corresponding to the test image in the frequency domain includes: performing discrete Fourier transform on the test image and reference image respectively to obtain first frequency domain information corresponding to the test image and second frequency domain information corresponding to the reference image; determining multiple first modulation transfer function values ​​of the test image in the frequency domain based on the first frequency domain information and the second frequency domain information; and generating a modulation transfer function curve based on the multiple first modulation transfer function values ​​of the test image in the frequency domain.

[0009] It should be understood that the Discrete Fourier Transform (DFT) can transform an image from a spatial domain described by pixel coordinates and gray values ​​to a frequency domain described by spatial frequency, spectral coordinates, and spectral intensity, facilitating subsequent analysis of image quality and evaluation of the imaging system. It should also be understood that the frequency involved in the embodiments of this application refers to the image frequency, also known as spatial frequency, which reflects the variation of the gray values ​​corresponding to pixels in the image at different pixel locations.

[0010] Optionally, in another possible implementation of the first aspect, determining multiple first modulation transfer function values ​​of the test image in the frequency domain based on the first frequency domain information and the second frequency domain information includes: determining the first power spectral density of the test image at each frequency based on the first frequency domain information, and determining the second power spectral density of the reference image at each frequency based on the second frequency domain information; and determining the ratio of the first power spectral density to the second power spectral density at each frequency as the first modulation transfer function value of the test image in the frequency domain.

[0011] It should be understood that power spectral density is used to describe how signal power varies with frequency. For images, power spectral density can show the energy distribution of an image at different frequencies, thus helping to analyze image details and noise characteristics.

[0012] Optionally, in another possible implementation of the first aspect, determining multiple first modulation transfer function values ​​of the test image in the frequency domain based on the first frequency domain information and the second frequency domain information includes: performing autocorrelation calculation based on the second frequency domain information to obtain autocorrelation function values ​​at each frequency; performing cross-correlation calculation based on the first frequency domain information and the second frequency domain information to obtain cross-correlation function values ​​at each frequency; and determining the ratio of the autocorrelation function value to the cross-correlation function value at each frequency as the first modulation transfer function value of the test image in the frequency domain.

[0013] It should be understood that the autocorrelation function is a function that describes the similarity between a signal and itself at different time delays. In the embodiments of this application, the autocorrelation function can be used to analyze the repetition patterns and structural characteristics of an image. The cross-correlation function is a function that describes the similarity between two signals. In the embodiments of this application, the cross-correlation function is used to compare the similarity between a reference image and a test image.

[0014] Optionally, in another possible implementation of the first aspect, each first modulation transfer function value includes function values ​​in multiple frequency directions. The above-mentioned generation of modulation transfer function curves based on multiple first modulation transfer function values ​​of the test image in the frequency domain includes: averaging the function values ​​in all frequency directions included in each first modulation transfer function value to obtain a second modulation transfer function value corresponding to each first modulation transfer function value; and generating a modulation transfer function curve based on each second modulation transfer function value.

[0015] It should be understood that the frequency components of an image are typically directional, meaning that the frequency response may differ in different directions. For example, the first modulation transfer function value at a certain frequency contains function components in both the horizontal and vertical frequency directions. By averaging the function values ​​in different frequency directions, the performance of the imaging system across all frequency directions can be more intuitively reflected.

[0016] Optionally, in another possible implementation of the first aspect, generating a modulation transfer function curve based on each second modulation transfer function value includes: smoothing each second modulation transfer function value to obtain a third modulation transfer function value corresponding to each second modulation transfer function value; and generating a modulation transfer function curve based on each third modulation transfer function value. The smoothing process is used to eliminate noise and outliers in the data and extract the basic trends and characteristics of the data.

[0017] Optionally, in another possible implementation of the first aspect, after generating the reference image based on the test image, the method further includes: performing at least one alignment operation between the test image and the reference image, namely, brightness alignment, color alignment, and spatial alignment.

[0018] It should be understood that brightness alignment can eliminate brightness differences caused by varying lighting conditions. Color alignment ensures the consistency of the test image and the reference image in color space. Spatial alignment ensures the consistency of the test image and the reference image in spatial location.

[0019] Optionally, in another possible implementation of the first aspect, the method further includes: when the test image is a non-linear image, determining flat regions in the test image, and linearizing the test image based on the flat regions in the test image.

[0020] Optionally, in another possible implementation of the first aspect, the method further includes: when the reference image is a nonlinear image, determining a flat region in the reference image, and linearizing the reference image based on the flat region in the reference image.

[0021] It should be understood that, for the convenience of subsequent frequency domain analysis, if the test image and reference image are nonlinear, they need to be converted into linearized images respectively. In addition, to avoid noise introduced by complex textures or lighting variations, flat regions in the test image and reference image are usually selected, and then linearized based on the flat regions.

[0022] Optionally, in another possible implementation of the first aspect, the above-mentioned generation of a reference image based on the test image includes: acquiring multiple frames of first images, which are obtained by the imaging system capturing multiple frames of a real scene, and the multiple frames of first images include the test image; fusing the multiple frames of first images to obtain a second image; and performing image signal processing on the second image to obtain a reference image. Thus, through the above-mentioned multi-frame fusion and image processing steps, the final reference image has better image quality and detail, facilitating subsequent frequency information calculation.

[0023] Optionally, in another possible implementation of the first aspect, fusing multiple frames of the first image to obtain the second image includes: fusing multiple frames of the first image using one or more of the following methods: inter-frame registration, ghost detection, and super-resolution, to obtain the second image.

[0024] Optionally, in another possible implementation of the first aspect, the above image signal processing includes one or more of the following: black level correction, white balance gain, demosaic, color restoration, and tone mapping.

[0025] Optionally, in another possible implementation of the first aspect, the method further includes: determining flat regions in the test image and flat regions in the reference image respectively; performing noise estimation on the test image based on the flat regions in the test image and the flat regions in the reference image to obtain a noise estimation result, wherein the noise estimation result and / or the modulation transfer function curve are used to indicate the imaging capability of the imaging system. In addition to analyzing the imaging capability of the imaging system using the modulation transfer function curve, the imaging capability of the imaging system can also be analyzed by comparing the noise estimation results of the test image and the reference image.

[0026] Secondly, embodiments of this application provide an imaging system evaluation apparatus, which includes a unit composed of software and / or hardware for performing the imaging system evaluation method of the first aspect.

[0027] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device is able to implement any of the methods described in the first aspect above.

[0028] Fourthly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to enable the electronic device to execute any of the methods described in the first aspect.

[0029] Optionally, the chip system may also include a memory electrically connected to the processor.

[0030] Optionally, the chip system may also include a communication interface.

[0031] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, enable the electronic device to perform any of the methods described in the first aspect.

[0032] Sixthly, embodiments of this application provide a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device is able to implement any of the methods described in the first aspect above. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram illustrating an applicable scenario of an embodiment of this application;

[0035] Figure 2 This is a schematic flowchart of an imaging system evaluation method provided in an embodiment of this application;

[0036] Figure 3 This is an example diagram of an image segmentation algorithm based on a quadtree provided in an embodiment of this application;

[0037] Figure 4 This is a schematic flowchart illustrating the process of determining the value of the first modulation transfer function provided in an embodiment of this application;

[0038] Figure 5This is a schematic diagram of a reference image and three test images provided in an embodiment of this application;

[0039] Figure 6 This is a schematic diagram of the modulation transfer function curve provided in the embodiments of this application;

[0040] Figure 7 This is a schematic diagram of the structure of an imaging system evaluation device provided in an embodiment of this application;

[0041] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0043] With the continuous development of electronic devices, people have become accustomed to using electronic devices to take photos and record videos to document their lives. The imaging capabilities of electronic devices can be reflected by parameters such as image resolution and contrast. From the user's perspective, this can be expressed as the degree to which the image reproduces the real object being photographed and the level of texture detail in the image.

[0044] It is evident that the imaging system capabilities of electronic devices can impact the user experience. Therefore, manufacturers evaluate the imaging system of electronic devices during the research and development process or before the devices leave the factory to ensure that the imaging system capabilities of the electronic devices meet user needs as much as possible.

[0045] The modulation transfer function (MTF) describes an imaging system's ability to transfer image details at different frequencies, and is therefore frequently used to evaluate the imaging capabilities of such systems. Traditionally, when evaluating an imaging system using the MTF, a test image is obtained by capturing a fixed pattern (i.e., a reference image, such as a hypotenuse plot, Siemens star plot, or dead leaf plot) using the imaging system. The MTF curve is then generated by comparing the test image and the fixed image. However, these fixed patterns differ significantly from real-world scenes, and require high image precision. This makes it difficult to accurately assess the degree to which the image captured by the imaging system reproduces the real-world scene, resulting in low accuracy in the evaluation results.

[0046] In view of this, embodiments of this application provide an imaging system evaluation method, which mainly obtains real scene images by the imaging system, uses the real scene images as test images, and generates high-quality images based on the real scene images as reference images. Finally, the test images and reference images are converted to the frequency domain to calculate frequency information, thereby obtaining a modulation transfer function curve that can reflect the imaging capability of the imaging system. In this way, the degree of reproduction of the real scene by the images captured by the imaging system can be effectively evaluated, making the evaluation results more accurate.

[0047] Figure 1 This is a schematic diagram illustrating an applicable scenario of an embodiment of this application. For example... Figure 1 As shown, the scenario includes at least a first electronic device 110 and a second electronic device 120. The second electronic device 120 includes the imaging system to be evaluated, and the first electronic device 110 includes computer program code for implementing the technical solution provided in this application.

[0048] In one possible implementation, the second electronic device 120 can automatically capture test images and transmit the captured test images to the first electronic device 110. The first electronic device 110 can run stored computer program code to evaluate its imaging system using the test images captured by the second electronic device 120.

[0049] Figure 1 The first electronic device 110 is illustrated as a desktop computer, and the second electronic device 120 is illustrated as a mobile phone. It should be understood that the embodiments of this application do not limit the types of the first electronic device 110 and the second electronic device 120. The first electronic device 110 can be any device whose computing power meets the program code execution requirements of the technical solution provided in this application, and the second electronic device 120 can be any device with imaging capabilities. For example, in other embodiments of this application, the first electronic device 110 or the second electronic device 120 can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, laptop, ultra-mobile personal computer (UMPC), netbook, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices, etc.

[0050] It should be understood that the second electronic device may be a mobile phone, tablet computer, or other device that includes the imaging system to be evaluated, or it may be the imaging system to be evaluated itself. This application does not limit this.

[0051] In another embodiment, the second electronic device itself may include computer program code for implementing the technical solutions provided in this application. That is, the second electronic device can independently realize the automatic capture of test images and realize the evaluation of the imaging system based on the test images.

[0052] Reference Figure 2 The diagram illustrates a flowchart of an imaging system evaluation method provided in an embodiment of this application. The following section... Figure 2 The steps shown will be explained.

[0053] Step 201: Obtain the test image and generate a reference image based on the test image.

[0054] In this embodiment, the test image is obtained by capturing a real scene using an imaging system. The reference image and the test image contain the same image content, but the image quality of the reference image is higher than that of the test image.

[0055] Alternatively, the imaging system can be a high-quality camera, such as a DSLR camera, which offers excellent image quality and flexible lens options, enabling it to capture more detail and a higher dynamic range.

[0056] In one embodiment, the reference image can be obtained through a series of processes, including multi-frame shooting, multi-frame fusion, and image processing of a real scene by an imaging system. Specifically, firstly, multiple first images are acquired by the imaging system shooting the real scene in multiple frames, which include the test image. Then, the multiple first images are fused to obtain a second image. Finally, image signal processing (ISP) is performed on the second image to obtain the reference image. Through the above steps of multi-frame fusion and image processing, the final reference image has good image quality and detail, which facilitates subsequent frequency information calculation.

[0057] During multi-frame shooting of a real-world scene, the optimal focusing distance of the imaging system can be fixed to ensure the focus remains on the desired scene, preventing blurry initial images due to focus changes. Additionally, the priority exposure time can be fixed to select an appropriate exposure time, ensuring the initial image is neither overexposed nor underexposed, especially in environments with significant lighting variations. A tripod can be used if necessary to effectively reduce imaging system shake and ensure a sharp initial image, particularly in low-light conditions.

[0058] In one embodiment, the process of fusing multiple frames of the first image to obtain the second image can be achieved using one or more of the following methods: inter-frame registration, ghost detection, denoising, and super-resolution. Inter-frame registration is the process of aligning images captured at different times or from different perspectives. This can be achieved through methods such as feature point matching and optical flow. Accurate inter-frame registration can reduce image deviations caused by imaging system jitter or scene changes. In dynamic scenes, moving objects may create ghosting in the image, affecting the quality of the final image. Ghost detection can help identify these ghostings and remove them using methods such as weighted averaging, improving image clarity and realism. Denoising can remove noise generated during the shooting process. For example, spatial denoising (such as median filtering) and frequency domain denoising (such as wavelet transform) techniques can be used to smooth the image without losing detail. Super-resolution is a technique that generates a high-resolution image using multiple frames of low-resolution images. By combining multiple frames of the first image and employing image reconstruction, interpolation algorithms, and other methods, more details and textures can be recovered, enhancing image details and improving image quality.

[0059] In one embodiment, a reference image can be obtained by performing image signal processing on the second image using one or more of the following methods: black level correction, white balance gain, demosaicing, color restoration, and tone mapping. Black level correction adjusts the image by subtracting the black level value of each pixel to ensure the accuracy of black areas and eliminate baseline noise generated by the image sensor in the absence of light. White balance gain adjusts the gain of the red-green-blue (RGB) color channels according to the color temperature of the scene's light source, ensuring that white objects appear white in the image. Demosaicing uses interpolation algorithms to fill unsampled pixels with color, converting the original image captured by the imaging system into a full-color image. Color restoration corrects the image based on a color space model, making the colors more natural and restoring the true colors of the image. Tone mapping uses techniques such as curve adjustment or histogram equalization to optimize the tonal range of the image, thereby adjusting the image's brightness and contrast for better visual effects on different display devices. Compared to the test image, the reference image obtained through the above image signal processing has higher image quality and is closer to the real scene.

[0060] Step 202: Convert the test image and reference image to the frequency domain to calculate frequency information and generate the modulation transfer function curve corresponding to the test image in the frequency domain. The modulation transfer function curve is used to indicate the imaging capability of the imaging system.

[0061] In one possible implementation, the test and reference images can be transformed to the frequency domain using the Discrete Fourier Transform (DFT). The DFT transforms the image from a spatial domain described by pixel coordinates and grayscale values ​​to a frequency domain described by spatial frequency, spectral coordinates, and spectral intensity, facilitating subsequent image quality analysis and imaging system evaluation.

[0062] It should be understood that the frequency involved in the embodiments of this application refers to the frequency of the image, which can also be called the spatial frequency, reflecting the change of the gray value corresponding to the pixel of the image at different pixel positions.

[0063] In one embodiment, after generating the reference image based on the test image, both the test image and the reference image need to be preprocessed. Specifically, at least one alignment operation can be performed on the test image and the reference image, including brightness alignment, color alignment, and spatial alignment. Brightness alignment eliminates brightness differences caused by varying lighting conditions, ensuring that the test image and the reference image have similar brightness distributions, making subsequent frequency information calculations more accurate. Furthermore, the test image and the reference image may differ in color saturation, hue, etc., affecting frequency information calculations. Color alignment converts the images to the same color space or performs color correction, ensuring consistency between the test image and the reference image in color space. Spatial alignment ensures consistency between the test image and the reference image in spatial location.

[0064] In one possible implementation, to facilitate subsequent frequency domain analysis, if the test image and reference image are nonlinear, they need to be converted into linearized images respectively. Furthermore, to avoid noise introduced by complex textures or lighting variations, flat regions in the test and reference images are typically selected, and then linearization is performed on the test and reference images based on these flat regions. That is, when the test image is nonlinear, flat regions in the test image can be identified, and linearization is performed on the test image based on these flat regions. Similarly, when the reference image is nonlinear, flat regions in the reference image can be identified, and linearization is performed on the reference image based on these flat regions.

[0065] In one embodiment, the test image and reference image are in JPEG (joint photographic experts group) format. JPEG images are non-linear for real-world scenes and raw image formats (RAW), requiring brightness linearization before frequency information calculation. This process may include: first, extracting pixel brightness values ​​from selected flat regions (regions with small brightness variations); then analyzing the distribution of these brightness values ​​to identify their non-linear characteristics; next, using curve fitting methods (such as polynomial fitting or spline interpolation) to estimate the non-linear response curve; then, calculating the linearized corresponding value for each brightness value based on the non-linear response curve; and finally, applying the linearized brightness values ​​to the entire image to complete the brightness linearization operation.

[0066] As one possible implementation, flat region selection can be achieved using a quadtree-based image segmentation algorithm. Quadtree-based image segmentation is an efficient segmentation method that achieves hierarchical image segmentation by recursively dividing the image into four sub-regions (i.e., a quadtree structure). A quadtree is a data structure primarily used for partitioning and indexing two-dimensional space, particularly suitable for image processing. In image segmentation, quadtrees are commonly used to divide image regions, recursively dividing the image space into four sub-regions based on pixel values ​​or color information until a certain termination condition is met. An example diagram of a quadtree-based image segmentation algorithm is shown below. Figure 3 As shown, Figure 3 In the image (a), the test image or reference image is shown. The image can be divided into multiple image regions using a quadtree-based image segmentation algorithm. The segmentation results can be found in [reference needed]. Figure 3 (b) In Figure 3 Based on the image region segmentation results in (b), the image region with the largest area can be further defined as a flat region, or the image regions with the largest and second largest areas can be defined as flat regions. The specific determination can be made in combination with the actual application scenario and requirements.

[0067] In one embodiment, the frequency components of an image are typically directional, meaning the frequency response may differ in different directions. For example, a first modulation transfer function (MTF) value at a certain frequency contains function components in both horizontal and vertical frequency directions. Therefore, by averaging the function values ​​in different frequency directions, the performance of the imaging system across all frequency directions can be more intuitively reflected. That is, each of the aforementioned first MTF values ​​includes function values ​​in multiple frequency directions (e.g., horizontal, vertical, diagonal, etc.). Averaging the function values ​​in all frequency directions included in each first MTF value yields a second MTF value corresponding to each first MTF value. This second MTF value is independent of frequency direction. Then, a modulation transfer function (MTF) curve is generated based on each second MTF value.

[0068] In one embodiment, after obtaining all the second modulation transfer function (MTF) values, each MTF value can be smoothed to obtain a corresponding third MTF value. Furthermore, a modulation transfer function (MTF) curve is generated based on each third MTF value. The smoothing process is used to eliminate noise and outliers in the data and extract the basic trends and characteristics of the data.

[0069] In one possible implementation, the specific steps of step 202 are as follows: Figure 4 As shown, it includes:

[0070] Step 401: Perform Discrete Fourier Transform on the test image and the reference image respectively to obtain the first frequency domain information corresponding to the test image and the second frequency domain information corresponding to the reference image.

[0071] It should be noted that the Discrete Fourier Transform can transform an image from a spatial domain described by pixel coordinates and gray values ​​to a frequency domain described by spatial frequency, spectral coordinates, and spectral intensity, which facilitates subsequent analysis of image quality and evaluation of the imaging system.

[0072] Step 402: Determine multiple first modulation transfer function values ​​of the test image in the frequency domain based on the first frequency domain information and the second frequency domain information.

[0073] In this embodiment, step 402 can be performed using either power spectral density calculation or autocorrelation / cross-correlation calculation. Power spectral density describes the variation of signal power with frequency. For images, power spectral density displays the energy distribution at different frequencies, aiding in the analysis of image details and noise characteristics. The autocorrelation function describes the similarity between a signal and itself at different time delays. In this embodiment, the autocorrelation function can be used to analyze the repetition patterns and structural characteristics of an image. The cross-correlation function describes the similarity between two signals. In this embodiment, the cross-correlation function is used to compare the similarity between a reference image and a test image. These two calculation methods are described below.

[0074] In one embodiment, a first power spectral density of the test image at each frequency can be determined firstly based on first frequency domain information, and a second power spectral density of the reference image at each frequency can be determined based on second frequency domain information. Then, the ratio of the first power spectral density to the second power spectral density at each frequency is determined as the first modulation transfer function value of the test image in the frequency domain. Typically, the power spectral density is obtained by averaging the squared amplitude of the frequency domain image. For each frequency component, a frequency component can be calculated. By comparing the power spectral densities of the test image and the reference image, the image quality can be analyzed. Determining the ratio of the first power spectral density to the second power spectral density as the first modulation transfer function value quantifies the similarity between the reference image and the image to be evaluated.

[0075] In another embodiment, autocorrelation calculation can be performed first based on the second frequency domain information to obtain autocorrelation function values ​​at each frequency; then, cross-correlation calculation can be performed based on the first and second frequency domain information to obtain cross-correlation function values ​​at each frequency; finally, the ratio of the autocorrelation function value to the cross-correlation function value at each frequency is determined as the first modulation transfer function value of the test image in the frequency domain. Specifically, the frequency domain representations of the test image and the reference image are obtained through discrete Fourier transform; autocorrelation calculation based on the frequency domain representation of the reference image yields the autocorrelation function; and cross-correlation calculation based on the frequency domain representations of the test image and the reference image yields the cross-correlation function. Determining the ratio of the autocorrelation function value to the cross-correlation function value as the first modulation transfer function value quantifies the similarity between the reference image and the image to be evaluated.

[0076] Step 403: Generate a modulation transfer function curve that varies with frequency based on multiple first modulation transfer function values ​​of the test image in the frequency domain.

[0077] In this embodiment, after obtaining multiple first modulation transfer function (MTF) values ​​of the test image in the frequency domain using the two methods described above, the MTF values ​​at each frequency are then fitted to obtain a MTF curve indicating the imaging capability of the imaging system. It should be understood that ideally, the test image captured by the imaging system is completely identical to the reference image, in which case the MTF curve should be a straight line with a function value of 1. However, due to the performance factors of the imaging system, the function values ​​of the MTF curve at different frequencies are generally less than 1. It should also be understood that the closer the function value of the MTF curve at each frequency is to 1, the higher the similarity between the test image and the reference image, and the better the imaging capability of the imaging system.

[0078] The imaging system evaluation method provided in the above embodiments mainly involves capturing real scene images through the imaging system, using these real scene images as test images, and simultaneously generating high-quality images based on these real scene images as reference images. Finally, the test images and reference images are converted to the frequency domain for frequency information calculation to obtain a modulation transfer function curve that reflects the imaging capability of the imaging system. In this way, the degree to which the images captured by the imaging system reproduce the real scene can be effectively evaluated, making the evaluation results more reliable.

[0079] In one possible implementation, besides analyzing the imaging capability of the imaging system using the modulation transfer function curve, the imaging capability can also be analyzed by comparing the noise estimation results of the test image and the reference image. That is, noise estimation can be performed on the test image based on flat regions in both the test and reference images, yielding a noise estimation result. This noise estimation result and / or the modulation transfer function curve are used to indicate the imaging capability of the imaging system. Flat regions are chosen for noise estimation because they provide more stable statistical characteristics and reduce variations caused by texture and edges. Specifically, image noise variance, standard deviation, signal-to-noise ratio, and other data can be calculated and compared on both the test and reference images to obtain the noise estimation result for the test image.

[0080] To illustrate the effectiveness of the imaging system evaluation method provided in the above embodiments of this application, the following is combined with... Figure 5 and Figure 6 Please provide an explanation.

[0081] Figure 5 The diagram shows a reference image and three test images provided in an embodiment of this application. Figure 5 (a) is Figure 5 The reference images for the three test images in (b), (c), and (d) are respectively... Figure 5 (a) and Figure 5In (b), (c), and (d), modulation transfer function curves were generated using the imaging system evaluation method described above. Figure 5 The modulation transfer function curve corresponding to the test image shown in (b) is as follows: Figure 6 As shown in (a), Figure 5 The modulation transfer function curve corresponding to the test image shown in (c) is as follows: Figure 6 As shown in (b), Figure 5 The modulation transfer function curve corresponding to the test image shown in (d) is as follows: Figure 6 As shown in (c).

[0082] In an ideal scenario, the test image captured by the imaging system is completely identical to the reference image. Therefore, the modulation transfer function curve should be a straight line with a function value of 1, i.e. Figure 6 The reference lines shown in (a), (b), and (c) are used, but due to the performance factors of the imaging system, the modulation transfer function curve will generally have a value less than 1 at different frequencies. Figure 6 As shown, overall Figure 6 The modulation transfer function curve shown in (c) is closer to the reference line, indicating that... Figure 5 The test image shown in (d) is most similar to the reference image. Figure 5 The imaging system corresponding to (d) has better imaging capabilities.

[0083] The methods of the embodiments of this application have been described above with reference to the accompanying drawings. It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially, these steps are not necessarily executed in the order shown in the figures. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps. The apparatus of the embodiments of this application will now be described with reference to the accompanying drawings.

[0084] Reference Figure 7 This is a schematic diagram of the structure of an imaging system evaluation device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown. Figure 7 As shown, the imaging system evaluation apparatus 700 includes an acquisition unit 701 and a processing unit 702. The imaging system evaluation apparatus 700 can be integrated into an electronic device. For example, the imaging system evaluation apparatus 700 can also be used to perform... Figure 2 or Figure 4 The process is shown.

[0085] The imaging system evaluation device 700 can be used to perform any of the methods described above. For example, the acquisition unit 701 can be used to perform step 201, and the processing unit 702 can be used to perform step 202.

[0086] The imaging system evaluation device provided in this application mainly obtains real scene images by capturing images through the imaging system. While using the real scene images as test images, a high-quality image is also generated based on the real scene images as a reference image. Finally, the test image and the reference image are converted to the frequency domain to calculate frequency information and obtain the modulation transfer function curve that can reflect the imaging capability of the imaging system. In this way, the degree of reproduction of the real scene by the images captured by the imaging system can be effectively evaluated, making the evaluation results more reliable.

[0087] It should be noted that the foregoing explanation of the imaging system evaluation method embodiment also applies to the imaging system evaluation device 700 of this embodiment, and will not be repeated here.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0089] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. The electronic device may include the aforementioned chip to be calibrated, such as... Figure 8As shown, the electronic device 800 may include a central processing unit (CPU) 810, an external memory interface 820, an internal memory 821, a universal serial bus (USB) interface 830, a charging management module 840, a power management module 841, a battery 842, antenna 1, antenna 2, a mobile communication module 850, a wireless communication module 860, an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, a headphone jack 870D, a sensor module 880, buttons 890, a motor 891, an indicator 892, a camera 893, a display screen 894, and a subscriber identification module (SIM) card interface 895, etc. The sensor module 880 may include a pressure sensor 880A, a gyroscope sensor 880B, a barometric pressure sensor 880C, a magnetic sensor 880D, an accelerometer sensor 880E, a distance sensor 880F, a proximity sensor 880G, a fingerprint sensor 880H, a temperature sensor 880J, a touch sensor 880K, an ambient light sensor 880L, and a bone conduction sensor 880M, etc. It should be understood that the steps in the foregoing method embodiments are executed by the processor 810 of the electronic device.

[0090] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 800. In other embodiments of this application, the electronic device 800 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] For example, Figure 8 The processor 810 shown may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0092] The controller can serve as the nerve center and command center of the electronic device 800. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0093] The processor 810 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory. This memory can store instructions or data that the processor 810 has just used or that are used repeatedly. If the processor 810 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 810, and thus improves the efficiency of the system.

[0094] In some embodiments, the MIPI interface can be used to connect the processor 810 to peripheral devices such as the display screen 894 and the camera 893. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). The processor 810 and the display screen 894 communicate through the DSI interface to realize the display function of the electronic device 800.

[0095] In some embodiments, the GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. The GPIO interface can be used to connect the processor 810 to a camera 893, a display screen 894, a wireless communication module 860, an audio module 870, a sensor module 880, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0096] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 800. In other embodiments of this application, the electronic device 800 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0097] Electronic device 800 implements display functions through a GPU, a display screen 894, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 894 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 810 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0098] The display screen 894 is used to display images, videos, etc. The display screen 894 includes a display panel. In some embodiments, the electronic device 800 may include one or N display screens 894, where N is a positive integer greater than 1.

[0099] Internal memory 821 can be used to store computer executable program code, which includes instructions. Processor 810 executes various functional applications and data processing of electronic device 800 by running the instructions stored in internal memory 821. Internal memory 821 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 800 (such as audio data, phonebook, etc.). Furthermore, internal memory 821 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0100] Pressure sensor 880A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 880A can be disposed on display screen 894. There are many types of pressure sensors 880A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 880A, the capacitance between the electrodes changes. Electronic device 800 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 894, electronic device 800 detects the intensity of the touch operation based on pressure sensor 880A. Electronic device 800 can also calculate the touch position based on the detection signal from pressure sensor 880A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0101] Touch sensor 880K, also known as a "touch panel," can be located on display screen 894. The touch sensor 880K and display screen 894 together form a touchscreen, also known as a "touchscreen." Touch sensor 880K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 894. In other embodiments, touch sensor 880K may also be located on the surface of electronic device 800, in a different position than display screen 894.

[0102] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0104] This application also provides an electronic device comprising: one or more processors, a memory, and a computer program stored in the memory and executable on the one or more processors. When the one or more processors execute the computer program, the electronic device can perform the steps in any of the methods described above. This application also provides a computer-readable storage medium storing a computer program, which, when executed by an electronic device, can perform the steps in the various method embodiments described above.

[0105] The computer-readable medium may include at least: any entity or device capable of carrying computer program code to a photographic / electronic device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical discs. In some jurisdictions, according to legislation and patent practice, computer-readable media may not be electrical carrier signals or telecommunication signals.

[0106] This application provides a computer program product, which includes a computer program that, when executed by an electronic device, can implement the steps described in the various method embodiments above. The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0112] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0113] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0114] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0115] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An imaging system evaluation method, characterized in that, include: The system acquires multiple first images of a real scene by capturing multiple frames of the scene using an imaging system. The first images include a test image. The first images are then fused using a method including super-resolution to obtain a second image. The second image is then processed to obtain a reference image. The reference image and the test image contain the same image content, including a human face. The reference image has higher image quality and higher detail than the test image. Perform Discrete Fourier Transform on the test image and the reference image respectively to obtain the first frequency domain information corresponding to the test image and the second frequency domain information corresponding to the reference image; Based on the second frequency domain information, an autocorrelation function is used to perform autocorrelation calculation to obtain multiple autocorrelation function values ​​in the frequency domain; based on the first frequency domain information and the second frequency domain information, a cross-correlation function is used to perform cross-correlation calculation to obtain multiple cross-correlation function values ​​in the frequency domain; based on the ratio of the multiple autocorrelation function values ​​to the multiple cross-correlation function values, multiple first modulation transfer function values ​​of the test image in the frequency domain are determined; wherein, the autocorrelation function is used to analyze the repetition pattern and structural characteristics of the reference image, and the cross-correlation function is used to compare the similarity between the reference image and the test image; each first modulation transfer function value includes function values ​​in the horizontal frequency direction, the vertical frequency direction, and the diagonal frequency direction; For each first modulation transfer function value, the function values ​​in the horizontal frequency direction, vertical frequency direction, and diagonal frequency direction corresponding to the first modulation transfer function value are averaged to obtain the second modulation transfer function value corresponding to the first modulation transfer function value. Each second modulation transfer function value is smoothed to obtain a third modulation transfer function value corresponding to each second modulation transfer function value. The smoothing process is used to eliminate noise and outliers in all second modulation transfer function values ​​and extract the basic trends and features of all second modulation transfer function values. Based on each of the third modulation transfer function values, a modulation transfer function curve corresponding to the test image in the frequency domain is generated, and the modulation transfer function curve is used to indicate the imaging capability of the imaging system.

2. The method according to claim 1, characterized in that, After performing image signal processing on the second image to obtain the reference image, the method further includes: Perform at least one alignment operation among brightness alignment, color alignment, and spatial alignment on the test image and the reference image.

3. The method according to claim 1 or 2, characterized in that, The method further includes: In the case that the test image is a non-linear image, flat regions in the test image are identified, and the test image is linearized based on the flat regions in the test image to convert the test image into a linear image; or, In the case where the reference image is a non-linear image, flat regions in the reference image are identified, and the reference image is linearized based on the flat regions in the reference image to convert the reference image into a linear image.

4. The method according to claim 1 or 2, characterized in that, The method of fusing multiple frames of the first image to obtain the second image using a super-resolution approach includes: The second image is obtained by fusing the multiple frames of the first image using one or more of the following methods: inter-frame registration, ghost detection, and super-resolution.

5. The method according to claim 4, characterized in that, The image signal processing includes one or more of the following methods: black level correction, white balance gain, depigmentation, color restoration, and tone mapping.

6. The method according to claim 1 or 2, characterized in that, The method further includes: The flat regions in the test image and the flat regions in the reference image are determined respectively; Based on the flat regions in the test image and the flat regions in the reference image, noise estimation is performed on the test image to obtain noise estimation results. The noise estimation results and / or the modulation transfer function curve are used to indicate the imaging capability of the imaging system.

7. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 6.

8. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 6.

Images