Method, device, equipment and storage medium for estimating blur kernel of imaging system

Abstract

The present invention discloses a method, apparatus, device, and storage medium for estimating the blur kernel of an imaging system. The method comprises: obtaining an original checkerboard image of the imaging system's full field of view and a preset normalized field of view height; determining the spatial frequency response curve and chromatic aberration area difference of the original checkerboard image; inputting the normalized field of view height and the spatial frequency response curve into a first multilayer perceptron, outputting an original blur kernel; and inputting the normalized field of view height, the original blur kernel, and the chromatic aberration area difference into a second multilayer perceptron, outputting an estimated blur kernel. Using this method, calibration is completed by simply photographing a checkerboard image in a specific environment. The method has good generalization properties, is applicable to general imaging systems, does not require knowledge of lens parameters, and offers higher accuracy.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computational optics technology, and in particular to a method, apparatus, device, and storage medium for estimating a blur kernel of an imaging system. Background Art

[0002] Imaging systems enable a wide range of applications in numerous fields. However, their practical performance is inherently limited by spatially nonuniform aberrations. Accurately characterizing these aberrations is crucial for achieving high performance in applications such as digital photography, industrial inspection, autonomous driving, astronomical observation, and microscopy.

[0003] The point spread function (PSF) is a mathematical representation of the blur in an imaging system. Accurately modeling the PSF is crucial for improving imaging quality and image-related tasks. A PSF image is the light field distribution image of the output image when the input is an infinitely distant point light source (equivalent to parallel light). The actual PSF image can be the light spot measured at the optimal imaging surface. The PSF is dependent on the angle of the incident light and exhibits rotational symmetry. The PSF in an actual imaging system is complex, dependent not only on the lens but also on the sensor and subsequent image processing units. Taking these factors and manufacturing errors into account, accurate PSF estimation requires actual measurement or calibration. The goal is to find a simple calibration method, similar to camera noise calibration, that can achieve both universal and accurate PSF estimation.

[0004] While existing PSF modeling methods each have their advantages, achieving both universal and high-precision modeling remains challenging. Methods that directly analyze captured images to extract blur kernels without relying on detailed lens design parameters often suffer from low accuracy. For example, a proposed method for extracting blur kernels using a GAN network has been proposed. Summary of the Invention

[0005] Embodiments of the present invention provide a method, apparatus, device, and storage medium for estimating the blur kernel of an imaging system. Calibration is completed by simply photographing a chessboard in a specific environment. The method has good generalization properties and is applicable to general imaging systems. The method does not require knowledge of lens parameters and has higher accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a method for estimating a blur kernel of an imaging system, comprising:

[0007] Obtaining the original checkerboard image of the full field of view of the imaging system and the preset normalized field of view height;

[0008] Determining a spatial frequency response curve and a chromatic aberration area difference of the full-field original checkerboard image;

[0009] Inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel;

[0010] The normalized field of view height, the original blur kernel, and the color difference area difference are input into a second multi-layer perceptron, and an estimated blur kernel is output.

[0011] In a second aspect, an embodiment of the present invention further provides a device for estimating a blur kernel of an imaging system, the device comprising:

[0012] An acquisition module, used to acquire the original checkerboard image of the full field of view of the imaging system and a preset normalized field of view height;

[0013] a determination module, configured to determine a spatial frequency response curve and a chromatic aberration area difference of the full-field original checkerboard image;

[0014] a first optimization module, configured to input the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and output an original blur kernel;

[0015] The second optimization module is used to input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0016] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising:

[0017] one or more processors;

[0018] a storage device for storing one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for estimating the blur kernel of an imaging system provided by an embodiment of the present disclosure.

[0020] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to implement the method for estimating the blur kernel of an imaging system provided by an embodiment of the present disclosure.

[0021] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for estimating the blur kernel of the imaging system provided by the embodiment of the present disclosure.

[0022] The present invention discloses a method, apparatus, device, and storage medium for estimating the blur kernel of an imaging system. The method comprises: obtaining an original checkerboard image of the imaging system's full field of view and a preset normalized field of view height; determining the spatial frequency response curve and chromatic aberration area difference of the original checkerboard image; inputting the normalized field of view height and the spatial frequency response curve into a first multilayer perceptron, outputting an original blur kernel; and inputting the normalized field of view height, the original blur kernel, and the chromatic aberration area difference into a second multilayer perceptron, outputting an estimated blur kernel. Using this method, calibration is completed by simply photographing a checkerboard image in a specific environment. The method has good generalization properties, is applicable to general imaging systems, does not require knowledge of lens parameters, and offers higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 A flowchart of a method for estimating a blur kernel of an imaging system provided by an embodiment of the present disclosure;

[0025] Figure 2 This is an example diagram of an optimization method for estimating a blur kernel of an imaging system provided by an embodiment of the present disclosure;

[0026] Figure 3 A schematic structural diagram of a blur kernel estimation device for an imaging system provided by an embodiment of the present disclosure;

[0027] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0035] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0036] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0037] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0038] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0039] Example 1

[0040] Figure 1 This is a flowchart of a method for estimating a blur kernel of an imaging system provided in an embodiment of the present disclosure. The embodiment of the present disclosure is suitable for providing a solution to the problem that methods that do not rely on detailed lens design parameters but directly analyze captured images to extract blur kernels are usually not accurate. The method can be executed by an imaging system blur kernel estimation method device, which can be implemented in the form of software and / or hardware, and optionally, can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0041] like Figure 1 As shown, the embodiment of the present disclosure provides a method for estimating a blur kernel of an imaging system, which may specifically include the following steps:

[0042] S110 , obtaining a full-field original checkerboard image of the imaging system and a preset normalized field height.

[0043] In this embodiment, the field of view height is a measure of the imaging range of an optical system, typically expressed as an angle or length. In an optical system, the field of view height describes the maximum range within which the system can clearly image. In this embodiment, the preset normalized field of view height can be multiple values ​​between 0 and 1. The full-field original checkerboard image is a specific pattern used for camera calibration. It consists of alternating black and white cells arranged according to a certain rule to form a flat calibration plate.

[0044] Specifically, a preset normalized field of view height is obtained and an imaging system is used to shoot a chessboard with a full field of view as an original chessboard picture of the full field of view of the imaging system.

[0045] S120 , determining a spatial frequency response curve and a chromatic aberration area difference of the original checkerboard image of the full field of view.

[0046] In this embodiment, spatial frequency response is a measure that describes the imaging system's ability to respond to different spatial frequencies (i.e., the level of image detail). It assesses the system's resolution, or the level of detail it can resolve, by measuring the change in contrast at different spatial frequencies. Chromatic aberration area difference refers to the difference in area between different color regions in an image due to chromatic aberration under certain conditions.

[0047] Specifically, an edge detection algorithm is used to extract edge information from the original full-field checkerboard image and smooth the edge information to generate a spatial frequency response curve. The average color value of different color areas in the original full-field checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

[0048] Based on the above embodiment, determining the spatial frequency response curve and the chromatic aberration area difference of the full-field original checkerboard image includes the following steps:

[0049] a1) Use an edge detection algorithm to extract edge information from the original checkerboard image of the full field of view and smooth the edge information to generate a spatial frequency response curve.

[0050] b1) Calculate the average color value of different color areas in the full-field original checkerboard image, and determine the color difference area difference based on the average color value.

[0051] The edge detection algorithm may include a Canny edge detection algorithm, a Sobel edge detection algorithm, or a Roberts edge detection algorithm. The smoothing method may include a moving average, a Gaussian filter, or a median filter.

[0052] Specifically, an edge detection algorithm is used to extract edge information from the original full-field checkerboard image and smooth the edge information to generate a spatial frequency response curve. The average color value of different color areas in the original full-field checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

[0053] S130: Input the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and output an original blur kernel.

[0054] Specifically, the normalized field of view height and the spatial frequency response curve are input into a first multilayer perceptron, coefficients related to the normalized field of view height and wavelength are output, and the coefficients are transformed to generate a simulated spatial frequency response curve. The average value of the sum of squares of the differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve is determined. The weights and biases of the first multilayer perceptron are adjusted based on the average value, and the step of inputting the normalized field of view height into the first multilayer perceptron is returned to and executed until the average value is less than a preset threshold or the maximum number of iterations is reached. The coefficients related to the normalized field of view height and wavelength at the time of stopping are output, and the original blur kernel is determined based on the coefficients related to the normalized field of view height and wavelength.

[0055] Figure 2 This is an example diagram of an optimization method for estimating the blur kernel of an imaging system provided by an embodiment of the present disclosure. Figure 2 As shown,

[0056] Based on the above embodiment, the normalized field of view height and the spatial frequency response curve are input into the first multi-layer perceptron, and the output of the original blur kernel includes the following steps:

[0057] a2) The normalized field of view height is input to a first multilayer perceptron, which outputs coefficients related to the normalized field of view height and wavelength, and transforms the coefficients to generate a simulated spatial frequency response curve.

[0058] b2) determining the average of the sum of the squares of the differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve.

[0059] c2) adjusting the weights and biases of the first multilayer perceptron according to the average value, returning to the step of using the normalized field of view height as input to the first multilayer perceptron, and stopping when the average value is less than a preset threshold or a maximum number of iterations is reached, outputting coefficients related to the normalized field of view height and wavelength at the time of stopping, and determining the original blur kernel based on the coefficients related to the normalized field of view height and wavelength.

[0060] Based on the above embodiment, the weights and biases of the first multilayer perceptron are adjusted according to the average value, including: defining the optimization objective function of the first multilayer perceptron as minimizing the average value, calculating the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron, and updating the weights and biases accordingly.

[0061] Specifically, the mathematical definition of the blur kernel is:

[0062]

[0063] Where A(P) is the aperture function, W is the wavefront aberration, and λ is the wavelength. Since the aperture function A(P) is usually known, the point spread function can be easily calculated given the wavefront aberration W. The actual PSF is multidimensional and depends on the normalized field of view height. To facilitate the estimation of the PSF, the wavefront aberration is decomposed into a set of weighted linear combinations of basis functions as follows:

[0064]

[0065] Where Wpqr is a coefficient related to the normalized field of view height H and wavelength λ, Q is a set of

[0066]

[0067] Continuing from the above, the first multilayer perceptron takes the normalized field height H as input and outputs the coefficient Wpqr. After some changes, this coefficient can be used to obtain the simulated spatial frequency response curve SFR*.

[0068]

[0069] The goal of optimization is to adjust the weights and biases of the first multilayer perceptron to ensure that SFR* is consistent with the spatial frequency response curve, as shown below:

[0070]

[0071] S140: Input the normalized field of view height, the original blur kernel, and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0072] Based on the above embodiment, the normalized field height, the original blur kernel, and the color difference area difference are input into the second multi-layer perceptron, and outputting the estimated blur kernel includes the following steps:

[0073] a3) Input the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and output the estimated color difference and area difference.

[0074] b3) determining the difference between the estimated color difference area difference and the color difference area difference corresponding to each color difference.

[0075] c3) adjusting the weights and biases of the second multilayer perceptron based on the difference, returning to the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron, and stopping when the difference is less than a preset threshold or the maximum number of iterations is reached. Outputting the blur kernel displacement of each channel, and adjusting the original blur kernel based on the blur kernel displacement of each channel to generate an estimated blur kernel.

[0076] Based on the above embodiment, the weights and biases of the second multilayer perceptron are adjusted according to the difference, including: defining the optimization objective function of the second multilayer perceptron as minimizing the difference, calculating the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron, and updating the weights and biases accordingly.

[0077] In this embodiment, the second multilayer perceptron has the same principle and mechanism as the first multilayer perceptron. The optimization objective function of the second multilayer perceptron is defined as minimizing the difference. The gradient of the loss function with respect to the weights and biases of the second multilayer perceptron is calculated, and the weights and biases are updated accordingly. The loss function is calculated as follows:

[0078]

[0079] Specifically, the normalized field of view height and the original blur kernel are input into the second multi-layer perceptron, the estimated color difference area difference is output, the difference corresponding to each color difference between the estimated color difference area difference and the color difference area difference is determined, the weight and bias of the second multi-layer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron is returned to execute until the difference is less than a preset threshold or the maximum number of iterations is reached, the blur kernel displacement of each channel is output, and the original blur kernel is adjusted according to the blur kernel displacement of each channel, and the displacement is applied to the red and blue blur kernels to move their positions. The red, green and blue blur kernels together constitute a full-band blur kernel as the estimated blur kernel.

[0080] This method achieves high accuracy due to its precise model. Starting from wavefront aberrations, the PSF is derived from diffraction propagation. This method is theoretically simple to optimize precisely. Wavefront aberrations are expressed using a wavefront basis. The optimization goal is to learn the coefficients of these wavefront basis, requiring only about 10 parameters, rather than directly optimizing the two-dimensional distribution of the wavefront, which typically requires tens to hundreds of parameters. The blur kernel model is based on diffraction propagation. Specifically, the wavefront aberration is represented by summing the weights of the wavefront basis, which is then transformed to obtain the PSF.

[0081] The present invention has been verified through simulations and experiments. On simulated point spread function images, the PSF obtained by this method is closer to the true value. In actual measurement results, the quality of the image deblurred by this method is better than that of other methods based on no-reference indicators.

[0082] The present invention discloses a method for estimating the blur kernel of an imaging system. The method comprises: obtaining an original checkerboard image of the imaging system's full field of view and a preset normalized field of view height; determining the spatial frequency response curve and chromatic aberration area difference of the original checkerboard image of the full field of view; inputting the normalized field of view height and spatial frequency response curve into a first multi-layer perceptron, outputting the original blur kernel; and inputting the normalized field of view height, original blur kernel, and chromatic aberration area difference into a second multi-layer perceptron, outputting the estimated blur kernel. Using this method, calibration is completed by simply capturing a single checkerboard image in a specific environment. The method has good generalization properties, is applicable to general imaging systems, does not require knowledge of lens parameters, and offers higher accuracy.

[0083] Figure 3 The present invention also provides a schematic diagram of a device structure for estimating a blur kernel of an imaging system. Figure 3 As shown, an acquisition module 210, a determination module 220, a first optimization module 230 and a second optimization module 240.

[0084] An acquisition module 210 is configured to acquire an original checkerboard image of the entire field of view of the imaging system and a preset normalized field of view height;

[0085] A determination module 220 is configured to determine a spatial frequency response curve and a color difference area difference of the full-field original checkerboard image;

[0086] A first optimization module 230 is configured to input the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron and output an original blur kernel;

[0087] The second optimization module 240 is configured to input the normalized field of view height, the original blur kernel, and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0088] The technical solution provided by the embodiment of the present disclosure utilizes this method: only one checkerboard image needs to be photographed in a specific environment to complete calibration. At the same time, it has good generalization and is applicable to general imaging systems. It does not require knowledge of lens parameters and has higher accuracy.

[0089] Furthermore, the determination module 220 may be configured to:

[0090] Extracting edge information from the full-field original checkerboard image using an edge detection algorithm and performing smoothing on the edge information to generate the spatial frequency response curve;

[0091] The average color value of different color areas in the full-field original checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

[0092] Furthermore, the first optimization module 230 may be configured to:

[0093] Taking the normalized field of view height as input to the first multi-layer perceptron, outputting coefficients related to the normalized field of view height and wavelength, and transforming the coefficients to generate a simulated spatial frequency response curve;

[0094] determining an average of the sum of squares of differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve;

[0095] Adjust the weights and biases of the first multilayer perceptron according to the average value, return to the step of taking the normalized field of view height as input to the first multilayer perceptron, and stop when the average value is less than a preset threshold or reaches a maximum number of iterations, output the coefficients related to the normalized field of view height and wavelength at the time of stopping, and determine the original blur kernel based on the coefficients related to the normalized field of view height and wavelength.

[0096] Furthermore, the first optimization module 230 may be configured to:

[0097] The optimization objective function of the first multilayer perceptron is defined as minimizing the average value, the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron is calculated, and the weights and biases are updated accordingly.

[0098] Furthermore, the second optimization module 240 may be configured to:

[0099] Inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and outputting an estimated color difference and area difference;

[0100] Determining a difference value corresponding to each color difference between the estimated color difference area difference and the color difference area difference;

[0101] The weights and biases of the second multilayer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron is returned to. The process stops when the difference is less than a preset threshold or a maximum number of iterations is reached, and the blur kernel displacement of each channel is output. The original blur kernel is adjusted according to the blur kernel displacement of each channel to generate the estimated blur kernel.

[0102] Furthermore, the second optimization module 240 may be configured to:

[0103] The optimization objective function of the second multilayer perceptron is defined as minimizing the difference, the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron is calculated, and the weights and biases are updated accordingly.

[0104] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0105] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is provided. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0106] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0107] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0108] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for estimating the blur kernel of the imaging system.

[0109] In some embodiments, the imaging system blur kernel estimation method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the imaging system blur kernel estimation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the imaging system blur kernel estimation method in any other suitable manner (e.g., via firmware).

[0110] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0115] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0116] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0117] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for estimating blur kernel of an imaging system, characterized in that: include: Obtaining the original checkerboard image of the full field of view of the imaging system and the preset normalized field of view height; Determining a spatial frequency response curve and a chromatic aberration area difference of the full-field original checkerboard image; Inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel; The normalized field of view height, the original blur kernel, and the color difference area difference are input into a second multi-layer perceptron, and an estimated blur kernel is output.

2. The method according to claim 1, characterized in that Determining the spatial frequency response curve and the chromatic aberration area difference of the full-field original checkerboard image includes: Extracting edge information from the full-field original checkerboard image using an edge detection algorithm and performing smoothing on the edge information to generate the spatial frequency response curve; The average color value of different color areas in the full-field original checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

3. The method according to claim 1, characterized in that The step of inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron and outputting an original blur kernel comprises: Taking the normalized field of view height as input to the first multi-layer perceptron, outputting coefficients related to the normalized field of view height and wavelength, and transforming the coefficients to generate a simulated spatial frequency response curve; determining an average of the sum of squares of differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve; Adjust the weights and biases of the first multilayer perceptron according to the average value, return to the step of taking the normalized field of view height as input to the first multilayer perceptron, and stop when the average value is less than a preset threshold or reaches a maximum number of iterations, output the coefficients related to the normalized field of view height and wavelength at the time of stopping, and determine the original blur kernel based on the coefficients related to the normalized field of view height and wavelength.

4. The method according to claim 3, characterized in that The adjusting the weights and biases of the first multi-layer perceptron according to the average value includes: The optimization objective function of the first multilayer perceptron is defined as minimizing the average value, the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron is calculated, and the weights and biases are updated accordingly.

5. The method according to claim 1, wherein The step of inputting the normalized field height, the original blur kernel, and the color difference area difference into a second multi-layer perceptron and outputting an estimated blur kernel comprises: Inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and outputting an estimated color difference and area difference; Determining a difference value corresponding to each color difference between the estimated color difference area difference and the color difference area difference; The weights and biases of the second multilayer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron is returned to. The process stops when the difference is less than a preset threshold or a maximum number of iterations is reached, and the blur kernel displacement of each channel is output. The original blur kernel is adjusted according to the blur kernel displacement of each channel to generate the estimated blur kernel.

6. The method according to claim 5, characterized in that The adjusting the weights and biases of the second multilayer perceptron according to the difference includes: The optimization objective function of the second multilayer perceptron is defined as minimizing the difference, the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron is calculated, and the weights and biases are updated accordingly.

7. A device for estimating blur kernel of an imaging system, characterized in that: include: An acquisition module, used to acquire the original checkerboard image of the full field of view of the imaging system and a preset normalized field of view height; a determination module, configured to determine a spatial frequency response curve and a chromatic aberration area difference of the full-field original checkerboard image; a first optimization module, configured to input the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and output an original blur kernel; The second optimization module is used to input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6.

Images