Method, device and equipment for estimating fuzzy kernel of imaging system and storage medium

Abstract

The invention discloses an estimation method and device for a blurring kernel of an imaging system, equipment and a storage medium. The method comprises the steps that a full-view-field original checkerboard picture and a preset normalized view field height of the imaging system are acquired; determining a spatial frequency response curve and a chromatic aberration area difference of the full-view-field original checkerboard picture; inputting the normalized view field height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blurring kernel; and inputting the normalized view field height, the original blurring kernel and the chromatic aberration area difference into a second multi-layer perceptron, and outputting an estimated blurring kernel. According to the method, only one checkerboard needs to be shot in a specific environment to complete calibration, meanwhile, generalization is good, the method is suitable for a general imaging system, lens parameters do not need to be known, and precision is higher.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computational optics technology, and in particular to a method, device, equipment and storage medium for estimating a blur kernel of an imaging system. Background Art

[0002] Imaging systems enable a wide range of applications in numerous fields. However, their practical performance is inherently limited by spatially non-uniform aberrations. Accurate characterization of these aberrations is critical to achieving high performance in areas such as digital photography, industrial inspection, autonomous driving, astronomical observation, and microscopy.

[0003] The point spread function (PSF) is a mathematical representation of blur in an imaging system. Accurately modeling the PSF is of great significance for improving imaging quality and image-related tasks. The point spread function image refers to the light field distribution image of the output image when the input object is an infinitely far point light source (equivalent to parallel light). The actual point spread function image can be the light spot measured at the optimal imaging surface. The PSF is related to the angle of the incident light and exhibits rotational symmetry. The point spread function in an actual imaging system is very complex, not only related to the lens, but also to the sensor and subsequent image processing units. Considering the above factors and manufacturing errors, accurate estimation of the PSF requires actual measurement or calibration. It is hoped to find a simple calibration method, like camera noise calibration, to achieve a universal and accurate PSF estimation.

[0004] Although the existing PSF modeling methods have their own advantages, it is still very difficult to achieve both universal and high-precision modeling. Methods that directly analyze the captured images to extract blur kernels without relying on detailed lens design parameters are usually not very accurate. For example, it is proposed to obtain the blur kernel through a GAN network. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, equipment and storage medium for estimating the blur kernel of an imaging system, which only requires shooting a chessboard in a specific environment to complete the calibration. At the same time, it has good generalization and is applicable to general imaging systems. It does not require knowing the lens parameters and has higher accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a method for estimating a blur kernel of an imaging system, comprising:

[0007] Obtaining the original chessboard image of the full field of view of the imaging system and the preset normalized field of view height;

[0008] Determine the spatial frequency response curve and the chromatic aberration area difference of the full-field original checkerboard image;

[0009] Inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel;

[0010] The normalized field of view height, the original blur kernel and the color difference area difference are input into a second multi-layer perceptron, and an estimated blur kernel is output.

[0011] In a second aspect, an embodiment of the present invention further provides a device for estimating a blur kernel of an imaging system, the device comprising:

[0012] An acquisition module, used to acquire the original chessboard image of the full field of view of the imaging system and a preset normalized field of view height;

[0013] A determination module, used to determine a spatial frequency response curve and a color difference area difference of the full-field original checkerboard image;

[0014] A first optimization module, used for inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel;

[0015] The second optimization module is used to input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0016] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0017] one or more processors;

[0018] a storage device for storing one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for estimating the blur kernel of an imaging system provided by an embodiment of the present disclosure.

[0020] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to implement the method for estimating the blur kernel of an imaging system provided by the embodiments of the present disclosure.

[0021] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for estimating the blur kernel of an imaging system provided by the embodiments of the present disclosure is implemented.

[0022] The present invention discloses a method, device, equipment and storage medium for estimating blur kernel of an imaging system. The method comprises: obtaining a full-field original chessboard picture of an imaging system and a preset normalized field-of-view height; determining a spatial frequency response curve and a color difference area difference of the full-field original chessboard picture; inputting the normalized field-of-view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel; inputting the normalized field-of-view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and outputting an estimated blur kernel. By using this method: only one chessboard needs to be photographed in a specific environment to complete the calibration, and the method has good generalization, is applicable to general imaging systems, does not require knowing lens parameters, and has higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 A flowchart of a method for estimating a blur kernel of an imaging system provided by an embodiment of the present disclosure;

[0025] Figure 2 An example diagram of optimization of a method for estimating a blur kernel of an imaging system provided by an embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram of the structure of a blur kernel estimation device for an imaging system provided by an embodiment of the present disclosure;

[0027] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0035] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0036] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0037] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0038] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0039] Embodiment 1

[0040] Figure 1 A flowchart of a method for estimating a blur kernel of an imaging system provided in an embodiment of the present disclosure is applicable to providing a solution to the problem that methods that directly analyze captured images to extract blur kernels without relying on detailed lens design parameters are usually not accurate. The method can be executed by an imaging system blur kernel estimation method device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, a server, etc.

[0041] like Figure 1 As shown, a method for estimating a blur kernel of an imaging system provided by an embodiment of the present disclosure may specifically include the following steps:

[0042] S110, obtaining a full-field original chessboard image of the imaging system and a preset normalized field of view height.

[0043] In this embodiment, the field of view height is a measure of the imaging range of the optical system, usually expressed as an angle or length. In an optical system, the field of view height describes the maximum range in which the system can clearly image. In this embodiment, the preset normalized field of view height can be a plurality of values ​​between 0 and 1. The full-field original checkerboard image is a specific pattern used for camera calibration, which consists of black and white cells, which are arranged according to certain rules to form a plane calibration plate.

[0044] Specifically, a preset normalized field of view height is obtained and an imaging system is used to shoot a chessboard with a full field of view as an original chessboard picture with a full field of view of the imaging system.

[0045] S120 , determining a spatial frequency response curve and a chromatic aberration area difference of the original chessboard image of the full field of view.

[0046] In this embodiment, spatial frequency response is a measure of the imaging system's ability to respond to different spatial frequencies (i.e., image detail). It evaluates the system's resolution, i.e., the level of detail that the system can resolve, by measuring the system's contrast changes at different spatial frequencies. Color difference area difference refers to the area difference between different color regions in an image caused by color difference under certain conditions.

[0047] Specifically, an edge detection algorithm is used to extract edge information in the original chessboard image of the full field of view and the edge information is smoothed to generate a spatial frequency response curve. The average color value of different color areas in the original chessboard image of the full field of view is calculated, and the color difference area difference is determined based on the average color value.

[0048] Based on the above embodiment, determining the spatial frequency response curve and the color difference area difference of the original chessboard picture of the full field of view includes the following steps:

[0049] a1) Use edge detection algorithm to extract edge information in the original chessboard image of the full field of view and smooth the edge information to generate a spatial frequency response curve.

[0050] b1) Calculate the average color value of different color areas in the original chessboard image of the full field of view, and determine the color difference area difference based on the average color value.

[0051] The edge detection algorithm may include a Canny edge detection algorithm, a Sobel edge detection algorithm or a Roberts edge detection algorithm. The smoothing method may include a moving average, a Gaussian filter or a median filter.

[0052] Specifically, an edge detection algorithm is used to extract edge information in the original chessboard image of the full field of view and the edge information is smoothed to generate a spatial frequency response curve. The average color value of different color areas in the original chessboard image of the full field of view is calculated, and the color difference area difference is determined based on the average color value.

[0053] S130, inputting the normalized field of view height and the spatial frequency response curve into the first multi-layer perceptron, and outputting the original blur kernel.

[0054] Specifically, the normalized field of view height and the spatial frequency response curve are input into the first multilayer perceptron, the coefficients related to the normalized field of view height and the wavelength are output, and the coefficients are transformed to generate a simulated spatial frequency response curve. The average value of the sum of squares of the differences between the corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve is determined. The weights and biases of the first multilayer perceptron are adjusted according to the average value, and the step of using the normalized field of view height as the input to the first multilayer perceptron is returned to be executed until the average value is less than a preset threshold or the maximum number of iterations is reached, and the coefficients related to the normalized field of view height and the wavelength at the time of stopping are output, and the original blur kernel is determined according to the coefficients related to the normalized field of view height and the wavelength.

[0055] Figure 2 This is an example diagram of an optimization of a method for estimating blur kernel of an imaging system provided by an embodiment of the present disclosure. Figure 2 As shown,

[0056] Based on the above embodiment, the normalized field height and the spatial frequency response curve are input into the first multi-layer perceptron, and the output of the original blur kernel includes the following steps:

[0057] a2) taking the normalized field of view height as input to the first multi-layer perceptron, outputting coefficients related to the normalized field of view height and wavelength, and transforming the coefficients to generate a simulated spatial frequency response curve.

[0058] b2) determining the average of the sum of the squares of the differences between the simulated spatial frequency response curve and corresponding points of the spatial frequency response curve.

[0059] c2) adjusting the weights and biases of the first multilayer perceptron according to the average value, returning to execute the step of taking the normalized field of view height as the input to the first multilayer perceptron, stopping when the average value is less than a preset threshold or reaching a maximum number of iterations, outputting the coefficients related to the normalized field of view height and wavelength at the time of stopping, and determining the original blur kernel according to the coefficients related to the normalized field of view height and wavelength.

[0060] On the basis of the above embodiment, the weights and biases of the first multilayer perceptron are adjusted according to the average value, including: defining the optimization objective function of the first multilayer perceptron as minimizing the average value, calculating the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron, and updating the weights and biases accordingly.

[0061] Specifically, the mathematical definition of the blur kernel is:

[0062]

[0063] Where A(P) is the aperture function, W is the wave aberration, and λ is the wavelength. Since the aperture function A(P) is usually known, the point spread function can be easily calculated given the wave aberration W. The actual PSF is multidimensional and is related to the normalized field of view height. In order to facilitate the estimation of the PSF, the wave aberration is decomposed into a set of weighted linear combinations of basis functions as follows:

[0064]

[0065] Where Wpqr is a coefficient related to the normalized field of view height H and wavelength λ, Q is a set of

[0066]

[0067] As mentioned above, the first multi-layer perceptron takes the normalized field height H as input and outputs the coefficient Wpqr. After some changes, the coefficient can obtain the simulated spatial frequency response curve SFR*.

[0068]

[0069] The goal of optimization is to adjust the weights and biases of the first multilayer perceptron to ensure that SFR* is consistent with the spatial frequency response curve, as shown below:

[0070]

[0071] S140, input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0072] On the basis of the above embodiment, the normalized field height, the original blur kernel and the color difference area difference are input into the second multi-layer perceptron, and the output estimated blur kernel includes the following steps:

[0073] a3) Input the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and output the estimated color difference area difference.

[0074] b3) Determine the difference between the estimated color difference area difference and the color difference area difference corresponding to each color difference.

[0075] c3) adjusting the weights and biases of the second multilayer perceptron according to the difference, returning to the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron until the difference is less than a preset threshold or the maximum number of iterations is reached, outputting the blur kernel displacement of each channel, and adjusting the original blur kernel according to the blur kernel displacement of each channel to generate an estimated blur kernel.

[0076] On the basis of the above embodiment, the weights and biases of the second multilayer perceptron are adjusted according to the difference, including: defining the optimization objective function of the second multilayer perceptron as minimizing the difference, calculating the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron, and updating the weights and biases accordingly.

[0077] In this embodiment, the second multilayer perceptron has the same principle and mechanism as the first multilayer perceptron. The optimization objective function of the second multilayer perceptron is defined as minimizing the difference. The gradient of the loss function with respect to the weight and bias of the second multilayer perceptron is calculated, and the weight and bias are updated accordingly. The loss function is calculated as follows:

[0078]

[0079] Specifically, the normalized field of view height and the original blur kernel are input into the second multi-layer perceptron, the estimated color difference area difference is output, the difference between the estimated color difference area difference and the color difference area difference corresponding to each color difference is determined, the weight and bias of the second multi-layer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron is returned to execute until the difference is less than a preset threshold or the maximum number of iterations is reached, the blur kernel displacement of each channel is output, and the original blur kernel is adjusted according to the blur kernel displacement of each channel, and the displacement is applied to the red and blue blur kernels to move their positions. The red, green and blue blur kernels together form a full-band blur kernel as the estimated blur kernel.

[0080] The reasons for the high accuracy of this method are: the model is accurate, starting from the wave aberration, the PSF is obtained by diffraction propagation, and the precise optimization is simple in theory. The wave aberration is expressed by the wavefront basis. The goal of optimization is to learn the coefficients of the wavefront basis. The number of parameters is only about 10, rather than directly optimizing the two-dimensional distribution of the wavefront, the latter of which generally has a number of parameters of the order of tens to hundreds. The blur kernel model is modeled based on diffraction propagation. Specifically, the wave aberration is expressed by summing the weights of the wavefront basis, and then the PSF is obtained by transformation.

[0081] The present invention has been verified by simulation and experiment. In the simulated point spread function image, the PSF obtained by the method is closer to the real value. In the actual measurement results, from the perspective of no reference index, the quality of the image deblurred by the method is better than that of other methods.

[0082] The present invention discloses a method for estimating blur kernel of an imaging system, the method comprising: obtaining a full-field original chessboard picture of the imaging system and a preset normalized field-of-view height; determining a spatial frequency response curve and a color difference area difference of the full-field original chessboard picture; inputting the normalized field-of-view height and the spatial frequency response curve into a first multi-layer perceptron, outputting the original blur kernel; inputting the normalized field-of-view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, outputting the estimated blur kernel. Using this method: only one chessboard needs to be photographed in a specific environment to complete the calibration, and the method has good generalization, is applicable to general imaging systems, does not require knowing lens parameters, and has higher accuracy.

[0083] Figure 3 The present invention also provides a schematic diagram of a device structure of a blur kernel estimation method for an imaging system. Figure 3 As shown, an acquisition module 210, a determination module 220, a first optimization module 230 and a second optimization module 240.

[0084] An acquisition module 210 is used to acquire an original chessboard image of the full field of view of the imaging system and a preset normalized field of view height;

[0085] A determination module 220, configured to determine a spatial frequency response curve and a color difference area difference of the full-field original checkerboard image;

[0086] A first optimization module 230, configured to input the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and output an original blur kernel;

[0087] The second optimization module 240 is used to input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

[0088] The technical solution provided by the embodiment of the present disclosure utilizes this method: only one chessboard needs to be photographed in a specific environment to complete the calibration, and at the same time, it has good generalization and is applicable to general imaging systems, does not require knowledge of lens parameters, and has higher accuracy.

[0089] Furthermore, the determination module 220 may be used to:

[0090] Extract edge information from the full-field original chessboard image using an edge detection algorithm and perform smoothing on the edge information to generate the spatial frequency response curve;

[0091] The average color value of different color areas in the full-field original checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

[0092] Furthermore, the first optimization module 230 may be used to:

[0093] Taking the normalized field of view height as input to the first multi-layer perceptron, outputting coefficients related to the normalized field of view height and wavelength, and transforming the coefficients to generate a simulated spatial frequency response curve;

[0094] determining an average of the sum of squares of differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve;

[0095] The weights and biases of the first multilayer perceptron are adjusted according to the average value, and the step of taking the normalized field of view height as input to the first multilayer perceptron is returned to be executed until the average value is less than a preset threshold or the maximum number of iterations is reached, and the coefficients related to the normalized field of view height and wavelength at the time of stopping are output, and the original blur kernel is determined according to the coefficients related to the normalized field of view height and wavelength.

[0096] Furthermore, the first optimization module 230 may be used to:

[0097] The optimization objective function of the first multilayer perceptron is defined as minimizing the average value, and the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron is calculated, and the weights and biases are updated accordingly.

[0098] Furthermore, the second optimization module 240 may be used to:

[0099] Inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and outputting an estimated color difference area difference;

[0100] Determine the difference between the estimated color difference area difference and the color difference area difference corresponding to each color difference;

[0101] The weights and biases of the second multilayer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron is returned to be executed until the difference is less than a preset threshold or the maximum number of iterations is reached, and the blur kernel displacement of each channel is output, and the original blur kernel is adjusted according to the blur kernel displacement of each channel to generate the estimated blur kernel.

[0102] Furthermore, the second optimization module 240 may be used to:

[0103] The optimization objective function of the second multilayer perceptron is defined as minimizing the difference, the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron is calculated, and the weights and biases are updated accordingly.

[0104] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0105] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is given. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0106] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0107] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0108] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as an estimation method for a blur kernel of an imaging system.

[0109] In some embodiments, the method for estimating the blur kernel of an imaging system may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for estimating the blur kernel of an imaging system described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for estimating the blur kernel of an imaging system in any other appropriate manner (e.g., by means of firmware).

[0110] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0112] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0115] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0116] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0117] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for estimating blur kernel of an imaging system, characterized in that: include: Obtaining the original chessboard image of the full field of view of the imaging system and the preset normalized field of view height; Determine the spatial frequency response curve and the chromatic aberration area difference of the full-field original checkerboard image; Inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel; The normalized field of view height, the original blur kernel and the color difference area difference are input into a second multi-layer perceptron, and an estimated blur kernel is output.

2. The method according to claim 1, characterized in that The determining of the spatial frequency response curve and the color difference area difference of the full-field original checkerboard picture includes: Extract edge information from the full-field original chessboard image using an edge detection algorithm and perform smoothing on the edge information to generate the spatial frequency response curve; The average color value of different color areas in the full-field original checkerboard image is calculated, and the color difference area difference is determined based on the average color value.

3. The method according to claim 1, characterized in that The step of inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron and outputting an original blur kernel comprises: Taking the normalized field of view height as input to the first multi-layer perceptron, outputting coefficients related to the normalized field of view height and wavelength, and transforming the coefficients to generate a simulated spatial frequency response curve; determining an average of the sum of squares of differences between corresponding points of the simulated spatial frequency response curve and the spatial frequency response curve; The weights and biases of the first multilayer perceptron are adjusted according to the average value, and the step of taking the normalized field of view height as input to the first multilayer perceptron is returned to be executed until the average value is less than a preset threshold or the maximum number of iterations is reached, and the coefficients related to the normalized field of view height and wavelength at the time of stopping are output, and the original blur kernel is determined according to the coefficients related to the normalized field of view height and wavelength.

4. The method according to claim 3, characterized in that The step of adjusting the weight and bias of the first multilayer perceptron according to the average value includes: The optimization objective function of the first multilayer perceptron is defined as minimizing the average value, and the gradient of the loss function with respect to the weights and biases of the first multilayer perceptron is calculated, and the weights and biases are updated accordingly.

5. The method according to claim 1, characterized in that The step of inputting the normalized field of view height, the original blur kernel, and the color difference area difference into a second multi-layer perceptron and outputting an estimated blur kernel comprises: Inputting the normalized field of view height and the original blur kernel into the second multi-layer perceptron, and outputting an estimated color difference area difference; Determine the difference between the estimated color difference area difference and the color difference area difference corresponding to each color difference; The weights and biases of the second multilayer perceptron are adjusted according to the difference, and the step of inputting the normalized field of view height and the original blur kernel into the second multilayer perceptron is returned to be executed until the difference is less than a preset threshold or the maximum number of iterations is reached, and the blur kernel displacement of each channel is output, and the original blur kernel is adjusted according to the blur kernel displacement of each channel to generate the estimated blur kernel.

6. The method according to claim 5, characterized in that The step of adjusting the weight and bias of the second multilayer perceptron according to the difference includes: The optimization objective function of the second multilayer perceptron is defined as minimizing the difference, the gradient of the loss function with respect to the weights and biases of the second multilayer perceptron is calculated, and the weights and biases are updated accordingly.

7. A device for estimating blur kernel of an imaging system, characterized in that: include: An acquisition module, used to acquire the original chessboard image of the full field of view of the imaging system and a preset normalized field of view height; A determination module, used to determine a spatial frequency response curve and a color difference area difference of the full-field original checkerboard image; A first optimization module, used for inputting the normalized field of view height and the spatial frequency response curve into a first multi-layer perceptron, and outputting an original blur kernel; The second optimization module is used to input the normalized field of view height, the original blur kernel and the color difference area difference into a second multi-layer perceptron, and output an estimated blur kernel.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6 when the computer instructions are executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for estimating a blur kernel of an imaging system according to any one of claims 1 to 6.

Images