Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure discloses an image processing method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The specific implementation scheme is: performing image region division on a first image collected by a camera module to obtain a first image region set corresponding to the first image; obtaining first compensation information corresponding to at least one first image region in the first image region set based on a first point spread function; performing compensation processing on the at least one first image region based on the first compensation information corresponding to the at least one first image region to obtain a second image; and obtaining a second point spread function based on a first parameter set corresponding to the first image and / or a second parameter set corresponding to the second image. By using the present disclosure, the quality of the image can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With economic and technological development, and the increasing pace of life and work intensity, improving user experience has become a central focus across all industries. When users need to capture images, they can use a mobile device to do so. However, as users' demands for mobile device photography continue to rise, improving the image quality of images captured by mobile devices has become a key concern. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium, the main purpose of which is to improve image quality.

[0004] According to one aspect of this disclosure, an image processing method is provided, comprising:

[0005] The first image captured by the camera module is divided into image regions to obtain a set of first image regions corresponding to the first image.

[0006] Based on the first point diffusion function, obtain the first compensation information corresponding to at least one first image region in the first image region set;

[0007] Based on the first compensation information corresponding to the at least one first image region, compensation processing is performed on the at least one first image region to obtain a second image;

[0008] The second point diffusion function is obtained based on the first parameter set corresponding to the first image and / or the second parameter set corresponding to the second image.

[0009] Optionally, obtaining the second point diffusion function based on the first parameter set corresponding to the first image and / or the second parameter set corresponding to the second image includes:

[0010] Based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image, obtain a parameter ratio set;

[0011] Based on the set of parameter ratios, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function;

[0012] or

[0013] If the first parameter set or the second parameter set does not meet the parameter conditions, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0014] Optionally, the parameter ratio set includes a sharpness ratio and a noise ratio, and the step of adjusting the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function includes:

[0015] Obtain the first sharpness value of any image region in the first image region set in the first image and the second sharpness value of any image region in the second image;

[0016] The ratio of the second sharpness value to the first sharpness value is obtained to obtain the sharpness ratio.

[0017] Obtain the first noise value corresponding to the all-white image region and the all-black image region in the first image, and the second noise value corresponding to the all-white image region and the all-black image region in the second image;

[0018] The ratio of the second noise value to the first noise value is obtained to obtain the noise ratio.

[0019] Based on the sharpness ratio and the noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0020] Optionally, the parameter ratio set includes a sharpness ratio and a noise ratio, and the step of adjusting the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function includes:

[0021] Obtain a set of historical parameter ratios, wherein the set of historical parameter ratios includes at least one historical sharpness ratio and at least one historical noise ratio;

[0022] Obtain the maximum sharpness ratio among the sharpness ratio and the at least one historical sharpness ratio; obtain the maximum noise ratio among the noise ratio and the at least one historical noise ratio.

[0023] Based on the maximum sharpness ratio and the maximum noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0024] Optionally, after obtaining the second point diffusion function, the process includes:

[0025] Acquire the third image captured by the camera module;

[0026] The third image is divided into image regions to obtain a set of second image regions corresponding to the third image.

[0027] Based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set;

[0028] Based on the second compensation information corresponding to the at least one second image region, the third image is processed to obtain the fourth image.

[0029] Optionally, the step of dividing the first image captured by the camera module into image regions to obtain a first image region set corresponding to the first image includes:

[0030] Obtain the module identifier corresponding to the camera module;

[0031] Obtain the region division information corresponding to the module identifier;

[0032] Based on the region division information, the first image captured by the camera module is divided into image regions to obtain a first image region set corresponding to the first image.

[0033] Optionally, obtaining the second point diffusion function includes:

[0034] Obtain the set of initial coefficients corresponding to the first point diffusion function and the set of coefficient ranges corresponding to at least one initial coefficient in the set of initial coefficients;

[0035] Traverse the set of coefficient ranges corresponding to any initial coefficient in the set of initial coefficients to obtain the target coefficient corresponding to the initial coefficient;

[0036] By traversing the initial set of coefficients, the target set of coefficients corresponding to the first point diffusion function is obtained;

[0037] Based on the target set of coefficients, the at least one initial coefficient corresponding to the first point diffusion function is adjusted to obtain the second point diffusion function.

[0038] According to another aspect of this disclosure, an image processing apparatus is provided, characterized in that it comprises:

[0039] The image acquisition unit is used to divide the first image acquired by the camera module into image regions to obtain a first image region set corresponding to the first image.

[0040] The information acquisition unit is used to acquire first compensation information corresponding to at least one first image region in the first image region set based on the first point diffusion function;

[0041] An image processing unit is configured to perform compensation processing on the at least one first image region based on first compensation information corresponding to at least one first image region to obtain a second image.

[0042] The function acquisition unit is used to acquire the second point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image.

[0043] Optionally, the function acquisition unit includes a ratio acquisition subunit and a coefficient adjustment subunit. The function acquisition unit is used to acquire the second point diffusion function after adjusting the first point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image.

[0044] The ratio acquisition subunit is used to acquire a set of parameter ratios based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image;

[0045] The coefficient adjustment subunit is used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function;

[0046] or

[0047] The coefficient adjustment subunit is further configured to adjust the coefficients of the first point diffusion function to obtain the second point diffusion function if the first parameter set or the second parameter set does not meet the parameter conditions.

[0048] Optionally, the parameter ratio set includes a sharpness ratio and a noise ratio. The coefficient adjustment subunit, used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function, specifically performs the following:

[0049] Obtain the first sharpness value of any image region in the first image region set in the first image and the second sharpness value of any image region in the second image;

[0050] The ratio of the second sharpness value to the first sharpness value is obtained to obtain the sharpness ratio.

[0051] Obtain the first noise value corresponding to the all-white image region and the all-black image region in the first image, and the second noise value corresponding to the all-white image region and the all-black image region in the second image;

[0052] The ratio of the second noise value to the first noise value is obtained to obtain the noise ratio.

[0053] Based on the sharpness ratio and the noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0054] Optionally, the parameter ratio set includes a sharpness ratio and a noise ratio. The coefficient adjustment subunit, used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function, specifically performs the following:

[0055] Obtain a set of historical parameter ratios, wherein the set of historical parameter ratios includes at least one historical sharpness ratio and at least one historical noise ratio;

[0056] Obtain the maximum sharpness ratio among the sharpness ratio and the at least one historical sharpness ratio; obtain the maximum noise ratio among the noise ratio and the at least one historical noise ratio.

[0057] Based on the maximum sharpness ratio and the maximum noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0058] Optionally, the device further includes a fourth image acquisition unit, used to acquire a second point diffusion function after adjusting the first point diffusion function, and then acquire a third image captured by the camera module;

[0059] The third image is divided into image regions to obtain a set of second image regions corresponding to the third image.

[0060] Based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set;

[0061] Based on the second compensation information corresponding to the at least one second image region, the third image is processed to obtain the fourth image.

[0062] Optionally, the image acquisition unit, when dividing the first image acquired by the camera module into image regions to obtain a first image region set corresponding to the first image, specifically uses the following methods:

[0063] Obtain the module identifier corresponding to the camera module;

[0064] Obtain the region division information corresponding to the module identifier;

[0065] Based on the region division information, the first image captured by the camera module is divided into image regions to obtain a first image region set corresponding to the first image.

[0066] Optionally, the function acquisition unit, when acquiring the second point diffusion function after adjusting the first point diffusion function, is specifically used for:

[0067] Obtain the set of initial coefficients corresponding to the first point diffusion function and the set of coefficient ranges corresponding to at least one initial coefficient in the set of initial coefficients;

[0068] Traverse the set of coefficient ranges corresponding to any initial coefficient in the set of initial coefficients to obtain the target coefficient corresponding to the initial coefficient;

[0069] By traversing the initial set of coefficients, the target set of coefficients corresponding to the first point diffusion function is obtained;

[0070] Based on the target set of coefficients, the at least one initial coefficient corresponding to the first point diffusion function is adjusted to obtain the second point diffusion function.

[0071] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0072] At least one processor; and

[0073] A memory communicatively connected to the at least one processor; wherein,

[0074] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in any one of the preceding aspects.

[0075] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the preceding aspects.

[0076] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in any one of the preceding aspects.

[0077] In one or more embodiments of this disclosure, by dividing the first image captured by the camera module into image regions, a set of first image regions corresponding to the first image is obtained. Based on a first point spread function, first compensation information corresponding to at least one first image region in the set of first image regions is obtained. This first compensation information can be used to compensate at least one first image region to obtain a second image. Based on a first parameter set corresponding to the first image and a second parameter set corresponding to the second image, a second point spread function is obtained. Therefore, the terminal can adjust the first point spread function to obtain an adjusted second point spread function. The first point spread function can be optimized, and different compensation information can be used to compensate the image. This can reduce the problems of unclear images and severe noise when directly acquiring images based on the first point spread function, improve the quality of the acquired images, and thus improve the user experience.

[0078] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0079] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0080] Figure 1 This diagram illustrates a background of an image processing method provided in an embodiment of the present disclosure.

[0081] Figure 2 This diagram illustrates a system architecture of an image processing method provided in an embodiment of the present disclosure.

[0082] Figure 3 This diagram illustrates the decrease in near-field high-view resolution provided in an embodiment of the present disclosure.

[0083] Figure 4 A schematic flowchart of the first image processing method provided in the embodiments of this disclosure is shown;

[0084] Figure 5 A flowchart illustrating a second image processing method provided in an embodiment of this disclosure is shown.

[0085] Figure 6 This diagram illustrates the structure for controlling a camera to capture a first image, as provided in an embodiment of this disclosure.

[0086] Figure 7 This diagram illustrates the PSF shape of different image regions provided in embodiments of this disclosure.

[0087] Figure 8This diagram illustrates the single-dimensional distribution of PSF in the central image region provided in an embodiment of this disclosure.

[0088] Figure 9 This diagram illustrates a PSF deconvolution image processing method provided in an embodiment of this disclosure.

[0089] Figure 10 This diagram illustrates the interaction between the server and the image processing system provided in an embodiment of this disclosure.

[0090] Figure 11 This diagram illustrates a compensation process for a third image provided in an embodiment of the present disclosure.

[0091] Figure 12 This diagram illustrates an example of a terminal interface provided in an embodiment of the present disclosure.

[0092] Figure 13 This diagram illustrates the structure of a first image processing apparatus provided in an embodiment of the present disclosure.

[0093] Figure 14 This diagram illustrates the structure of a second image processing apparatus provided in an embodiment of the present disclosure.

[0094] Figure 15 This diagram illustrates the structure of a third image processing apparatus provided in an embodiment of the present disclosure.

[0095] Figure 16 This is a block diagram of an electronic device used to implement the image processing method of the embodiments of this disclosure. Detailed Implementation

[0096] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0097] With the development of science and technology and the increasing pace of life and work intensity, improving user experience has become a central focus for all industries. When users need to take pictures, they can use the terminal to do so.

[0098] Figure 1 This diagram illustrates a background illustration of an image processing method provided in an embodiment of the present disclosure. Figure 1As shown, when a user needs to take an image, they can do so by clicking the camera application on the terminal. When the terminal detects that the user has clicked the application, it can control the camera to capture the image. The terminal can either display the captured image on the screen or save the captured image upon receiving a shooting command.

[0099] Figure 2 This diagram illustrates a system architecture of an image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, when a user needs to take a picture, they click the camera application set on terminal 10, and send a shooting command to camera 11 by clicking the shooting button in the camera application. When camera 11 receives the shooting command, it takes a picture and sends the picture to processor 12 of terminal 10 via a wire. However, the picture has problems such as poor clarity and severe noise, resulting in poor image quality and a poor user experience.

[0100] According to some embodiments, the camera on the terminal is limited by its physical size, resulting in technical problems such as high sensitivity and imaging limitations. Especially when the camera uses a small-sized module, as the sensor size increases, the ratio of the total length of the optical lens to the focal length decreases, leading to a decline in image quality at different distances. For example, while the image quality is optimal at long distances, the closer the distance, the greater the field curvature in the high field of view, and there is a significant decrease in image sharpness and resolution at the edges of the field of view. Figure 3 As shown, this problem severely affects the image quality of close-up shots and reduces the user's shooting experience.

[0101] In some embodiments, when replacing the camera on the terminal with new components, such as Fresnel lenses, super lenses, or variable focal length lenses, problems such as large differences in field curvature due to tolerance and different focus distances may occur, resulting in a decrease in the image quality of the captured images and a reduction in the user's shooting experience.

[0102] The present disclosure will now be described in detail with reference to specific embodiments.

[0103] In the first embodiment, such as Figure 4 As shown, Figure 4 The diagram illustrates a flowchart of a first image processing method provided in an embodiment of this disclosure. This method can be implemented using a computer program and can run on an image processing device. The computer program can be integrated into an application or run as a standalone utility application.

[0104] The image processing device can be a terminal with a camera, including but not limited to: wearable electronic devices, handheld electronic devices, personal computers, tablets, in-vehicle electronic devices, smartphones, computing electronic devices, or other processing electronic devices connected to a wireless modem. The terminal may have different names in different networks, such as: user electronic device, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile electronic device, user terminal, terminal, wireless communication electronic device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), 5G network, 4G network, 3G network, or terminals in future evolved networks.

[0105] Specifically, the image processing method includes:

[0106] S101, Divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image;

[0107] According to some embodiments, the camera can have both video recording and still image capture capabilities. When the camera captures images through its lens, the photosensitive component circuitry and control components within the camera process the images and convert them into digital signals that the terminal can recognize. These signals are then input to the terminal's processor via a parallel port for image reconstruction.

[0108] In some embodiments, the first image refers to the image obtained after image reconstruction by the terminal when the terminal controls the camera to take a picture, that is, the image acquired by the camera module. This first image does not specifically refer to a fixed image. It refers to the image on the camera's imaging field of view, and not the image displayed on the terminal's display interface. For example, the first image will change accordingly when the shooting position changes. It will also change accordingly when the shooting angle changes. Furthermore, it will change accordingly when the camera model changes. The first image can be acquired in real-time or pre-captured.

[0109] As is easily understood, an image region refers to the image area after an image has been divided into regions. Specifically, the first image region refers to the image region obtained after dividing the first image into regions. This first image region does not specifically refer to a fixed image region. For example, when the first image changes or the way the image regions of the first image are divided changes, the first image region can also change accordingly. For example, when the terminal divides the first image into 21*21 regions, the first image region can be one of those 21*21 regions. Similarly, when the terminal divides the first image into 9*9 regions, the first image region can be one of those 9*9 regions.

[0110] Optionally, the first image region set refers to the collective comprised of first image regions. This first image region set does not specifically refer to a fixed set. For example, when the first image changes, the first image region set may also change accordingly. For instance, when the method of dividing the image regions of the first image changes, and the number of image regions included in the first image region set changes, the first image region set may also change accordingly.

[0111] It is easy to understand that when the terminal executes the image processing method, the terminal can divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image.

[0112] S102, based on the first point diffusion function, obtain the first compensation information corresponding to at least one first image region in the first image region set;

[0113] According to some embodiments, the first point spread function refers to the point spread function (PSF), which describes the response of an imaging system to a point light source (object). This first point spread function does not specifically refer to a fixed point spread function. For example, the first point spread function will change accordingly when the camera model changes. PSF includes, but is not limited to, the Zernike function or the Seidel formula.

[0114] In some embodiments, the first compensation information refers to the information used when performing compensation processing on the first image region. This first compensation information does not specifically refer to any fixed information. For example, when the position of the first image region changes in the first image, the first compensation information corresponding to that first image region will also change accordingly.

[0115] It is easy to understand that the terminal divides the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image. Based on the first point diffusion function, it obtains the first compensation information corresponding to at least one first image region in the set of first image regions.

[0116] S103, based on the first compensation information corresponding to at least one first image region, perform compensation processing on at least one first image region to obtain a second image;

[0117] In some embodiments, the second image refers to an image obtained by the terminal after performing compensation processing on at least one first image region in the set of first image regions based on a first point spread function. The second image does not specifically refer to a fixed image. For example, when the first image changes, the second image will change accordingly. When the first point spread function changes, the first point spread function will also change accordingly. When the terminal processes the first image based on the first point spread function in different ways, the first point spread function will also change accordingly.

[0118] According to some embodiments, when the terminal obtains first compensation information corresponding to at least one first image region in the first image region set, the terminal can perform compensation processing on at least one first image region in the first image region set based on the first compensation information corresponding to at least one first image region to obtain a second image corresponding to the first image.

[0119] S104, based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image, obtain the second point diffusion function.

[0120] According to some embodiments, the first parameter refers to the PSF parameter corresponding to the first image. This PSF parameter is not specifically a fixed parameter. For example, when the first image changes, the PSF parameter will change accordingly. The first parameter is not specifically a fixed parameter. For example, the PSF parameter corresponding to the first image can be parameter A or parameter B.

[0121] In some embodiments, the first parameter set refers to the set of PSF parameters corresponding to the first image. This first parameter set does not specifically refer to a fixed set. For example, when the first image changes, the first parameter set will change accordingly. When the number of aberrations corresponding to the PSF changes, the first parameter set will change accordingly.

[0122] In some embodiments, the second parameter refers to the PSF parameter corresponding to the second image. This PSF parameter is not specifically a fixed parameter. For example, when the second image changes, the PSF parameter will change accordingly; when the number of aberrations corresponding to the PSF changes, the PSF parameter will change accordingly. The second parameter is not specifically a fixed parameter; for example, the PSF parameter corresponding to the second image can be parameter A and parameter B.

[0123] In some embodiments, the second parameter set refers to the set of PSF parameters corresponding to the second image. This second parameter set does not specifically refer to a fixed set. For example, when the second image changes, the second parameter set will change accordingly. When the number of aberrations corresponding to the PSF changes, the second parameter set will change accordingly. For example, when the number of second parameters changes, the second parameter set may also change accordingly.

[0124] In some embodiments, adjustment refers to the process of optimizing the first point spread function when the terminal obtains the first parameter set and the second parameter set. This adjustment method is not specifically defined by any particular fixed method. For example, the terminal can adjust the coefficients of the first point spread function, adjust the parameters of the first point spread function, or adjust the number of aberrations in the first point spread function.

[0125] In some embodiments, the second point spread function refers to the point spread function obtained after the terminal adjusts the first point spread function. This point spread function does not specifically refer to a fixed point spread function. For example, the second point spread function will change accordingly when the adjustment method changes. The second point spread function may also change accordingly when the first point spread function changes.

[0126] In some embodiments, after obtaining the second point spread function, the terminal can acquire a fourth image based on the second point spread function. The fourth image refers to the image obtained by the terminal after compensating the image captured by the camera module based on the second point spread function. This fourth image does not specifically refer to a fixed image. For example, when the second point spread function changes, the fourth image will change accordingly. For example, when the image processing time point changes, the fourth image may also change accordingly.

[0127] It is easy to understand that when the terminal acquires the first image and the second image, it can acquire the first parameter set corresponding to the first image and the second parameter set corresponding to the second image. When the terminal acquires the first parameter set and the second parameter set, it can adjust the first point spread function to acquire the second point spread function.

[0128] In one or more embodiments of this disclosure, by dividing the first image captured by the camera module into image regions, a set of first image regions corresponding to the first image is obtained. Based on a first point spread function, first compensation information corresponding to at least one first image region in the set of first image regions is obtained. This first compensation information can be used to compensate at least one first image region to obtain a second image. Based on a first parameter set corresponding to the first image and a second parameter set corresponding to the second image, a second point spread function is obtained. Therefore, the terminal can adjust the first point spread function to obtain an adjusted second point spread function. The first point spread function can be optimized, and different compensation information can be used to compensate the image. This can reduce the problems of unclear images and severe noise when directly acquiring images based on the first point spread function, improve the quality of the acquired images, and thus improve the user experience.

[0129] Please see Figure 5 , Figure 5 A flowchart illustrating a second image processing method provided in an embodiment of this disclosure is shown.

[0130] Specific

[0131] S201, Divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image;

[0132] The specific process is as described above, and will not be repeated here.

[0133] According to some embodiments, when the terminal divides the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image, the terminal can obtain a module identifier corresponding to the camera module, and the terminal can obtain region division information corresponding to the module identifier. Based on the region division information, the terminal can divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image. Based on a first point spread function, the terminal can obtain first compensation information corresponding to at least one first image region in the set of first image regions.

[0134] In some embodiments, the region division information may be, for example, information about the number of first image regions, or information about the size of the first image regions. That is, the terminal may divide the first image into a fixed number of first image regions, or divide the first image into at least one first image region of a fixed size.

[0135] According to some embodiments, the camera can be a digital camera or an analog camera, and the terminal can control the camera to acquire the first image in a digital or analog manner.

[0136] For example, when the camera is an analog camera, the terminal sends a shooting command to the drive control unit inside the camera via the display. Simultaneously, the camera's internal detector, such as a gyroscope, sends a detection signal to the drive control unit. The lens sends a negative feedback signal to the drive control unit. The drive control unit generates a drive signal based on the shooting command, detection signal, and negative feedback signal. The drive motor controls the lens movement according to the drive signal. The sensor converts the light signal captured by the lens into an electrical signal, performs analog-to-digital conversion, and sends the digital electrical signal to the terminal. The terminal acquires the digital electrical signal through the image acquisition unit and converts it into an image signal. The display control unit acquires the image signal and controls its display on the display terminal, such as... Figure 6 As shown.

[0137] It is easy to understand that when the terminal sends a shooting command to the camera, the terminal can divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image.

[0138] S202, based on the first compensation information corresponding to at least one first image region, perform compensation processing on at least one first image region to obtain a second image;

[0139] The specific process is as described above, and will not be repeated here.

[0140] According to some embodiments, image quality is affected by many factors during actual video recording, such as lens resolution, sensor noise, focusing accuracy, and the use of noise reduction algorithms in image processing. Among these factors, the lens's optical imaging transfer function is the foundation of imaging principles.

[0141] In some embodiments, the relationship between the object and the image is determined according to formula (1):

[0142] I Obs (x)=∫I true Formula (1) is (t)PSF(xt)dt+Noise

[0143] Among them, I Obs Images acquired by the camera; I true The image is the ideal image; PSF is the PSF corresponding to the camera; Noise is the noise.

[0144] In some embodiments, an example diagram illustrating the PSF shape of different image regions may be as follows: Figure 7 As shown, in the high field-of-view image regions closer to the edges, the PSF shape becomes more irregular due to increased aberrations. In the middle image regions, due to smaller aberrations, the PSF distribution is closer to a Gaussian shape. The single-dimensional distribution of the PSF in the central image region can be, for example, as shown below. Figure 8 As shown.

[0145] According to some embodiments, the terminal performs compensation processing on the first image based on the first compensation information, for example, by processing the first image using PSF deconvolution, such as... Figure 9 As shown, Object is the first image corresponding to the captured object, Image is the second image after processing the first image, and PSF is the PSF corresponding to the camera.

[0146] It is easy to understand that when the terminal acquires the first image, the terminal can, for example, perform compensation processing on at least one first image region based on the first compensation information corresponding to at least one first image region to obtain the second image.

[0147] S203, based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image, obtain the parameter ratio set;

[0148] The specific process is as described above, and will not be repeated here.

[0149] According to some embodiments, the parameter ratio refers to the ratio of a first parameter to a second parameter corresponding to the first parameter, and this parameter ratio does not specifically refer to the ratio of a particular parameter. For example, if the first parameter is a1 sharpness value and the second parameter is a2 sharpness value, the parameter ratio is the ratio of the first parameter a1 sharpness value to the second parameter a2 sharpness value.

[0150] In some embodiments, when the terminal obtains parameter ratios, it can add these parameter ratios to the same set to obtain a parameter ratio set. This parameter ratio set does not refer to a specific set of parameter ratios. For example, when the type of parameter ratio changes, the parameter ratio set will change accordingly. When the number of parameter ratios changes, the parameter ratio set will also change accordingly. The parameter ratio set may, for example, include sharpness ratios and noise ratios.

[0151] Optionally, the PSF needs to satisfy two conditions: improving image sharpness and controlling noise in flat image regions. Image sharpness can be evaluated, for example, by the contrast ratio at different frequencies, i.e., the ratio of the first sharpness value of any image region in the first image set in the first image to the second sharpness value of that image region in the second image. The noise ratio of flat image regions can be, for example, the ratio of the signal-to-noise ratio of the first image to the signal-to-noise ratio of the second image.

[0152] In some embodiments, when the Seidel function is used to describe the PSF, the wavefront aberration is determined according to formula (2):

[0153] W(ρ,θ)=Sρ 4 +Cρ 3 cosθ+Aρ 2cos 2 θ+Uρ 2 +Gρcosθ formula (2)

[0154] Where W represents wavefront aberration, S represents spherical aberration, C represents coma, A represents astigmatism, U represents field curvature, and G represents distortion. Adjusting the first-point function can be achieved, for example, by adjusting the five coefficients S, C, A, U, and G. The second-point function can be obtained by adjusting these five coefficients in the first-point function.

[0155] In some embodiments, when the terminal obtains the parameter ratio set, the terminal may obtain a first sharpness value corresponding to any image region in the first image and a second sharpness value corresponding to that image region in the second image. The terminal may obtain the ratio of the second sharpness value to the first sharpness value to obtain a sharpness ratio. When the terminal obtains the sharpness ratio, it may add the sharpness ratio to the parameter ratio set.

[0156] Optionally, the first image region set refers to the set of image regions obtained by dividing the image when the terminal acquires the image. The terminal may use the same image region division method for the first image and the second image.

[0157] Optionally, the terminal obtains the first sharpness value in the first image corresponding to any image region in the first image according to formula (3):

[0158]

[0159] Where K1 is the first sharpness value, v = 0.1 * Ny frequency, Ny frequency is the sampling frequency, MTF is the contrast value corresponding to the current frequency, MTF is obtained by using the knife edge algorithm to calculate the black and white diagonal edges of the checkerboard in the image area, and CSF refers to the human eye sensitivity response distribution.

[0160] Optionally, the terminal obtains the second sharpness value corresponding to the image region in the second image according to formula (4):

[0161]

[0162] Where K2 is the second sharpness value, v = 0.1 * Ny frequency, and CSF refers to the human eye sensitivity response distribution.

[0163] Optionally, the resolution ratio could be, for example, K = K2 / K1.

[0164] In some embodiments, when the terminal obtains the parameter ratio set, the terminal can obtain a first noise value corresponding to the all-white image region and the all-black image region in the first image, and a second noise value corresponding to the all-white image region and the all-black image region in the second image. The terminal can obtain the ratio of the second noise value to the first noise value to obtain the noise ratio.

[0165] Optionally, the first noise value S1 can be, for example, the mean of the squared differences between the average signal values ​​in the all-white and all-black image regions of the first image, i.e., the square root of each pixel signal minus the average value, calculated separately for the RGB channels. The second noise value S2 can be, for example, the mean of the squared differences between the average signal values ​​in the all-white and all-black image regions of the second image. The noise ratio can be, for example, S = S2 / S1.

[0166] For example, the first noise value corresponding to the all-white image region and the all-black image region in the first image acquired by the terminal is X1, and the second noise value corresponding to the all-white image region and the all-black image region in the second image is X2. The noise ratio can be, for example, X2 / X1.

[0167] In some embodiments, when the terminal obtains the sharpness ratio and the noise ratio, the terminal can adjust the coefficients of the first point spread function based on the sharpness ratio and the noise ratio to obtain the second point spread function.

[0168] It is easy to understand that when the terminal acquires the first image and the second image, it can acquire the first parameter set corresponding to the first image and the second parameter set corresponding to the second image. When the terminal acquires the first parameter set and the second parameter set, it can acquire the parameter ratio set. The parameter ratio set is used to adjust the coefficients of the first point spread function to obtain the second point spread function.

[0169] S204, Based on the parameter ratio set, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function;

[0170] The specific process is as described above, and will not be repeated here.

[0171] According to some embodiments, coefficient adjustment refers to the process whereby, when the terminal obtains a set of parameter ratios, it can adjust the PSF coefficients according to the parameter ratios to obtain the adjusted PSF coefficients. The terminal can then redetermine the point spread function based on the adjusted PSF coefficients, thereby obtaining the second point spread function.

[0172] In some embodiments, when the terminal obtains a second point diffusion function after adjusting the first point diffusion function, the terminal can obtain the initial coefficient set corresponding to the first point diffusion function and a set of coefficient ranges corresponding to at least one initial coefficient in the initial coefficient set. By traversing the set of coefficient ranges corresponding to any initial coefficient in the initial coefficient set, the terminal can obtain the target coefficient corresponding to the initial coefficient. By traversing the initial coefficient set, the terminal can obtain the target coefficient set corresponding to the first point diffusion function. Based on the target coefficient set, the terminal can adjust at least one initial coefficient corresponding to the first point diffusion function to obtain the second point diffusion function.

[0173] Optionally, the initial coefficient refers to the initial value assigned to the PSF based on the lens design values. This initial coefficient is not specific to any particular initial coefficient. For example, when the lens design values ​​change, this initial coefficient will change accordingly. The initial coefficient can be, for example, S0, C0, A0, U0, or G0.

[0174] Optionally, when the terminal obtains the initial coefficients, it adds them to the same set to obtain an initial coefficient set. This initial coefficient set does not refer to a specific set of initial coefficients. For example, when the type of initial coefficients changes, this initial coefficient set will change accordingly. When the assigned values ​​of the initial coefficients change, this initial coefficient set will also change accordingly.

[0175] Optionally, when the terminal obtains the initial coefficients, it can obtain the coefficient range corresponding to each initial coefficient. For example, the coefficient range of S can be S... min <S<S max The coefficient range of C can be C min <C<C max The range of coefficients for A can be A min <A<A max The coefficient range of U can be U min <U<U max The coefficient range of G can be G min <G<G max .

[0176] Optionally, when the terminal obtains the coefficient range, it adds the coefficient range to the same set, resulting in a set of coefficient ranges. This set of coefficient ranges does not specifically refer to any particular coefficient range. For example, when the type of coefficient range changes, this set of coefficient ranges will change accordingly. When the assigned value of a coefficient range changes, the coefficient range will also change accordingly.

[0177] Optionally, the target coefficient refers to the coefficient that satisfies the requirements for image sharpness and the noise requirement for flat image regions. This target coefficient is not specifically defined as any one target coefficient. For example, the target coefficient will change accordingly when the type of target coefficient changes. Target coefficients could be, for example, S1, C1, A1, U1, and G1. When the terminal obtains the target coefficients, it adds them to the same set to obtain the target coefficient set.

[0178] It is easy to understand that when the terminal obtains the second point diffusion function after adjusting the first point diffusion function, it can traverse the coefficient range corresponding to at least one coefficient, which can reduce the parameter determination time and reduce the calculation of coefficients when the terminal determines the second point diffusion function.

[0179] In some embodiments, the terminal adjusts the coefficients of the first point spread function based on the sharpness ratio and the noise ratio to obtain the second point spread function. When both the sharpness ratio and the noise ratio are greater than 1, it indicates that the compensation processing is effective and does not increase noise. The terminal can add these sharpness ratio and noise ratio to a historical parameter ratio set for selection. The terminal can select the maximum sharpness ratio and the maximum noise ratio from the historical parameter ratio set as the optimal group and store it in the camera's memory.

[0180] In some embodiments, the parameter ratio set includes a sharpness ratio and a noise ratio. When the terminal adjusts the coefficients of the first point spread function based on the parameter ratio set to obtain the second point spread function, the terminal can obtain a historical parameter ratio set. This historical parameter ratio set includes at least one historical sharpness ratio and at least one historical noise ratio. The terminal can obtain the maximum sharpness ratio among the sharpness ratios and at least one historical sharpness ratio, and obtain the maximum noise ratio among the noise ratios and at least one historical noise ratio. Based on the maximum sharpness ratio and the maximum noise ratio, the terminal can adjust the coefficients of the first point spread function to obtain the second point spread function. In other words, the terminal can select the maximum sharpness ratio and the maximum noise ratio to optimize the first point spread function, thus selecting the optimal second point spread function and applying it to the corresponding camera module. This can improve image acquisition quality and enhance the user experience.

[0181] It is easy to understand that the historical parameter ratio set refers to the parameter ratios that were obtained before the parameter ratio set was obtained. This historical parameter ratio set does not specifically refer to a fixed set of parameter ratios; for example, when the number of historical parameter ratios changes, this historical parameter ratio set can also change accordingly.

[0182] In some embodiments, when the terminal obtains a second point spread function adjusted from the first point spread function based on a first parameter set corresponding to the first image and a second parameter set corresponding to the second image, the terminal can send the first image and the second image to the server. The first image and the second image instruct the server to adjust the first point spread function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image to obtain the second point spread function. The server can send the second point spread function to the terminal. The terminal can obtain the second point spread function sent by the server. Sending the first image and the second image to the server reduces the overhead of coefficient adjustment, reduces the terminal's computational process, and increases the terminal's usage time.

[0183] Optionally, the interactive graph for server-side image processing can be, for example, as follows: Figure 10 As shown, the terminal sends the first and second images to the server for processing, which can reduce the terminal's computation and thus reduce terminal wear and tear.

[0184] It is easy to understand that when the terminal obtains the set of parameter ratios, it can obtain the optimal parameter ratios. Based on the optimal parameter ratios, the terminal can adjust the coefficients of the first-point diffusion function to obtain the second-point diffusion function.

[0185] In some embodiments, when the terminal obtains a second point diffusion function based on a first parameter set corresponding to the first image and / or a second parameter set corresponding to the second image, if either the first or second parameter set does not meet the parameter conditions, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function. For example, if the first parameter set does not meet the parameter conditions, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function. Alternatively, if the second parameter set does not meet the parameter conditions, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0186] In some embodiments, the first parameter set refers to the parameter set corresponding to the first image, and the second parameter set refers to the parameter set corresponding to the second image. The parameter condition is used to determine whether to adjust the first point spread function. This parameter condition does not specifically refer to a fixed condition. For example, when the types of parameters included in the first parameter set change, the parameter condition can also change accordingly.

[0187] According to some embodiments, the parameter condition may be, for example, that the sharpness value is greater than a sharpness threshold. For instance, when the sharpness value in the second parameter set is less than the sharpness threshold, the terminal can adjust the first point spread function to obtain the second point spread function.

[0188] S205, acquire the third image captured by the camera module;

[0189] The specific process is as described above, and will not be repeated here.

[0190] According to some embodiments, the third image refers to the user setting the image card and ambient lighting, and selecting the focal length, shooting distance and temperature sampling point module, dividing the object distance into multiple equal points by multiples, dividing the imaging field of view into N×M image regions, and using the terminal to control the camera to shoot at different positions to obtain different third images.

[0191] Optionally, the chart refers to an image test chart. The chart can be, for example, a resolution test chart (Spatial Frequency Response, SFR), a Siemens star chart, or a multi-frequency modulation transfer function (MTF) evaluation chart.

[0192] Optionally, the SFR test chart TE261 features a tilted checkerboard background with five low-contrast tilted edges and grayscale. The SFR test chart is used to determine the SFR of digital capture electronics and is the default chart used in combination with AF-Box and STEVE.

[0193] In simple terms, when a user sets the graphics card and ambient lighting, and selects the focal length, shooting distance, and temperature sampling point module, they can send a shooting command to the camera via the terminal. The terminal can then obtain the third image captured by the camera module.

[0194] S206, Divide the third image into image regions to obtain a set of second image regions corresponding to the third image;

[0195] The specific process is as described above, and will not be repeated here.

[0196] According to some embodiments, the second image region set refers to the set of regions obtained after the terminal divides the third image into image regions when it acquires the third image. For example, the third image is divided into at least two sub-image regions on average. For example, the terminal can divide the third image into 21*21 second image regions. The terminal can also divide the third image into 11*11 second image regions. The terminal adds the divided second image regions to the same set to obtain the second image region set.

[0197] It is easy to understand that when the terminal acquires a third image, it can divide the third image into at least two second image regions. The terminal adds the divided second image regions to the same set, thus obtaining a set of second image regions corresponding to the third image.

[0198] S207, based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set;

[0199] The specific process is as described above, and will not be repeated here.

[0200] According to some embodiments, the second compensation information refers to the compensation value obtained when evaluating the sharpness and noise of each second image region distribution; that is, the second compensation information refers to the compensation information corresponding to the second point spread function. This second compensation information does not specifically refer to a fixed compensation value. For example, when the second image region changes, the second compensation information will change accordingly. When the third image changes, the second compensation information will also change accordingly. For example, the second compensation information can be a data table of different N*M two-dimensional matrices corresponding to different focal lengths, different object distances, and different image fields of view.

[0201] In some embodiments, when the terminal evaluates the sharpness and noise of each second image region distribution, it uses the noise ratio and sharpness ratio to obtain multiple sampling points. The PSF compensation information corresponding to the second image region differs depending on the focal length, distance, and temperature. When the terminal obtains the second compensation information corresponding to at least one second image region, it can store this compensation information in a memory module.

[0202] It is easy to understand that when the terminal obtains the second point diffusion function, it can obtain the second compensation information corresponding to at least one second image region in the second image region set.

[0203] In some embodiments, when the terminal obtains the second compensation information corresponding to at least one image region in the second image region set based on the second point diffusion function, for example, the terminal may obtain the function expression corresponding to the second point diffusion function and perform an inverse transformation on the function expression to obtain the second compensation information corresponding to at least one image region in the second image region set.

[0204] S208, based on the second compensation information corresponding to at least one second image region, the third image is processed to obtain the fourth image.

[0205] The specific process is as described above, and will not be repeated here.

[0206] According to some embodiments, when the terminal obtains second compensation information corresponding to at least one second image region, the terminal can select compensation information stored in the memory module based on shooting information, such as camera model, chip information, shooting distance, environmental data, and shooting conditions. The terminal can perform compensation processing on second image regions with different focal lengths, shooting distances, and fields of view. This compensation processing includes, but is not limited to, intermediate interpolation calculations. Figure 11 As shown, where, Figure 11 (a) shows the PSF distribution of a single sub-image region. Figure 11(b) is the initial SFR image. Figure 11 (c) is the compensated SFR image.

[0207] In some embodiments, when processing a third image, the terminal may process one third image or multiple third images. For example, the terminal may control the camera to take pictures at preset intervals before and after the focus distance to acquire multiple third images. The terminal may also control the camera to change the exposure conditions and take pictures at different exposure levels to acquire multiple third images. The terminal may process and superimpose multiple third images into a single image.

[0208] It is easy to understand that when the terminal obtains second compensation information corresponding to at least one second image region, it can select the compensation information stored in the memory module to perform compensation processing on the third image to obtain the fourth image. An example illustration of the terminal displaying the fourth image could be... Figure 12 .

[0209] In this embodiment, a first image captured by a control camera is acquired; a second image is obtained by processing the first image using a first point spread function (PSF); and a parameter ratio set is obtained based on a first parameter set corresponding to the first image and a second parameter set corresponding to the second image. Therefore, adjusting the first PSF can improve the clarity of the acquired image and reduce the impact of noise, thus improving the quality of the acquired image. Secondly, by adjusting the coefficients of the first PSF based on the parameter ratio set, a second PSF can be obtained, which optimizes the PSF and improves the quality of the acquired image. This means that optimized compensation information can be used to compensate for the image, reducing the problems of unclear images and severe noise when directly acquiring images based on the first PSF, thereby improving the quality of the acquired image and enhancing the user experience. Furthermore, a third image captured by the control camera is divided into image regions to obtain a set of second image regions corresponding to the third image. Based on the second PSF, second compensation information corresponding to at least one second image region in the second image region set is obtained. Based on the second compensation information corresponding to at least one second image region, the third image is processed to obtain a fourth image, which improves the accuracy of PSF compensation information acquisition and enhances the quality of the acquired image. Finally, by acquiring lens compensation information and processing the captured images to compensate for lens aberrations, we can reduce large differences in field curvature caused by tolerance variations and different focus distances, thereby improving image quality and enhancing the user experience.

[0210] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0211] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0212] Please see Figure 13 This illustration shows a schematic diagram of the structure of a first image processing apparatus provided in an exemplary embodiment of the present disclosure. The image processing apparatus can be implemented as all or part of an apparatus through software, hardware, or a combination of both. The image processing apparatus 1300 includes an image acquisition unit 1301, an information acquisition unit 1302, an image processing unit 1303, and a function acquisition unit 1304, wherein:

[0213] The image acquisition unit 1301 is used to divide the first image acquired by the camera module into image regions to obtain a first image region set corresponding to the first image.

[0214] The information acquisition unit 1302 is used to acquire first compensation information corresponding to at least one first image region in the first image region set based on the first point diffusion function;

[0215] Image processing unit 1303 is used to perform compensation processing on at least one first image region based on first compensation information corresponding to at least one first image region to obtain a second image;

[0216] The function acquisition unit 1304 is used to acquire the second point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image.

[0217] Optional, Figure 14 A schematic diagram of the structure of a second image processing apparatus provided in an embodiment of this disclosure is shown, such as... Figure 14 As shown, the function acquisition unit 1304 includes a ratio acquisition subunit 1314 and a coefficient adjustment subunit 1324. The function acquisition unit 1304 is used to acquire the second point diffusion function after adjusting the first point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image.

[0218] The ratio acquisition subunit 1314 is used to acquire a set of parameter ratios based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image.

[0219] Coefficient adjustment subunit 1324 is used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function or

[0220] The coefficient adjustment subunit is further configured to adjust the coefficients of the first point diffusion function to obtain the second point diffusion function if the first parameter set or the second parameter set does not meet the parameter conditions.

[0221] Optionally, the parameter ratio set includes sharpness ratio and noise ratio. The coefficient adjustment subunit 1324 is used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function. Specifically, it is used for:

[0222] Obtain the first sharpness value of any image region in the first image region set in the first image and the second sharpness value of the image region in the second image;

[0223] Obtain the ratio of the second sharpness value to the first sharpness value to get the sharpness ratio;

[0224] Obtain the first noise value corresponding to the all-white image region and the all-black image region in the first image, and the second noise value corresponding to the all-white image region and the all-black image region in the second image;

[0225] The ratio of the second noise value to the first noise value is obtained to get the noise ratio.

[0226] Based on the sharpness ratio and noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0227] Optionally, the parameter ratio set includes sharpness ratio and noise ratio. The coefficient adjustment subunit 1324 is used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function. Specifically, it is used for:

[0228] Obtain a set of historical parameter ratios, which includes at least one historical sharpness ratio and at least one historical noise ratio;

[0229] Obtain the sharpness ratio and the maximum sharpness ratio among at least one historical sharpness ratio; obtain the noise ratio and the maximum noise ratio among at least one historical noise ratio.

[0230] Based on the maximum sharpness ratio and the maximum noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

[0231] Optional, Figure 15 A schematic diagram of the structure of the third image processing apparatus provided in the embodiments of this disclosure is shown, such as... Figure 15As shown, the image processing device 1300 also includes a fourth image acquisition unit 1305, which is used to acquire a second point spread function after adjusting the first point spread function, and then acquire a third image captured by the camera module.

[0232] The third image is divided into image regions to obtain a set of second image regions corresponding to the third image.

[0233] Based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set;

[0234] Based on the second compensation information corresponding to at least one second image region, the third image is processed to obtain the fourth image.

[0235] Optionally, the image acquisition unit 1301, when dividing the first image acquired by the camera module into image regions to obtain a set of first image regions corresponding to the first image, is specifically used for:

[0236] Obtain the module identifier corresponding to the camera module;

[0237] Obtain the region division information corresponding to the module identifier;

[0238] Based on the region segmentation information, the first image captured by the camera module is divided into image regions to obtain a set of first image regions corresponding to the first image.

[0239] Optionally, the function acquisition unit 1305 is used to acquire the second point diffusion function after adjusting the first point diffusion function, specifically for:

[0240] Obtain the set of initial coefficients corresponding to the first point diffusion function and the set of coefficient ranges corresponding to at least one initial coefficient in the set of initial coefficients;

[0241] Iterate through the set of coefficient ranges corresponding to any initial coefficient in the initial coefficient set to obtain the target coefficient corresponding to the initial coefficient;

[0242] Traverse the initial coefficient set to obtain the target coefficient set corresponding to the first point diffusion function;

[0243] Based on the target set of coefficients, at least one initial coefficient corresponding to the first point diffusion function is adjusted to obtain the second point diffusion function.

[0244] It should be noted that the image processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when executing the image processing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0245] The sequence numbers of the embodiments disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0246] In this embodiment, the image acquisition unit can divide the first image captured by the camera module into image regions to obtain a set of first image regions corresponding to the first image; the information acquisition unit can obtain first compensation information corresponding to at least one first image region in the set of first image regions based on a first point spread function; the image processing unit can perform compensation processing on at least one first image region based on the first compensation information corresponding to at least one first image region to obtain a second image; and the function acquisition unit can obtain a second point spread function based on a first parameter set corresponding to the first image and a second parameter set corresponding to the second image. Therefore, the first point spread function can be adjusted to obtain an adjusted second point spread function, and the image can be acquired based on the adjusted second point spread function. This means that different compensation information can be used to compensate the image, reducing the problem of unclear images and severe noise when directly acquiring images based on the first point spread function, improving the quality of the acquired image, and thus enhancing the user experience.

[0247] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0248] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0249] Figure 16A schematic block diagram of an example electronic device 1600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable electronic devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0250] like Figure 16 As shown, the electronic device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1602 or a computer program loaded from a storage unit 1608 into a random access memory (RAM) 1603. The RAM 1603 may also store various programs and data required for the operation of the electronic device 1600. The computing unit 1601, ROM 1602, and RAM 1603 are interconnected via a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.

[0251] Multiple components in electronic device 1600 are connected to I / O interface 1605, including: input unit 1606, such as keyboard, mouse, etc.; output unit 1607, such as various types of monitors, speakers, etc.; storage unit 1608, such as disk, optical disk, etc.; and communication unit 1609, such as network card, modem, wireless transceiver, etc. Communication unit 1609 allows electronic device 1600 to exchange information / data with other electronic devices through computer networks such as the Internet and / or various telecommunications networks.

[0252] The computing unit 1601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1601 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1600 via ROM 1602 and / or communication unit 1609. When the computer program is loaded into RAM 1603 and executed by the computing unit 1601, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1601 may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

[0253] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic electronic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0254] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0255] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or electronic devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage electronics, magnetic storage electronics, or any suitable combination of the foregoing.

[0256] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0257] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0258] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0259] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0260] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, characterized in that, include: The first image captured by the camera module is divided into image regions to obtain a set of first image regions corresponding to the first image. Based on the first point diffusion function, obtain the first compensation information corresponding to at least one first image region in the first image region set; Based on the first compensation information corresponding to the at least one first image region, compensation processing is performed on the at least one first image region to obtain a second image; Based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image, obtain a parameter ratio set; Based on the set of parameter ratios, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function, wherein the set of parameter ratios includes sharpness ratio and noise ratio; Acquire the third image captured by the camera module; The third image is divided into image regions to obtain a set of second image regions corresponding to the third image. Based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set; Based on the second compensation information corresponding to the at least one second image region, the third image is processed to obtain the fourth image.

2. The method according to claim 1, characterized in that, The step of adjusting the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function includes: Obtain the first sharpness value of any image region in the first image region set in the first image and the second sharpness value of any image region in the second image; The ratio of the second sharpness value to the first sharpness value is obtained to obtain the sharpness ratio. Obtain the first noise value corresponding to the all-white image region and the all-black image region in the first image, and the second noise value corresponding to the all-white image region and the all-black image region in the second image; The ratio of the second noise value to the first noise value is obtained to obtain the noise ratio. Based on the sharpness ratio and the noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

3. The method according to claim 1, characterized in that, The step of adjusting the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function includes: Obtain a set of historical parameter ratios, wherein the set of historical parameter ratios includes at least one historical sharpness ratio and at least one historical noise ratio; Obtain the maximum sharpness ratio among the sharpness ratio and the at least one historical sharpness ratio; obtain the maximum noise ratio among the noise ratio and the at least one historical noise ratio. Based on the maximum sharpness ratio and the maximum noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

4. The method according to claim 1, characterized in that, The step of dividing the first image captured by the camera module into image regions to obtain a first image region set corresponding to the first image includes: Obtain the module identifier corresponding to the camera module; Obtain the region division information corresponding to the module identifier; Based on the region division information, the first image captured by the camera module is divided into image regions to obtain a first image region set corresponding to the first image.

5. The method according to claim 1, characterized in that, The step of adjusting the coefficients of the first point diffusion function to obtain the second point diffusion function includes: Obtain the set of initial coefficients corresponding to the first point diffusion function and the set of coefficient ranges corresponding to at least one initial coefficient in the set of initial coefficients; Traverse the set of coefficient ranges corresponding to any initial coefficient in the set of initial coefficients to obtain the target coefficient corresponding to the initial coefficient; By traversing the initial set of coefficients, the target set of coefficients corresponding to the first point diffusion function is obtained; Based on the target set of coefficients, the at least one initial coefficient corresponding to the first point diffusion function is adjusted to obtain the second point diffusion function.

6. An image processing apparatus, characterized in that, include: The image acquisition unit is used to divide the first image acquired by the camera module into image regions to obtain a first image region set corresponding to the first image. The information acquisition unit is used to acquire first compensation information corresponding to at least one first image region in the first image region set based on the first point diffusion function; An image processing unit is configured to perform compensation processing on the at least one first image region based on first compensation information corresponding to at least one first image region to obtain a second image. The function acquisition unit is used to acquire the second point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image; The function acquisition unit includes a ratio acquisition subunit and a coefficient adjustment subunit. The function acquisition unit is used to acquire the second point diffusion function after adjusting the first point diffusion function based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image. The ratio acquisition subunit is used to acquire a set of parameter ratios based on the first parameter set corresponding to the first image and the second parameter set corresponding to the second image; The coefficient adjustment subunit is used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function, wherein the parameter ratio set includes a sharpness ratio and a noise ratio. The device further includes a fourth image acquisition unit, which is used to acquire a second point diffusion function after adjusting the first point diffusion function, and then acquire a third image captured by the camera module. The third image is divided into image regions to obtain a set of second image regions corresponding to the third image. Based on the second point diffusion function, obtain the second compensation information corresponding to at least one second image region in the second image region set; Based on the second compensation information corresponding to the at least one second image region, the third image is processed to obtain the fourth image.

7. The apparatus according to claim 6, characterized in that, The coefficient adjustment subunit, used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function, specifically is used for: Obtain the first sharpness value of any image region in the first image region set in the first image and the second sharpness value of any image region in the second image; The ratio of the second sharpness value to the first sharpness value is obtained to obtain the sharpness ratio. Obtain the first noise value corresponding to the all-white image region and the all-black image region in the first image, and the second noise value corresponding to the all-white image region and the all-black image region in the second image; The ratio of the second noise value to the first noise value is obtained to obtain the noise ratio. Based on the sharpness ratio and the noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

8. The apparatus according to claim 6, characterized in that, The coefficient adjustment subunit, used to adjust the coefficients of the first point diffusion function based on the parameter ratio set to obtain the second point diffusion function, specifically is used for: Obtain a set of historical parameter ratios, wherein the set of historical parameter ratios includes at least one historical sharpness ratio and at least one historical noise ratio; Obtain the maximum sharpness ratio among the sharpness ratio and the at least one historical sharpness ratio; obtain the maximum noise ratio among the noise ratio and the at least one historical noise ratio. Based on the maximum sharpness ratio and the maximum noise ratio, the coefficients of the first point diffusion function are adjusted to obtain the second point diffusion function.

9. The apparatus according to claim 6, characterized in that, The image acquisition unit, when dividing the first image acquired by the camera module into image regions to obtain a first image region set corresponding to the first image, specifically performs the following: Obtain the module identifier corresponding to the camera module; Obtain the region division information corresponding to the module identifier; Based on the region division information, the first image captured by the camera module is divided into image regions to obtain a first image region set corresponding to the first image.

10. The apparatus according to claim 6, characterized in that, The function acquisition unit, when acquiring the second point diffusion function after adjusting the first point diffusion function, is specifically used for: Obtain the set of initial coefficients corresponding to the first point diffusion function and the set of coefficient ranges corresponding to at least one initial coefficient in the set of initial coefficients; Traverse the set of coefficient ranges corresponding to any initial coefficient in the set of initial coefficients to obtain the target coefficient corresponding to the initial coefficient; By traversing the initial set of coefficients, the target set of coefficients corresponding to the first point diffusion function is obtained; Based on the target set of coefficients, the at least one initial coefficient corresponding to the first point diffusion function is adjusted to obtain the second point diffusion function.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images