Image edge brightness correction method, device and system

Abstract

The present application provides an image edge brightness correction method, device and system, the correction method includes: photographing a target plate with a specific pattern to obtain an image of the specific pattern; determining the abnormal area of ​​the image edge through the standard brightness gradient curve of the image, and calculating the brightness deviation coefficient of the abnormal area; determining the correctable area in the abnormal area based on the standard resolution change curve of the image; and performing brightness correction on the correctable area according to the brightness deviation coefficient. At the same time, the image after brightness correction is reconstructed through a trained convolutional neural network model to obtain a large field of view image with high edge clarity, so that the corrected image can have the characteristics of high edge clarity, smooth image and strong sense of reality while retaining the maximum field of view.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to a method and device for image edge brightness correction. Background Art

[0002] As wide-angle selfies and macro photography become "necessary" camera functions for portable electronic products such as mobile phones, more and more portable electronic products are beginning to use large field of view (FOV) camera modules. Compared with conventional camera modules, large FOV camera modules have a larger viewing range, and users can get a wider picture at the same shooting position. At the same time, images taken under macro conditions also have effects that conventional camera modules do not have, such as obtaining detailed information of the subject at close range. However, the edge area of ​​the image of the large FOV camera module will inevitably have a problem of reduced brightness.

[0003] In addition, the free-form surface technology used to improve the distortion of large FOV camera modules will also bring about the same problem, that is, the brightness of the edge area of ​​the image decreases, resulting in a smaller area that can actually be displayed in the captured image.

[0004] In addition, as electronic products such as smartphones become thinner, the height of the camera modules they are equipped with is also getting smaller and smaller. Under the same conditions, the edge areas of images taken by camera modules with large FOV, macro and other characteristics will experience a problem of decreased brightness.

[0005] When the demand for large FOV camera modules and ultra-thin mobile phones becomes inevitable, the correction method for the brightness of the edge area of ​​the captured image has also become an urgent need. At present, most of the known methods at home and abroad are to achieve brightness correction by constructing a functional relationship between the brightness of the light source itself and the image data displayed by the terminal, and adjusting the data of the displayed image. However, in the existing methods, it is usually impossible to judge the effective information of the image, resulting in the problem of imaging distortion in the edge area of ​​the image.

[0006] Therefore, there is an urgent need for an image edge brightness correction method and device that can solve the problem of image brightness reduction and ensure that the image is not distorted. Summary of the invention

[0007] The present application provides a method and a device using the method that can at least solve or partially solve at least one of the above-mentioned shortcomings in the prior art.

[0008] On the one hand, the present application provides a method for correcting image edge brightness, the method comprising: photographing a target plate having a specific pattern to obtain an image of the specific pattern; determining an abnormal area at the edge of the image through a standard brightness gradient curve of the image, and calculating a brightness deviation coefficient of the abnormal area; determining a correctable area in the abnormal area based on a standard resolution change curve of the image; and performing brightness correction on the correctable area according to the brightness deviation coefficient.

[0009] According to an embodiment of the present application, the method also includes: training a convolutional neural network model for image edge brightness correction; and reconstructing the image after brightness correction through the convolutional neural network model.

[0010] According to an embodiment of the present application, training a convolutional neural network model for image edge brightness correction includes: building a convolutional neural network model with a feature extraction layer and a reconstruction layer; selecting a brightness abnormality image for training and a standard image corresponding to the brightness abnormality image; defining an edge area of ​​the brightness abnormality image; extracting feature data of the edge area in the feature extraction layer; optimizing the edge area in the reconstruction layer; comparing the optimized result image with the standard image to determine the difference; when the difference is greater than a predetermined value, adjusting the parameters of the feature extraction layer and the reconstruction layer according to the difference; and repeating the above steps until the difference is less than the predetermined value.

[0011] According to an embodiment of the present application, a target plate having a specific pattern is photographed, and obtaining an image of the specific pattern includes: arranging a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane, and drawing a plurality of strips extending from the center of the square plane to the four sides and four corners and equally dividing the square plane to form the specific pattern; and setting the center of the target plate on the optical axis of a camera module, and making the shooting range of the camera module equal to the size of the specific pattern, so as to obtain an image of the specific pattern.

[0012] According to an embodiment of the present application, arranging a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane also includes: arranging the first color block and the second color block in an area outside the strip in the square plane; setting the first color block and the second color block to different colors; and setting a marking point on at least one of the first color block and the second color block.

[0013] According to an implementation manner of the present application, the method also includes: generating a standard brightness gradient curve for the image, including: drawing a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; calculating the average value of the brightness gradient curves of multiple strips to obtain an average brightness gradient curve for the image; determining the edge field of view and the central field of view in the image; fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and merging the average brightness gradient curve within the central field of view with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

[0014] According to an embodiment of the present application, generating the standard brightness gradient curve also includes: comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view to calculate a deviation value; and correcting the standard brightness gradient curve using the deviation value.

[0015] According to an embodiment of the present application, fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view also includes: fitting the brightness gradient curve within the edge field of view using a least squares method.

[0016] According to an implementation manner of the present application, determining the abnormal area at the edge of the image through the standard brightness gradient curve of the image includes: setting a brightness threshold; subtracting the grayscale value of each marking point in the edge field of view from the corresponding value in the standard brightness gradient curve; and setting the first color block or the second color block where the marking point whose difference exceeds the brightness threshold is located as the abnormal area.

[0017] According to an embodiment of the present application, calculating the brightness deviation coefficient of the abnormal area includes: setting the ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

[0018] According to an embodiment of the present application, the method also includes: generating a standard resolution change curve of the image, including: obtaining the resolution of each marked point in the image and drawing the resolution change curve of the image; fitting the resolution change curve in the edge field of view according to the resolution change curve of the image in the central field of view; and merging the resolution change curve of the image in the central field of view with the fitted resolution change curve in the edge field of view to generate the standard resolution change curve.

[0019] According to an embodiment of the present application, fitting the resolution change curve in the edge field of view according to the resolution change curve of the image in the central field of view also includes: fitting the resolution change curve in the edge field of view using the least squares method.

[0020] According to an embodiment of the present application, obtaining the resolution of each marked point in the image and drawing a resolution change curve of the image further includes: obtaining the SFR value of each marked point in the image and drawing an SFR value change curve of the image.

[0021] According to an implementation manner of the present application, based on the standard resolution change curve of the image, determining the correctable area in the abnormal area includes: setting a resolution threshold; comparing the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and setting the abnormal area where the marked points below the resolution threshold are located as an area with severe information loss, and setting the abnormal area where the marked points above the resolution threshold are located as a correctable area.

[0022] According to an embodiment of the present application, performing brightness correction on the correctable area according to the brightness deviation coefficient further includes: using the brightness deviation coefficient to obtain a brightness-corrected image by least squares fitting.

[0023] On the other hand, the present application also provides an image edge brightness correction system, the system comprising: a memory storing computer-readable instructions; and a processor connected to the memory, executing the instructions to complete the following operations: determining an abnormal area at the edge of the image according to a standard brightness gradient curve of an image with a specific pattern of a target plate, and calculating a brightness deviation coefficient of the abnormal area; determining a correctable area in the abnormal area based on a standard resolution change curve of the image; and performing brightness correction on the correctable area according to the brightness deviation coefficient.

[0024] According to an embodiment of the present application, the processor executes the instruction to further: train a convolutional neural network model for image edge brightness correction; and reconstruct the image after brightness correction through the convolutional neural network model.

[0025] According to an embodiment of the present application, the processor executes the instructions to train a convolutional neural network model for image edge brightness correction, including: building a convolutional neural network model with a feature extraction layer and a reconstruction layer; selecting a brightness abnormality image and a standard image corresponding to the brightness abnormality image for training; defining the edge area of ​​the brightness abnormality image; extracting feature data of the edge area in the feature extraction layer; optimizing the edge area in the reconstruction layer; comparing the optimized result image with the standard image to determine the difference; when the difference is greater than a predetermined value, adjusting the parameters of the feature extraction layer and the reconstruction layer according to the difference; and repeating the above steps until the difference is less than the predetermined value.

[0026] According to an embodiment of the present application, the specific pattern includes: a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally, wherein the first color block and the second color block are arranged in the area outside the strips in the square plane; the first color block and the second color block have different colors; and at least one of the first color block and the second color block is provided with a marking point.

[0027] According to an implementation manner of the present application, the processor executes the instruction to further: generate a standard brightness gradient curve for the image, including: drawing a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; taking the average of multiple strip brightness gradient curves to obtain an average brightness gradient curve for the image; determining the edge field of view and the central field of view in the image; fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and merging the average brightness gradient curve within the central field of view with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

[0028] According to an embodiment of the present application, the processor executes the instruction to further: generate the standard brightness gradient curve, including: comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view with each other, calculating the deviation value; and correcting the standard brightness gradient curve with the deviation value.

[0029] According to an embodiment of the present application, the processor executes the instruction to fit the brightness gradient curve within the edge field of view using the least squares method.

[0030] According to an implementation manner of the present application, based on the target plate, the processor executes the instructions to determine the abnormal area at the edge of the image according to the following steps: setting a brightness threshold; subtracting the grayscale value of each marked point in the edge field of view from the corresponding value in the standard brightness gradient curve; and setting the first color block or the second color block where the marked point whose difference exceeds the brightness threshold is located as the abnormal area.

[0031] According to an embodiment of the present application, the processor executes the instruction to set the ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

[0032] According to an implementation manner of the present application, the processor executes the instruction to further: generate a standard resolution change curve of the image, including: obtaining the resolution of each landmark point in the image to draw the resolution change curve of the image; fitting the resolution change curve in the edge field of view according to the resolution change curve of the image in the central field of view; and merging the resolution change curve of the image in the central field of view with the fitted resolution change curve in the edge field of view to generate the standard resolution change curve.

[0033] According to an embodiment of the present application, the processor executes the instruction to use the least squares method to fit the resolution change curve within the edge field of view.

[0034] According to an embodiment of the present application, the processor executes the instruction to obtain the SFR value of each marked point in the image and draw an SFR value change curve of the image.

[0035] According to an embodiment of the present application, the processor executes the instruction to determine the correctable area in the abnormal area through the following processing: setting a resolution threshold; comparing the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and setting the abnormal area where the marked points below the resolution threshold are located as an area with severe information loss, and setting the abnormal area where the marked points above the resolution threshold are located as a correctable area.

[0036] According to an embodiment of the present application, the processor executes the instruction to obtain a brightness-corrected image by least squares fitting using the brightness deviation coefficient.

[0037] On the other hand, the present application also provides a device for an image edge brightness correction method, the device comprising: a module fixture for fixing a camera module to be tested; a target plate with a specific pattern, the center of the target plate is set on the optical axis of the camera module, and the shooting range of the camera module is equal to the size of the specific pattern to obtain an image with the specific pattern; an image data generator for obtaining the grayscale value and resolution of the image and correcting the edge brightness of the image; and a reconstructor, which is provided with a convolutional neural network model for correcting the image edge brightness.

[0038] According to an embodiment of the present application, the convolutional neural network model includes: a feature extraction layer, which is used to extract features from an image to be processed or a result image of a previous edge brightness correction process to obtain multiple feature maps; and a reconstruction layer, which is located after the feature extraction layer, and is used to perform deconvolution processing on the extracted multiple feature maps to reconstruct the image to be processed based on the extracted multiple feature maps.

[0039] According to an embodiment of the present application, the device also includes a trainer for training the convolutional neural network model, wherein the trainer is configured to: select a brightness abnormality image for training and a standard image corresponding to the brightness abnormality image; define an edge area of ​​the brightness abnormality image; extract feature data of the edge area in the feature extraction layer; optimize the edge area in the reconstruction layer; compare the optimized result image with the standard image to determine the difference; when the difference is greater than a predetermined value, adjust the parameters of the feature extraction layer and the reconstruction layer according to the difference; and repeat the above steps until the difference is less than the predetermined value.

[0040] According to an embodiment of the present application, the specific pattern includes: a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally, wherein the first color block and the second color block are arranged in the area outside the strips in the square plane; the first color block and the second color block have different colors; and a marking point is set on at least one of the first color block and the second color block.

[0041] According to an embodiment of the present application, the image data generator is configured to: determine the abnormal area at the edge of the image through the standard brightness gradient curve of the image, and calculate the brightness deviation coefficient of the abnormal area; determine the correctable area in the abnormal area based on the standard resolution change curve of the image; and perform brightness correction on the correctable area according to the brightness deviation coefficient.

[0042] According to an embodiment of the present application, the image data generator is configured to make a difference between the grayscale value of each marking point in the edge field of view and the corresponding value in the standard brightness gradient curve; and set the first color block or the second color block where the marking point whose difference exceeds a preset brightness threshold is located as the abnormal area.

[0043] According to an embodiment of the present application, the image data generator is configured to set the ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

[0044] According to an embodiment of the present application, the image data generator is configured to compare the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and set the abnormal area where the marked points below a preset resolution threshold are located as an area with severe information loss, and set the abnormal area where the marked points above the resolution threshold are located as a correctable area.

[0045] According to an embodiment of the present application, the image data generator is configured to use the brightness deviation coefficient to obtain a brightness-corrected image through least squares fitting.

[0046] According to an embodiment of the present application, the image data generator is further configured to generate a standard brightness gradient curve for the image, including: drawing a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; taking the average of the brightness gradient curves of multiple strips to obtain an average brightness gradient curve for the image; determining the edge field of view and the central field of view in the image; fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and merging the average brightness gradient curve within the central field of view with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

[0047] According to an embodiment of the present application, the image data generator is further configured to generate a standard brightness gradient curve for the image, including: comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view, calculating a deviation value; and correcting the standard brightness gradient curve using the deviation value.

[0048] According to an embodiment of the present application, the image data generator is further configured to generate a standard resolution change curve of the image, including: obtaining the resolution of each marked point in the image to draw the resolution change curve of the image; fitting the resolution change curve in the edge field of view according to the resolution change curve of the image in the central field of view; and merging the resolution change curve of the image in the central field of view with the fitted resolution change curve in the edge field of view to generate the standard resolution change curve.

[0049] According to at least one solution of the image edge brightness correction method and device provided above in the present application, at least one of the following beneficial effects can be achieved:

[0050] 1. The present application provides a novel method for image edge brightness correction. For images taken by a large FOV camera module, a neural network model is used to perform pixel-level processing on the edge area, so that the corrected image can have the characteristics of high edge clarity, smooth image, and strong sense of reality while retaining the maximum FOV.

[0051] 2. The present application provides a new device for image edge brightness correction. Without the need to introduce a third-party measurement device, the image taken by a large FOV camera module can be effectively repaired by using a reconstruction system with a convolutional neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0053] Figure 1 is a flow chart of an image edge brightness correction method according to an embodiment of the present application;

[0054] Figure 2 It is a specific pattern of a target plate in an image edge brightness correction device according to one embodiment of the present application;

[0055] Figure 3 is a schematic diagram of an image edge brightness correction method according to another embodiment of the present application;

[0056] Figure 4 is a flow chart of a method for training a convolutional neural network model according to one embodiment of the present application; and

[0057] Figure 5 A schematic diagram of the structure of a computer system of a terminal device or server suitable for implementing the present application is shown. DETAILED DESCRIPTION

[0058] In order to better understand the present application, a more detailed description will be made of various aspects of the present application with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application, and are not intended to limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0059] It should be noted that in this specification, the expressions first, second, third, etc. are only used to distinguish one feature from another feature, and do not represent any limitation on the feature. Therefore, without departing from the teaching of this application, the first color block discussed below may also be referred to as the second color block. And vice versa.

[0060] In the drawings, the thickness, size and shape of the components have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not drawn strictly to scale. As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for the inherent deviations in measurements or calculations that would be recognized by one of ordinary skill in the art.

[0061] It should also be understood that expressions such as "include", "including", "have", "contain" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0062] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0063] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. In addition, unless explicitly limited or contradictory to the context, the specific steps included in the camera module recorded in the present application are not necessarily limited to the recorded order, but can be performed in any order or in parallel. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0064] Figure 1 The figure is a flow chart of a method for correcting image edge brightness according to one embodiment of the present application.

[0065] Reference Figure 1 According to an embodiment of the present application, an image edge brightness correction method 100 includes: step S101, photographing a target plate with a specific pattern to obtain an image of the specific pattern; step S102, determining the abnormal area of ​​the image edge through the standard brightness gradient curve of the image, and calculating the brightness deviation coefficient of the abnormal area; step S103, determining the correctable area in the abnormal area based on the standard resolution change curve of the image; and step S104, performing brightness correction on the correctable area according to the brightness deviation coefficient.

[0066] The above steps will be described in detail below.

[0067] The image to be corrected in step S101 can be obtained by a camera module under test fixed on a module fixture shooting a target board with a specific pattern. The center of the target board can be set on the optical axis of the camera module, and it is required that the shooting range of the camera module is equal to the size of the specific pattern, so that the specific pattern on the target board can fill the entire image to be corrected.

[0068] Figure 2 is the specific pattern of the target board in the image edge brightness correction device according to an embodiment of the present application. The specific pattern includes: a plurality of first color blocks and second color blocks with the same size and arranged alternately drawn on a square (including square and rectangle) plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally. Further, the first color blocks and second color blocks in the specific pattern can be set in the area outside the strips in the square plane, and marking points are set in each of the first color blocks and second color blocks. The first color blocks and second color blocks can have different colors to facilitate testing the resolution of the image to be corrected with the above specific pattern.

[0069] As Figure 2 shown, in one embodiment, the specific pattern of the target board is generally square in shape, with 8 strips drawn in a generally "rice" character shape, and alternately arranged first color blocks and second color blocks with different colors, such as white blocks and black blocks, are drawn in the area outside the strips. A cross mark can also be drawn at the center of the target board for positioning to facilitate the shooting of the camera module under test.

[0070] Marking points can be set in at least one of the first color blocks and second color blocks. For example, the region of interest (ROI) for resolution testing in the first color block or the second color block can be selected as the marking point for the color block.

[0071] In step S102, the gray values of the sampling points in the image to be corrected can be obtained by, for example, an image data generator in the image edge brightness correction device, and then the standard brightness gradient curve of the image to be corrected can be determined. In one embodiment of the present application, it can be achieved through the following steps:

[0072] Draw the brightness gradient curve of each strip according to the gray values of the sampling points distributed on each strip in the image to be corrected. For example, the center position of the image to be corrected can be set as the coordinate origin, an XY coordinate system can be established, and the coordinate values of the sampling points on each strip can be obtained. Then, with the gray values of the sampling points as the ordinate and the coordinate values of the sampling points as the abscissa, draw the brightness gradient curve of each strip.

[0073] Obtain the average value of the brightness gradient curves of multiple strips, and the average brightness gradient curve of the image to be corrected can be obtained.

[0074] The 0-0.8 field of view range in the image to be corrected is set as the central field of view, and the 0.8-1 field of view range is set as the edge field of view, wherein the 0.8-1 field of view is the edge area of ​​the image to be corrected in this application.

[0075] The brightness gradient curve in the edge field of view may be fitted using, for example, the least square method based on the average brightness gradient curve in the central field of view.

[0076] The average brightness gradient curve within the central field of view can be combined with the brightness gradient curve within the edge field of view after fitting to generate a standard brightness gradient curve for the image to be corrected.

[0077] Furthermore, in order to ensure that the standard brightness gradient curve is applicable to all edges including the four corners of the image, the uniformity of the four corners of the image to be corrected can be verified. Specifically, the brightness gradient curves of the eight strips within the edge field of view and located at the diagonal position in the image to be corrected are compared with each other, and their deviation values ​​are calculated. The deviation value is used to correct the standard brightness gradient curve generated above.

[0078] In one embodiment of the present application, a brightness threshold can be set, for example, 10-15% of the corresponding value in the standard brightness gradient curve can be set as the brightness threshold of each marked point. The gray value (actual measurement value) of each marked point in the edge of the image is subtracted from its corresponding value in the standard brightness gradient curve. When the difference of the marked point exceeds the brightness threshold, the first color block or the second color block where the marked point is located can be set as the abnormal area of ​​the image to be corrected.

[0079] At the same time, the ratio between the grayscale value of each marked point in the abnormal area and its corresponding value in the standard brightness gradient curve can also be set as the brightness deviation coefficient.

[0080] In step S103, the resolution of each marked point in the image to be corrected can be obtained by, for example, an image data generator in an image edge brightness correction device. Currently, evaluating the resolution of a camera module by SFR (Spatial Frequency Response) is a recognized method in the industry. In one embodiment of the present application, the resolution change curve of the image to be corrected can be obtained by using the SFR value of each marked point in the image to be corrected to draw an SFR value change curve. The specific steps are as follows:

[0081] The resolution of each marked point in the image to be corrected is obtained to draw the image resolution change curve. For example, the SFR value of each marked point in the image to be corrected can be obtained through the image data generator, and the SFR value change curve of the image to be corrected can be drawn with the SFR value of the marked point as the ordinate and the position value of the marked point as the abscissa.

[0082] The resolution variation curve of the image in the edge field of view of the image to be corrected can be fitted according to the resolution variation curve of the image in the central field of view.

[0083] The image resolution change curve within the central field of view is combined with the fitted image resolution change curve within the edge field of view to generate a standard resolution change curve for the image to be corrected.

[0084] The correctable area in the above abnormal area can be determined by the standard resolution change curve of the image to be corrected. Specifically, it can be achieved by the following steps:

[0085] A resolution threshold for determining an abnormal area is set. For example, 70% of the corresponding value of the resolution of the marked point in the standard resolution change curve can be set as the resolution threshold.

[0086] The resolution of each marker point in the above abnormal area is compared with its corresponding value in the standard resolution change curve.

[0087] The abnormal area where the mark point below the set resolution threshold is located is set as the area with severe information loss. For example, it can be set that when the resolution of the mark point is lower than 70% of the corresponding value in the standard resolution change curve, the abnormal area where the mark point is located is the area with severe information loss; the abnormal area where the mark point above the resolution threshold is located is set as the correctable area. For example, it can be set that when the resolution of the mark point is higher than 70% of the corresponding value in the standard resolution change curve, the abnormal area where the mark point is located is the correctable area.

[0088] The brightness deviation coefficient is used to fit the correctable area through the least square method to obtain a brightness-corrected image.

[0089] Furthermore, in one embodiment of the present application, a method for optimizing a brightness-corrected image using a trained convolutional neural network model is provided.

[0090] Figure 3 is a schematic diagram of an image edge brightness correction method 200 according to another embodiment of the present application. Figure 4 It is a flowchart of a method 300 for training a convolutional neural network model according to one embodiment of the present application.

[0091] like Figure 3As shown, the convolutional neural network model for image edge brightness correction may include at least one feature extraction layer 2021 and a reconstruction layer 2022 connected to the feature extraction layer 2021. The feature extraction layer 2021 is used to perform a convolution operation on the input image to obtain a feature map of the input image. In this embodiment, the feature extraction layer 2021 may include a plurality of convolution kernels to obtain a plurality of feature maps based on the input image, and different convolution kernels are used to extract different features in the input image. For example, the feature extraction layer in this embodiment may include 32 convolution kernels, which may obtain 32 feature maps for the input image, and these feature maps may reflect different features of the input image. The reconstruction layer 2022 is used to perform a deconvolution operation on the plurality of feature maps obtained in the previous feature extraction layer to reconstruct a clear image based on these feature maps. In other embodiments, the feature extraction layer may also include other numbers of convolution kernels without departing from the teachings of the present invention.

[0092] like Figure 3 As shown, for example, the first image 201 may correspond to the image after brightness correction described above. First, the first image 201 is input to the convolutional neural network model 202. In the convolutional neural network model 202, the first convolution layer 2021 may perform feature extraction on the input first image 201 to obtain, for example, 32 feature maps. Then, the second convolution layer 2021 may perform feature extraction on the basis of the 32 feature maps to obtain multiple feature maps, for example, 32 feature maps or 64 feature maps or more. It should be noted that the number of convolution kernels included in each feature extraction layer is not limited to 32 or 64, but may also be other numbers. In other words, the number of feature maps obtained by each feature extraction layer is not particularly limited. As described above, each subsequent feature extraction layer 2021 obtains new features based on the feature map output by the previous layer. After the last feature extraction layer 2021 completes the feature extraction, the obtained multiple feature maps are input to the reconstruction layer 2022. The reconstruction layer 2022 performs a deconvolution operation on the input multiple feature maps to reconstruct a first result image 203 that is clearer (the edge brightness is corrected again) than the first image 201 based on the multiple feature maps.

[0093] Next, the first result image 203 is input to the convolutional neural network model 202, and the first feature extraction layer 2021 performs feature extraction on the first result image 203, and obtains, for example, 32 feature maps (a total of 64 feature maps) for the image, and then the 64 feature maps are added to obtain 32 feature maps. In the addition process, for example, the feature maps obtained from two images respectively by the same convolution kernel can be added together. The 32 feature maps obtained by addition are input to the subsequent feature extraction layer, and the subsequent operations are the same as those described above for the processing of the first image 201, that is, the second result image 205 is obtained through the feature extraction operation of the subsequent feature extraction layer 2021 and the reconstruction operation of the reconstruction layer 2022.

[0094] Finally, the second result image 205 and the original image 206 are input into the convolutional neural network model 202 together, and the result image corresponding to the original image 206 can be obtained through the same processing process as the processing process for the first result image 203 described above, that is, after the second result image 205 and the original image 206 are subjected to the feature extraction operation of the multiple feature extraction layers 2021 in the convolutional neural network model 202 and the reconstruction operation of the reconstruction layer 2022, the result image can be obtained, that is, the clear image 207 reconstructed after edge brightness correction. The size of the clear image 207 is the same as that of the original image, but after reconstruction, the edge brightness is higher than the original image without distortion, and the image has better clarity.

[0095] In a method of one embodiment of the present application, the result image of the previous brightness correction is input into the convolutional neural network model in the next brightness correction process, so that the features of each image can be fully utilized, thereby achieving a better effect of the brightness correction process. Figure 4 As shown, such a processing flow can also enable the convolutional neural network model to be fully trained in the training process to be described below.

[0096] During the training process, firstly in step S301 , an abnormal brightness image (test image) and a corresponding standard image for training are selected from a test image set.

[0097] In step S302, the edge area of ​​the image with abnormal brightness may be defined first. For example, the image within the field of view range of 0-0.8 may be defined as the central field of view area, and the image within the field of view range of 0.8-1 may be defined as the edge field of view area (edge ​​area).

[0098] In step S303, the Figure 3 The described image edge brightness correction method is used to optimize the test image.

[0099] In step S304, the result image obtained by the last processing is compared with the standard image to determine the difference between the result image and the standard image. The difference can be measured by a loss function, and the loss function can be determined by a mean square error.

[0100] In step S305, it can be determined whether the above difference is greater than a predetermined value, and the predetermined value can be set according to actual conditions. When the difference is greater than the predetermined value, it means that the result image obtained by the convolutional neural network model has not yet met the requirements and further training is required. In this case, step S306 is executed to adjust the parameters of the convolutional neural network model according to the above difference. After completing the adjustment of the parameters, return to step S301 for the next round of training. The above process is repeated until the difference between the result image and the standard image is less than or equal to the predetermined value. When the difference between the result image and the standard image is equal to or less than the predetermined value, it means that the result image obtained by the convolutional neural network model has met the requirements. In this case, the training can be ended and the model weights after the above processing can be saved.

[0101] The trained convolutional neural network model can accurately reconstruct the edge brightness corrected image.

[0102] The present application also provides a device for edge brightness correction, the device comprising: a light source, a target plate with a specific pattern, a module fixture, an image data generator and a reconstructor. The module fixture is used to fix the camera module to be tested; the center of the target plate with a specific pattern can be set on the optical axis of the camera module, and the shooting range of the camera module is equal to the size of the specific pattern to obtain an image with the specific pattern; the image data generator is used to obtain the gray value, resolution and edge brightness of the image; and the reconstructor includes a trained convolutional neural network model to correct the image edge brightness.

[0103] Furthermore, in one embodiment, the edge brightness correction device further includes: an output device for outputting the image after the image edge brightness is corrected.

[0104] In one embodiment, the above-mentioned convolutional neural network model includes: a feature extraction layer, which is used to extract features from the image to be processed or the result image of the previous edge brightness correction processing to obtain multiple feature maps; and a reconstruction layer, located after the feature extraction layer, which is used to perform deconvolution processing on the extracted multiple feature maps to reconstruct the image to be processed based on the extracted multiple feature maps.

[0105] In one embodiment, the above-mentioned edge brightness correction device also includes a trainer for training a convolutional neural network model, wherein the trainer is configured to: select a brightness abnormality image for training and a standard image corresponding to the brightness abnormality image; define the edge area of ​​the brightness abnormality image; extract feature data of the edge area in the feature extraction layer; optimize the edge area in the reconstruction layer using the mean square error as the loss function; compare the optimized result image with the standard image to determine the difference; when the difference is greater than a predetermined value, adjust the parameters of the feature extraction layer and the reconstruction layer according to the difference; repeat the above steps until the difference is less than the predetermined value; and save the trained model weights.

[0106] In one embodiment, the specific pattern of the above-mentioned target plate may include: a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally, wherein the first color block and the second color block may have different colors and be arranged in the area outside the strips in the square plane, and a marking point may be set in at least one of the first color block and the second color block.

[0107] In one embodiment, the image data generator may be configured to: determine the abnormal area at the edge of the image to be corrected by a standard brightness gradient curve, and calculate the brightness deviation coefficient of the abnormal area; determine the correctable area in the abnormal area by a standard resolution change curve; and perform brightness correction on the correctable area according to the brightness deviation coefficient to obtain a brightness-corrected image.

[0108] Furthermore, the above-mentioned image data generator can be configured to: make a difference between the grayscale value of each marking point in the edge field of view and the corresponding value in the standard brightness gradient curve; and set the first color block or the second color block where the marking point whose difference exceeds the preset brightness threshold is located as an abnormal area.

[0109] The above-mentioned image data generator can be configured to: set the ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

[0110] The image data generator may be configured to: compare the resolution of each mark point in the abnormal area with the corresponding value in the standard resolution change curve; and set the abnormal area where the mark points below the preset resolution threshold are located as the area with severe information loss, and set the abnormal area where the mark points above the preset resolution threshold are located as the correctable area. Performing brightness correction on the correctable area according to the brightness deviation coefficient may include: using the brightness deviation coefficient to obtain a brightness-corrected image through least squares fitting.

[0111] In one embodiment, the above-mentioned image data generator can be configured to: generate a standard brightness gradient curve for the image to be corrected, including: drawing a brightness gradient curve for each strip in the image according to the grayscale values ​​of the sampling points distributed on each strip; calculating the average value of the brightness gradient curves of multiple strips to obtain an average brightness gradient curve of the image; determining the edge field of view and the central field of view in the image; fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and merging the average brightness gradient curve within the central field of view with the fitted brightness gradient curve within the edge field of view to generate a standard brightness gradient curve.

[0112] Furthermore, the image data generator may be configured to: compare the brightness gradient curves of eight strips located at diagonal positions in the edge field of view, calculate a deviation value; and correct the standard brightness gradient curve using the deviation value.

[0113] In one embodiment, the above-mentioned image data generator can be configured to: generate a standard resolution change curve of the image, including: obtaining the resolution of each marked point in the image to draw the resolution change curve of the image; fitting the resolution change curve of the image in the edge field of view according to the resolution change curve of the image in the central field of view; and merging the resolution change curve of the image in the central field of view with the fitted resolution change curve in the edge field of view to generate a standard resolution change curve.

[0114] Figure 5 A schematic diagram of the structure of a computer system 400 suitable for implementing a terminal device or a server of the present application is shown.

[0115] The present application also provides a computer system, which may be, for example, a mobile terminal, a personal computer (PC), a tablet computer, a server, etc. Figure 5 , which shows a schematic diagram of the structure of a computer system 400 suitable for implementing a terminal device or a server of the present application: Figure 5 As shown, the computer system 400 includes one or more processors, a memory, and a communication unit, etc. The one or more processors may be, for example, one or more central processing units (CPUs) 401, and / or one or more graphics processing units (GPUs) 413, etc. The processor may perform various appropriate actions and processes according to executable instructions stored in a read-only memory (ROM) 402 or executable instructions loaded from a storage unit 408 to a random access memory (RAM) 403. The communication unit 412 may include, but is not limited to, a network card.

[0116] The processor can communicate with the read-only memory 402 and / or the random access memory 403 to execute executable instructions, be connected to the communication unit 412 through the bus 404, and communicate with other target devices through the communication unit 412, so as to complete the operation corresponding to any one of the methods provided in the embodiments of the present application, for example: determining the abnormal area at the edge of the image to be corrected according to the standard brightness gradient curve of the image with a specific pattern of the target plate, and calculating the brightness deviation coefficient of the abnormal area; determining the correctable area in the abnormal area based on the standard resolution change curve of the image to be corrected; and performing brightness correction on the correctable area according to the brightness deviation coefficient.

[0117] Furthermore, the processor executes the above instructions to further train a convolutional neural network model for image edge brightness correction; and reconstructs the image after brightness correction through the convolutional neural network model.

[0118] Among them, the processor executing the above instructions to train a convolutional neural network model for image edge brightness correction may include: building a convolutional neural network model with a feature extraction layer and a reconstruction layer; selecting a brightness abnormality image and a standard image corresponding to the brightness abnormality image for training; defining the edge area of ​​the brightness abnormality image; extracting feature data of the edge area in the feature extraction layer; optimizing the edge area in the reconstruction layer; comparing the optimized result image with the standard image to determine the difference; when the difference is greater than a predetermined value, adjusting the parameters of the feature extraction layer and the reconstruction layer according to the difference; repeating the above steps until the difference is less than the predetermined value.

[0119] In one embodiment, the specific pattern of the image to be processed in the system includes: a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally, wherein the first color blocks and the second color blocks are arranged in areas outside the strips in the square plane; the first color blocks and the second color blocks have different colors; and a marking point is set in at least one of the first color blocks and the second color blocks.

[0120] In one embodiment, the processor executes the above instructions to further generate a standard brightness gradient curve for the image, including: drawing a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; calculating the average of multiple strip brightness gradient curves to obtain an average brightness gradient curve for the image; determining the edge field of view and the central field of view in the image; fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and merging the average brightness gradient curve within the central field of view with the fitted brightness gradient curve within the edge field of view to generate a standard brightness gradient curve.

[0121] Furthermore, the processor executes the above instructions to further generate a standard brightness gradient curve of the image, which also includes: comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view with each other to calculate the deviation value; and correcting the standard brightness gradient curve using the deviation value.

[0122] In one embodiment, the processor executes the above instructions to use the least square method to fit the brightness gradient curve within the image edge field of view.

[0123] In one embodiment, the processor executes the above instructions to determine the abnormal area at the edge of the image according to the following steps: setting a brightness threshold; subtracting the grayscale value of each marked point in the image edge field of view from the corresponding value in the standard brightness gradient curve; and setting the first color block or the second color block where the marked point whose difference exceeds the brightness threshold is located as an abnormal area.

[0124] In one embodiment, the processor executes the above instructions to set the ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

[0125] In one embodiment, the processor executes the above instructions to further generate a standard resolution change curve of the image, including: obtaining the resolution of each mark point in the image to draw the resolution change curve of the image, for example, obtaining the SFR value of each mark point in the image to draw the SFR value change curve of the image; fitting the resolution change curve within the edge field of view according to the resolution change curve of the image in the central field of view, for example, using the least squares method to fit the resolution change curve within the edge field of view; and merging the resolution change curve of the image in the central field of view with the fitted resolution change curve within the edge field of view to generate a standard resolution change curve.

[0126] In one embodiment, the processor executes the above instructions to use the least square method to fit the resolution change curve within the image edge field of view.

[0127] In one embodiment, the processor executes the above instructions to obtain the SFR value of each marked point in the image and draw an SFR value change curve of the image.

[0128] In one embodiment, the processor executes the above instructions to determine the correctable area of ​​the abnormal area of ​​the image through the following processing: setting a resolution threshold; comparing the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and setting the abnormal area where the marked points below the resolution threshold are located as the area with severe information loss, and setting the abnormal area where the marked points above the resolution threshold are located as the correctable area.

[0129] In one embodiment, the processor executes the above instructions to obtain a brightness-corrected image by least squares fitting using the brightness deviation coefficient.

[0130] In addition, various programs and data required for the operation of the device may also be stored in RAM 403. CPU 401, ROM 402 and RAM 403 are connected to each other via bus 404. In the case of RAM 403, ROM 402 is an optional module. RAM 403 stores executable instructions, or writes executable instructions into ROM 402 at runtime, and the executable instructions enable the processor 401 to perform operations corresponding to the above-mentioned communication method. Input / output (I / O) interface 405 is also connected to bus 404. Communication unit 412 can be integrated, or it can be configured to have multiple submodules (e.g., multiple IB network cards) and be linked on the bus.

[0131] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0132] If necessary, Figure 5 The architecture shown is only an optional implementation method. In the specific practice, the above can be modified according to actual needs. Figure 5 The number and type of components can be selected, deleted, added or replaced; different functional components can be set up by separate settings or integrated settings, for example, the GPU and CPU can be set up separately or the GPU can be integrated on the CPU, the communication unit can be set up separately or integrated on the CPU or GPU, etc. These alternative implementations all fall within the scope of protection disclosed in this application.

[0133] The above description is only an implementation method of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of protection involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for correcting image edge brightness, characterized in that: The method comprises: Photographing a target plate having a specific pattern to obtain an image of the specific pattern; Determine the abnormal area at the edge of the image by using the standard brightness gradient curve of the image, and calculate the brightness deviation coefficient of the abnormal area; Determining a correctable area in the abnormal area based on a standard resolution change curve of the image; and Performing brightness correction on the correctable area according to the brightness deviation coefficient, The standard brightness gradient curve is a combined curve of an average brightness gradient curve within the central field of view of the image and a brightness gradient curve within the edge field of view of the image, wherein the average brightness gradient curve within the central field of view of the image is an average value of brightness gradient curves of multiple strips in the image; and The standard resolution change curve is a combined curve of the resolution change curve of the image within the central field of view of the image and the resolution change curve within the edge field of view of the image, wherein the resolution change curve of the image is obtained by calculating the resolution of each marked point in the image.

2. The method according to claim 1, characterized in that The method further comprises: Training a convolutional neural network model for image edge brightness correction; and The image after brightness correction is reconstructed through the convolutional neural network model.

3. The method according to claim 2, characterized in that The convolutional neural network model trained for image edge brightness correction includes: Build a convolutional neural network model with feature extraction layer and reconstruction layer; Selecting an abnormal brightness image for training and a standard image corresponding to the abnormal brightness image; Defining an edge region of the image with abnormal brightness; Extracting feature data of the edge area at the feature extraction layer; Optimizing the edge region in the reconstruction layer; Comparing the optimized result image with the standard image to determine the difference; When the difference is greater than a predetermined value, adjusting parameters of the feature extraction layer and the reconstruction layer according to the difference; and Repeat the above steps until the difference is smaller than the predetermined value.

4. The method according to claim 1, characterized in that Photographing a target plate having a specific pattern to obtain an image of the specific pattern includes: Arranging a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane, and drawing a plurality of strips extending from the center of the square plane to four sides and four corners and equally dividing the square plane to form the specific pattern; and The center of the target plate is set on the optical axis of the camera module, and the shooting range of the camera module is made equal to the size of the specific pattern to obtain the image of the specific pattern.

5. The method according to claim 4, characterized in that The method of arranging a plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane further comprises: Disposing the first color block and the second color block in an area outside the strip in the square plane; Setting the first color block and the second color block to different colors; and A marking point is set on at least one of the first color block and the second color block.

6. The method according to claim 4, characterized in that The method further comprises: Generating a standard brightness gradient curve of the image, comprising: Draw a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; Calculating an average value of the brightness gradient curves of the plurality of strips to obtain an average brightness gradient curve of the image; determining a peripheral field of view and a central field of view in the image; Fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and The average brightness gradient curve within the central field of view is combined with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

7. The method according to claim 6, characterized in that Generating the standard brightness gradient curve also includes: Comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view, and calculating the deviation value; and The standard brightness gradient curve is corrected using the deviation value.

8. The method according to claim 6, characterized in that Fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view also includes: The least square method is used to fit the brightness gradient curve within the edge field of view.

9. The method according to claim 6, characterized in that Determining the abnormal area at the edge of the image by using the standard brightness gradient curve of the image includes: Set brightness threshold; Difference between the grayscale value of each marked point in the edge field of view and the corresponding value in the standard brightness gradient curve; and The first color block or the second color block where the marking point whose difference exceeds the brightness threshold is located is set as the abnormal area.

10. The method according to claim 9, characterized in that Calculating the brightness deviation coefficient of the abnormal area includes: The ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve is set as the brightness deviation coefficient.

11. The method according to claim 4, characterized in that The method further comprises: Generating a standard resolution change curve of the image, comprising: Obtaining the resolution of each marked point in the image and drawing a resolution change curve of the image; Fitting the resolution variation curve of the image within the edge field of view according to the resolution variation curve of the image within the central field of view; and The resolution change curve of the image within the central field of view is combined with the fitted resolution change curve within the edge field of view to generate the standard resolution change curve.

12. The method according to claim 11, characterized in that Fitting the resolution change curve within the edge field of view according to the resolution change curve of the image within the central field of view also includes: The least square method is used to fit the resolution change curve within the edge field of view.

13. The method according to claim 11, characterized in that Obtaining the resolution of each marked point in the image and drawing a resolution change curve of the image also includes: The SFR value of each marked point in the image is obtained to draw an SFR value change curve of the image.

14. The method according to claim 11, characterized in that Determining the correctable area in the abnormal area based on the standard resolution change curve of the image includes: Set the resolution threshold; Comparing the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and The abnormal area where the marking points below the resolution threshold are located is set as an area with severe information loss, and the abnormal area where the marking points above the resolution threshold are located is set as a correctable area.

15. The method according to claim 1, characterized in that Performing brightness correction on the correctable area according to the brightness deviation coefficient further includes: The brightness deviation coefficient is used to obtain a brightness-corrected image through least square fitting.

16. An image edge brightness correction system, characterized in that: The system comprises: a memory storing computer-readable instructions; and A processor, connected to the memory, executes the instructions to perform the following operations: Determine the abnormal area at the edge of the image according to the standard brightness gradient curve of the image with the specific pattern of the target plate, and calculate the brightness deviation coefficient of the abnormal area; Determining a correctable area in the abnormal area based on a standard resolution change curve of the image; and Performing brightness correction on the correctable area according to the brightness deviation coefficient, The standard brightness gradient curve is a combined curve of an average brightness gradient curve within the central field of view of the image and a brightness gradient curve within the edge field of view of the image, wherein the average brightness gradient curve within the central field of view of the image is an average value of brightness gradient curves of multiple strips in the image; and The standard resolution change curve is a combined curve of the resolution change curve of the image within the central field of view of the image and the resolution change curve within the edge field of view of the image, wherein the resolution change curve of the image is obtained by calculating the resolution of each marked point in the image.

17. The system according to claim 16, characterized in that The processor executes the instructions to further: Training a convolutional neural network model for image edge brightness correction; and The image after brightness correction is reconstructed through the convolutional neural network model.

18. The system according to claim 17, characterized in that The processor executes the instructions to train a convolutional neural network model for image edge brightness correction, including: Build a convolutional neural network model with feature extraction layer and reconstruction layer; Selecting an abnormal brightness image for training and a standard image corresponding to the abnormal brightness image; Defining an edge region of the image with abnormal brightness; Extracting feature data of the edge area at the feature extraction layer; Optimizing the edge region in the reconstruction layer; Comparing the optimized result image with the standard image to determine the difference; When the difference is greater than a predetermined value, adjusting parameters of the feature extraction layer and the reconstruction layer according to the difference; and Repeat the above steps until the difference is smaller than the predetermined value.

19. The system according to claim 16, characterized in that The specific patterns include: A plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally; The first color block and the second color block are arranged in the area outside the strip in the square plane; the first color block and the second color block have different colors; and at least one of the first color block and the second color block is provided with a marking point.

20. The system according to claim 16, characterized in that The processor executes the instructions to further: Generating a standard brightness gradient curve of the image, comprising: Draw a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; Calculating the average value of the plurality of strip brightness gradient curves to obtain the average brightness gradient curve of the image; determining a peripheral field of view and a central field of view in the image; Fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and The average brightness gradient curve within the central field of view is combined with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

21. The system according to claim 20, characterized in that The processor executes the instructions to further: Generating the standard brightness gradient curve includes: Comparing the brightness gradient curves of the eight strips located at the diagonal position in the edge field of view with each other to calculate the deviation value; and The standard brightness gradient curve is corrected using the deviation value.

22. The system according to claim 20, characterized in that The processor executes the instruction to fit the brightness gradient curve within the edge field of view using a least squares method.

23. The system according to claim 16, characterized in that The processor executes the instructions to determine the abnormal area of ​​the edge of the image according to the following steps: Set brightness threshold; Difference between the grayscale value of each marked point in the edge field of view and the corresponding value in the standard brightness gradient curve; and The first color block or the second color block where the marking point whose difference exceeds the brightness threshold is located is set as the abnormal area.

24. The system according to claim 16, characterized in that The processor executes the instruction to set the ratio between the grayscale value of each marking point in the abnormal area and the corresponding value in the standard brightness gradient curve as the brightness deviation coefficient.

25. The system according to claim 16, characterized in that The processor executes the instructions to further: Generating a standard resolution change curve of the image, comprising: Obtaining the resolution of each mark point in the image and drawing a resolution change curve of the image; Fitting the resolution variation curve of the image within the edge field of view according to the resolution variation curve of the image within the central field of view; and The resolution change curve of the image within the central field of view is combined with the fitted resolution change curve within the edge field of view to generate the standard resolution change curve.

26. The system according to claim 25, characterized in that The processor executes the instructions to fit the resolution variation curve within the edge field of view using the least square method.

27. The system according to claim 25, characterized in that The processor executes the instruction to obtain the SFR value of each marked point in the image and draw an SFR value change curve of the image.

28. The system according to claim 16, characterized in that The processor executes the instructions to determine a correctable area in the abnormal area by: Set the resolution threshold; Comparing the resolution of each marked point in the abnormal area with the corresponding value in the standard resolution change curve; and The abnormal area where the marking points below the resolution threshold are located is set as an area with severe information loss, and the abnormal area where the marking points above the resolution threshold are located is set as a correctable area.

29. The system according to claim 16, characterized in that The processor executes the instruction to obtain a brightness-corrected image by using the brightness deviation coefficient through least squares fitting.

30. A device for image edge brightness correction method, characterized in that: The device comprises: Module fixture, used to fix the camera module to be tested; A target plate with a specific pattern, the center of which is set on the optical axis of the camera module, and the shooting range of the camera module is equal to the size of the specific pattern to obtain an image with the specific pattern; an image data generator, configured to determine an abnormal region at the edge of the image by using a standard brightness gradient curve of the image, calculate a brightness deviation coefficient of the abnormal region, determine a correctable region in the abnormal region based on a standard resolution change curve of the image, and perform brightness correction on the correctable region according to the brightness deviation coefficient; and The reconstructor is provided with a convolutional neural network model to correct the image edge brightness. The standard brightness gradient curve is a combined curve of an average brightness gradient curve within the central field of view of the image and a brightness gradient curve within the edge field of view of the image, wherein the average brightness gradient curve within the central field of view of the image is an average value of brightness gradient curves of multiple strips in the image; and The standard resolution change curve is a combined curve of the resolution change curve of the image within the central field of view of the image and the resolution change curve within the edge field of view of the image, wherein the resolution change curve of the image is obtained by calculating the resolution of each marked point in the image.

31. The device according to claim 30, characterized in that The convolutional neural network model includes: A feature extraction layer, used for extracting features from the image to be processed or the image resulting from the previous edge brightness correction process to obtain a plurality of feature maps; and The reconstruction layer is located after the feature extraction layer and is used to perform deconvolution processing on the extracted multiple feature maps to reconstruct the image to be processed according to the extracted multiple feature maps.

32. The device according to claim 31, characterized in that Also included is a trainer for training the convolutional neural network model, wherein the trainer is configured as follows: Selecting an abnormal brightness image for training and a standard image corresponding to the abnormal brightness image; Defining an edge region of the image with abnormal brightness; Extracting feature data of the edge area at the feature extraction layer; Optimizing the edge region in the reconstruction layer; Comparing the optimized result image with the standard image to determine the difference; When the difference is greater than a predetermined value, adjusting parameters of the feature extraction layer and the reconstruction layer according to the difference; and Repeat the above steps until the difference is smaller than the predetermined value.

33. The device according to claim 30, characterized in that The specific patterns include: A plurality of first color blocks and second color blocks of the same size and arranged alternately on a square plane; and a plurality of strips extending from the center of the square plane to the four sides and four corners and dividing the square plane equally; The first color block and the second color block are arranged in the area outside the strip in the square plane; the first color block and the second color block have different colors; and a marking point is arranged on at least one of the first color block and the second color block.

34. The device according to claim 30, characterized in that The image data generator is configured as follows: The grayscale value of each marking point in the edge field of view is subtracted from the corresponding value in the standard brightness gradient curve, and the first color block or the second color block where the marking point whose difference exceeds a preset brightness threshold is located is set as the abnormal area.

35. The device according to claim 30, characterized in that The image data generator is configured as follows: The ratio between the grayscale value of each marked point in the abnormal area and the corresponding value in the standard brightness gradient curve is set as the brightness deviation coefficient.

36. The device according to claim 30, characterized in that The image data generator is configured as follows: The resolution of each marked point in the abnormal area is compared with the corresponding value in the standard resolution change curve; and the abnormal area where the marked points below the preset resolution threshold are located is set as a severe information loss area, and the abnormal area where the marked points above the resolution threshold are located is set as a correctable area.

37. The device according to claim 30, characterized in that The image data generator is configured as follows: The brightness deviation coefficient is used to obtain a brightness-corrected image through least square fitting.

38. The device according to claim 30, characterized in that The image data generator is further configured to generate a standard brightness gradient curve of the image, including: Draw a brightness gradient curve for each strip according to the grayscale values ​​of the sampling points distributed on each strip in the image; Calculating an average value of the brightness gradient curves of the plurality of strips to obtain an average brightness gradient curve of the image; determining a peripheral field of view and a central field of view in the image; Fitting the brightness gradient curve within the edge field of view according to the average brightness gradient curve within the central field of view; and The average brightness gradient curve within the central field of view is combined with the fitted brightness gradient curve within the edge field of view to generate the standard brightness gradient curve.

39. The device according to claim 38, characterized in that The image data generator is further configured to generate a standard brightness gradient curve of the image, including: Comparing the brightness gradient curves of eight strips located at diagonal positions in the edge field of view, and calculating the deviation value; and The standard brightness gradient curve is corrected using the deviation value.

40. The device according to claim 30, characterized in that The image data generator is further configured to generate a standard resolution change curve of the image, including: Obtaining the resolution of each marked point in the image and drawing a resolution change curve of the image; Fitting the resolution variation curve of the image within the edge field of view according to the resolution variation curve of the image within the central field of view; and The resolution change curve of the image within the central field of view is combined with the fitted resolution change curve within the edge field of view to generate the standard resolution change curve.

Images