Stray light quantification method based on high dynamic image synthesis
By splicing and grayscale conversion of the light source images captured by the camera module, segmenting and quantizing stray light areas, the subjectivity and unquantification of stray light detection in the prior art are solved, and the detection speed and imaging quality are improved.
Patent Information
- Application Number
- CN202010660844.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-07-10
AI Technical Summary
The existing stray light detection methods mainly rely on human eye judgments, have problems of subjectivity, misjudgment and leakage, and have failed to quantify stray light, affecting the imaging quality of the camera module.
By obtaining the light source image taken by the camera module to be detected, stitching and superimposing it and converting it into a grayscale image, marking and segmenting the stray light area, calculating the area, size and energy value of each block, and obtaining the quantized value of stray light.
The speed and accuracy of stray light detection are improved, and the subjectivity of human eye observation is avoided. The quantized value of stray light helps to eliminate stray light more accurately and improve the imaging quality of the camera module.
Smart Images

Figure CN113920009B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of optical devices, and more specifically, to a stray light quantification method. Background Art
[0002] With the development of the mobile phone industry, people have higher and higher requirements for the shooting functions of mobile phones, and the performance of mobile phone camera modules directly affects the shooting effect of the entire mobile phone. After the mobile phone camera module is assembled, due to light leakage, residual reflections of the transmitted optical surface, residual reflections of non-optical surfaces such as the inner wall of the lens barrel, and quality problems of the optical surface, the camera module will generate some stray light. These stray lights will increase the noise on the imaging surface, especially the stray light convergence points near the imaging surface will have a serious impact on the imaging of the camera module. These convergence points are also called "ghost images". Stray light and ghost images are a common anomaly of the camera module, which will affect the imaging quality of the camera module.
[0003] Existing stray light detection methods mainly include the following two methods: one detection method is to detect the lens before the camera module is assembled. The specific implementation method is to fix the lens on a specific bracket, use a light source to illuminate the lens 360°, and observe whether the lens generates stray light with the human eye; the other detection method is to detect whether the lens generates stray light after the camera module is assembled. The specific implementation method is to obtain an image through the video acquisition board of the camera module under a specific light source, and observe the obtained image with the human eye to determine whether the lens generates stray light. However, the above two methods both use the human eye to determine the situation of stray light, and the determination results are relatively subjective, and there will be problems of misjudgment, omissions and uncertainty in detection standards.
[0004] The existing detection method is mainly used to detect whether there is stray light in the camera module, and does not quantify the stray light. Quantifying the stray light detection results can improve the detection speed. The quantified value of the stray light is more helpful to accurately eliminate the stray light and improve the imaging quality of the camera module. Summary of the invention
[0005] The present application provides a method that can at least solve or partially solve at least one of the above-mentioned shortcomings in the prior art.
[0006] On the one hand, the present application provides a stray light quantification method, including: acquiring a light source image taken by a camera module to be detected; splicing and superimposing the light source images, and converting the spliced and superimposed light source images into a grayscale image; marking the stray light areas in the grayscale image, dividing each of the stray light areas into a plurality of connected stray light blocks; and quantizing the stray light blocks formed by the division to obtain a quantized value of the stray light.
[0007] According to an embodiment of the present application, quantizing the stray light blocks formed after segmentation to obtain the quantized value of the stray light includes: calculating the number of pixels included in each of the stray light blocks to obtain the area value of each of the stray light blocks; calculating the size of the circumscribed minimum rectangle of each of the stray light blocks to obtain the size of each of the stray light blocks; marking the position coordinates of each of the stray light blocks as the coordinates of the corresponding circumscribed minimum rectangle; and obtaining the energy value of each of the stray light blocks by an integration method.
[0008] According to an embodiment of the present application, marking the stray light area in the grayscale image and dividing each of the stray light areas to form a plurality of connected stray light blocks includes: acquiring a plurality of stray light areas with complete and continuous boundaries in the grayscale image; marking the stray light areas according to the pixel values in the stray light areas; and dividing each of the stray light areas to form a plurality of connected stray light blocks.
[0009] According to an implementation manner of the present application, obtaining multiple stray light areas with complete and continuous boundaries in the grayscale image includes: squaring the pixel values in the grayscale image to obtain a square grayscale image; using the edge detection operator Canny to calculate the square grayscale image to obtain boundary information of the stray light area; and marking the boundary information and performing connectivity processing to obtain multiple stray light areas with complete and continuous boundaries.
[0010] According to the implementation mode of the present application, obtaining the light source image taken by the camera module to be tested includes: setting a point light source and the camera module to be tested in a dark environment; clamping the camera module to be tested with a module clamp with an adjustable clamping angle; and adjusting the ISO value of the sensitivity of the photosensitive element of the camera module to be tested to obtain the light source image taken by the camera module to be tested under different ISO values.
[0011] According to an implementation manner of the present application, adjusting the sensitivity ISO value of the photosensitive element of the camera module to be tested to obtain the light source image captured by the camera module to be tested under the state of different ISO values includes: adjusting the camera module to be tested to realize that the image of the point light source captured by the camera module to be tested is a real image, and recording the ISO value E1 when the light source image is a real image and obtaining the light source image P1; and iteratively adjusting the ISO value to respectively obtain the light source image captured by the camera module to be tested under each ISO value state.
[0012] According to an embodiment of the present application, iteratively adjusting the ISO value to obtain light source images captured by the camera module to be tested under various ISO value states includes:
[0013] Adjust the ISO value En to 5×En-1, and obtain the light source image Pn taken by the camera module to be tested when the ISO value is En, wherein n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image taken by the camera module to be tested when the ISO value is En.
[0014] According to an embodiment of the present application, the light source images P1 to Pn include all types of stray light.
[0015] According to an implementation manner of the present application, the light source images are spliced and superimposed, and converting the spliced and superimposed light source images into grayscale images includes: performing bad pixel detection and denoising on the light source images P1 to Pn; selecting valid pixels for the light source images P1 to Pn that have completed the bad pixel detection and denoising, where n is an integer and n≥2; normalizing the light source images P1 to Pn that have completed the valid pixel selection according to their respective normalization coefficients; splicing and superimposing the normalized light source images P1 to Pn into a composite image P; and converting the composite image P into a grayscale image.
[0016] According to an embodiment of the present application, selecting effective pixels for the light source images P1 to Pn that have completed bad pixel detection and denoising processing includes: selecting pixels that meet the following conditions in the light source images P1 to Pn that have completed bad pixel detection and denoising processing, LSB≥128, wherein LSB is the least significant bit of the pixel value in the raw data RAW10 in the light source images P1 to Pn.
[0017] According to an embodiment of the present application, the normalization coefficient includes: a normalization coefficient X1=Enmax / E1 of the light source image P1; and a normalization coefficient Xn=Enmax / En of the light source image Pn, wherein Enmax is the ISO value adjusted for the last time; E1 is the ISO value when the light source image is a real image; and En is the ISO value adjusted for the nth time, and n is an integer and n≥2.
[0018] According to an embodiment of the present application, superimposing and splicing the light source images P1 to Pn that have completed normalization processing into a composite image P includes: superimposing the pixel values of repeated pixels in the light source images P1 to Pn that have completed normalization processing and calculating the average value; and splicing the average value into a composite image P.
[0019] According to an embodiment of the present application, converting the composite image P into a grayscale image includes: performing RGB interpolation on the composite image P according to the corresponding Bayer array; converting the interpolated RGB image into the grayscale image; and setting the pixel points in the grayscale image whose pixel values are greater than or equal to 1023 to 0.
[0020] According to an embodiment of the present application, segmenting each of the stray light regions to form a plurality of connected stray light blocks includes: segmenting each of the stray light regions to form the plurality of connected stray light blocks using a morphological watershed algorithm.
[0021] On the other hand, the present application also provides a stray light quantization system, including: a superposition device, which obtains the light source image taken by the camera module to be detected, splices and superimposes the light source images, and converts the spliced and superimposed light source images into a grayscale image; a segmentation device, which marks the stray light area in the grayscale image, and segments each of the stray light areas to form a plurality of connected stray light blocks; and a quantization device, which quantifies the stray light blocks formed by segmentation.
[0022] According to an embodiment of the present application, the quantization device includes: an area calculation module, which is used to calculate the number of pixels included in each of the stray light blocks to obtain the area value of each of the stray light blocks; a size calculation module, which is used to calculate the size of the circumscribed minimum rectangle of each of the stray light blocks to obtain the size of each of the stray light blocks; a marking module, which marks the position coordinates of each of the stray light blocks with the coordinates of the corresponding circumscribed minimum rectangle; and an energy value calculation module, which obtains the energy value of each of the stray light blocks by an integration method.
[0023] According to an embodiment of the present application, the superposition device includes: a superposition module, which is used to normalize the light source image and splice and superimpose the light source images that have completed the normalization processing; and a conversion module, which is used to convert the light source images that have completed the splicing and superposition into a grayscale image.
[0024] According to an embodiment of the present application, the segmentation device includes: a boundary extraction module, which is used to obtain a plurality of stray light regions with complete and continuous boundaries in the grayscale image, and mark the stray light regions according to the pixel values in the stray light regions; and a segmentation module, which uses a morphological watershed algorithm to segment each of the stray light regions to form a plurality of connected stray light blocks.
[0025] According to an embodiment of the present application, the method for obtaining multiple stray light areas with complete and continuous boundaries in the grayscale image includes: squaring the pixel values in the grayscale image to obtain a square grayscale image; using the edge detection operator Canny to calculate the square grayscale image to obtain boundary information of the stray light area; and marking the boundary information and performing connectivity processing to obtain multiple stray light areas with complete and continuous boundaries.
[0026] According to an embodiment of the present application, the format of the light source image is at least one of BMP, YUV and RAW.
[0027] According to an embodiment of the present application, the light source image is a light source image captured by the camera module to be tested under various ISO value states and obtained by iteratively adjusting the ISO value of the photosensitive element.
[0028] According to an embodiment of the present application, the light source image is obtained by iteratively adjusting the ISO value of the sensitivity of the photosensitive element. The light source image photographed by the camera module to be tested in each ISO value state includes: the light source image Pn photographed by the camera module to be tested in the state of the ISO value En obtained by adjusting the ISO value En to 5×En-1, wherein n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image photographed by the camera module to be tested in the state of the ISO value En.
[0029] According to an embodiment of the present application, the light source image includes all types of stray light.
[0030] According to the implementation manner of the present application, the superposition device also includes: a bad pixel detection module, which uses neighborhood traversal to screen bad pixels in multiple channels; and a denoising module, which uses neighborhood mean filter to perform denoising in multiple channels.
[0031] On the other hand, the present application also provides a method for using the above-mentioned stray light quantization system, the method comprising: inputting the information of the acquired light source image and the sensitivity value of the photosensitive element of the camera module to be tested into the stray light quantization system; using a neighborhood traversal to screen the bad pixels of the light source image in multiple channels; using a neighborhood mean filter to denoise the light source image after the bad pixel screening in multiple channels; normalizing the light source image after the bad pixel screening and denoising, and splicing and superimposing the normalized light source image to convert it into a grayscale image; obtaining the boundary information of the stray light, marking the boundary information and connecting it to obtain multiple stray light images with complete and continuous boundaries. A light area is defined, and the stray light area is marked according to the pixel value in the stray light area; each stray light area is segmented by a morphological watershed algorithm to form a plurality of connected stray light blocks; the number of pixels included in each stray light block is calculated to obtain the area value of each stray light block; the size of the circumscribed minimum rectangle of each stray light block is calculated to obtain the size of each stray light block; the position coordinates of each stray light block are marked as the coordinates of the corresponding circumscribed minimum rectangle; the energy value of each stray light block is obtained by an integration method; and the serial number, coordinates, size, shape, area value and energy value of the stray light block of the light source image are output.
[0032] According to at least one solution of the stray light quantization method provided above in the present application, at least one of the following beneficial effects can be achieved:
[0033] 1. The stray light quantification method based on high dynamic image synthesis provided in the present application can obtain quantitative values such as the stray light block number, position, size, shape, area, energy value, etc., which is conducive to improving the speed of stray light detection and the accuracy of the detection results, while avoiding the subjectivity of human eye observation in the prior art. The quantified value of stray light helps to more accurately eliminate the stray light of the camera module and improve the imaging quality of the camera module.
[0034] 2. The stray light quantification method provided in this application can be more intuitively and accurately benchmarked with the objective indicators of the camera module, thereby improving the efficiency and accuracy of mutual benchmarking and avoiding the waste of resources in the repeated benchmarking process. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0036] Figure 1 It is a flow chart of a stray light quantization method based on high dynamic image synthesis according to one embodiment of the present application.
[0037] Figure 2 is a normalization process diagram according to an embodiment of the present application; and
[0038] Figure 3 The present invention is a flowchart of a method for using a stray light quantization system based on high dynamic image synthesis according to an embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to better understand the present application, a more detailed description will be made of various aspects of the present application with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application, and are not intended to limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.
[0040] It should be noted that in this specification, the expressions first, second, third, etc. are only used to distinguish one feature from another feature, and do not represent any limitation on the feature. Therefore, without departing from the teaching of this application, the first value discussed below may also be referred to as the second value. And vice versa.
[0041] In the drawings, the thickness, size and shape of the components have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not drawn strictly to scale. As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for the inherent deviations in measurements or calculations that would be recognized by one of ordinary skill in the art.
[0042] It should also be understood that expressions such as "include", "including", "have", "contain" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.
[0043] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.
[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. In addition, unless explicitly limited or contradictory to the context, the specific steps included in the method described in the present application are not necessarily limited to the order described, but can be performed in any order or in parallel. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
[0045] Figure 1 FIG. 1 is a flow chart of a stray light quantification method based on high dynamic image synthesis according to an embodiment of the present application. Figure 1 As shown, the stray light quantification method of the present application is a high-efficiency and high-accuracy stray light quantification method based on high-dynamic image synthesis. It mainly includes:
[0046] S1, obtaining a light source image captured by a camera module to be detected.
[0047] In step S1, the point light source and the camera module to be tested are placed in a completely dark environment, and the camera module to be tested is clamped with a module fixture with an adjustable clamping angle to shoot the point light source. The clamping angle of the module fixture can be adjusted to ensure that the distance, angle, brightness and other conditions between the point light source and the camera module are stable and reproducible. The point light source is photographed in a completely dark environment, and the brightness level of different light source images can be adjusted by setting different ISO values of the photosensitive element of the camera module to be tested.
[0048] In one embodiment, when the imaging image of the point light source captured by the camera module to be tested appears as a real image, the ISO value of the camera module in this state is recorded, and the ISO value of this state is defined as the first ISO value E1, and the captured light source image P1 is obtained at the same time.
[0049] The ISO value of the camera module to be tested can be adjusted to 5 times of E1, the second ISO value of this state is recorded as E2, and the light source image P2 captured in this state is obtained.
[0050] Continue to adjust the ISO value of the camera module to be tested to 5 times of E2, record the third ISO value of this state as E3, and obtain the light source image P3 taken in this state.
[0051] By analogy, adjust the ISO value of the camera module to be tested to 5 times of En-1, record the ISO value of this state as En, and obtain the light source image Pn taken in this state, where n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image taken by the camera module to be tested when the ISO value is En.
[0052] The above process is continued until all types of stray light are fully reflected in the light source image captured by the camera module to be tested. That is to say, after determining the first ISO value of the camera module to be tested, the camera module to be tested captures a light source image at every 5-times ISO value setting, and the light source images captured under each ISO value state are obtained by iteratively adjusting the ISO value until all types of stray light are fully reflected in the picture, and the captured light source images are recorded as P1, P2, P3, P4, etc.
[0053] Specifically, use the camera module to be tested to shoot a point light source. When the image of the point light source is a real image, record the ISO value of the camera module to be tested in this state, and define the ISO value of this state as the first ISO value E1, and obtain the captured light source image P1 at the same time; adjust the ISO value of the camera module to be tested to 5 times the original E1, and use the second ISO value of this state as E2, and obtain the image P2 captured in this state; adjust the ISO value of the camera module to 5 times E2, and use the ISO value of this state as E3, and obtain the image P3 captured in this state..., and so on, until the ISO value of the camera module is adjusted to 5 times E7, and use the ISO value of this state as E8, and obtain the light source image P8 in this state. In other words, after determining the first ISO value of the camera module to be tested, shoot an image of the point light source at every 5-fold ISO setting, until all types of stray light are fully reflected in the captured image, and record each light source image as P1, P2, P3, P4...P8 respectively.
[0054] S2, stitching and superimposing the obtained light source images, and converting the stitched and superimposed light source images into grayscale images.
[0055] In step S2, since the captured light source image is often interfered by various noises during the generation, transmission and recording process, which in turn interferes with the calculation result of stray light, it is first necessary to perform bad pixel detection and denoising on the captured light source image. For example, a 13×13 neighborhood traversal screening method can be used for bad pixel detection, or a 5×5 neighborhood mean filter can be used for denoising.
[0056] The method of using 13×13 neighborhood traversal to screen bad pixel detection mainly includes: taking any pixel point in the light source image as an example, taking this pixel point as the detection point, setting a matrix with the detection point as the center and a size of 13×13, that is, the 13×13 neighborhood of the detection point, which can be recorded as N. Calculate the average value of the pixel values of all pixels except the detection point in this neighborhood N, which can be recorded as A; compare the pixel value of the detection point with the average value A of the pixel value obtained to obtain the difference value, regard the detection point with a larger difference value as an abnormal pixel point, and the abnormal pixel point can be replaced by the average value A' of the pixel values of all pixels except the abnormal pixel point in the 3×3 neighborhood centered on the abnormal pixel point; traverse all pixels in the entire light source image, repeat the above calculation steps, and screen out all abnormal pixels. In particular, for pixels that are less than 7 pixels (distance units) away from the edge of the light source image, a 13×13 matrix containing the pixel and with the boundary of the light source image as the neighborhood boundary can be selected as the detection point neighborhood N' to replace the neighborhood N described above for subsequent calculation steps.
[0057] The method of using a 5×5 neighborhood mean filter for denoising mainly includes: setting a 5×5 matrix with the target pixel as the center, taking the average of the pixel values of all pixels in the matrix, and replacing the pixel value of the target pixel with the average value, which is the mean filter, and the mean filter can be denoted as M; the mean filter M can be used to perform a convolution operation on the light source image that completes the above-mentioned bad pixel detection, and the result obtained is the light source image after filtering and removing noise. In particular, when performing convolution processing on a pixel point that is less than 3 pixels (distance unit) away from the edge of the light source image, the pixel value of the central pixel point in the adjacent matrix can be used to replace the pixel point for convolution calculation.
[0058] Specifically, in this embodiment, a group of light source images P1, P2, P3, P4 ... P8 obtained in step S1 are subjected to bad pixel detection and denoising to obtain a group of light source images P'1, P'2, P'3, P'4 ... P'8.
[0059] A group of light source images P'1, P'2, P'3, P'4 ... P'8 also needs to select effective pixels. Since a group of light source images P1, P2, P3, P4 ... P8 are taken in a completely dark environment, and the completely dark base of the original data RAW10 is 64LSB (Least Significant Bit, the least significant bit of the pixel value), the pixels that meet LSB≥128 in the group of light source images P'1, P'2, P'3, P'4 ... P'8 can be selected to form an effective light source image.
[0060] Figure 2It is a process diagram of normalization processing according to one embodiment of the present application.
[0061] like Figure 2 As shown, the acquired effective light source images are normalized and then spliced together to obtain an RGB image. The converted RGB image can also be further converted into a grayscale image for calculating a unified stray light component to avoid the different response values of different channels to the light signal affecting the test results.
[0062] Specifically, in one embodiment, a group of effective light source images P'1, P'2, P'3, P'4 ... P'8 can be normalized according to their respective normalization coefficients, the normalization coefficient of the light source image P1 is recorded as X1, which can be obtained by X1 = E8 / E1, and the normalization coefficient of the light source image Pn is recorded as Xn, which can be obtained by Xn = E8 / En. Wherein, E1 is the ISO value when the light source image is a real image, E8 is the ISO value adjusted for the 8th time, and En is the ISO value adjusted for the nth time, where n is an integer and 2≤n≤7.
[0063] Each effective light source image is calculated in combination with its own normalization coefficient, and the repeated pixels in the eight images are accumulated to obtain the average value, and finally a composite image is obtained by splicing and superposition, which can be recorded as P. If there are pixels without valid pixel values in a group of effective light source images P'1, P'2, P'3, P'4...P'8, they can be filled with the corresponding pixels in P'1 to P'8 that meet the least significant bit LSB of the pixel value ≥ 64.
[0064] The image P is interpolated with RGB according to the corresponding Bayer array, and then the interpolated RGB image is converted into a grayscale image. Furthermore, the pixel points in the grayscale image whose pixel values are greater than or equal to 1023 (that is, the pixel values of the pixel points are greater than or equal to 1023 in the original data Raw10) can be set to 0 to exclude overexposed stray light signals and light source signals, and finally a grayscale image is obtained, which can be recorded as Pgray.
[0065] S3, marking the stray light regions in the grayscale image, and segmenting each stray light region to form a plurality of connected stray light blocks.
[0066] In step S3, the pixel value of each pixel in the grayscale image Pgray is squared to obtain a square grayscale image, which can be recorded as P 2 gray, for image P 2Gray uses the edge detection operator Canny to calculate and obtain boundary information. Connect the boundary information to obtain multiple stray light areas with complete and continuous boundary contours, which are recorded as B1, B2, B3... On this basis, different stray light areas B1, B2, B3... are marked according to the pixel value levels in the stray light areas and combined with the marked boundaries. For example, the morphological watershed algorithm can be used to segment each stray light area B1, B2, B3..., and the grayscale image Pgray is converted into a gradient image. In the gradient image, the local minimum values of multiple pixel values are used as the "centroid" to form multiple connected stray light blocks with the surrounding neighborhoods. The stray light blocks can be recorded as F1, F2, F3...
[0067] Among them, the technical personnel in this field should understand that the basic idea of the morphological watershed algorithm is to convert the grayscale image into a gradient image, regard the gradient value as an undulating mountain, and regard the local minimum and its neighborhood as a "centroid" similar to a water collection basin (the center of the divided block) to divide the image into several connected blocks to obtain a better image segmentation effect. In this application, the method for segmenting each stray light area in the light source image is not limited. Under the guidance of the disclosure of this application, those skilled in the art can adopt other suitable area segmentation methods besides the morphological watershed algorithm to apply to different application scenarios.
[0068] S4, quantizing the stray light blocks F1, F2, F3, ... to obtain the quantized value of the stray light in the light source image.
[0069] In step S4, quantizing the stray light blocks F1, F2, F3, ... includes: calculating the number of pixels included in each stray light block to obtain the area value of each stray light block; calculating the size of the circumscribed minimum rectangle of each stray light block to obtain the size of each stray light block; marking the position coordinates of each stray light block with the coordinates of its corresponding circumscribed minimum rectangle; and obtaining the energy value of each stray light block by a double discrete integration method.
[0070] The method for obtaining the energy value of each stray light block may include: traversing each stray light block among the stray light blocks F1, F2, F3..., and accumulating the pixel values of the pixel points within the standard area of each stray light block; repeating the above calculation steps until all stray light blocks F1, F2, F3... are calculated, and the accumulated pixel value corresponding to each stray light block can be obtained, and can be recorded as V1, V2, V3... respectively, that is, the stray light energy value of each stray light block.
[0071] In one embodiment, a rectangular coordinate system XOY can be established on the plane where the grayscale image Pgray is located, and the relative positions of the stray light blocks F1, F2, F3, etc. on the coordinate system remain unchanged, and the coordinates of the circumscribed minimum rectangle of each stray light block F1, F2, F3, etc., such as the upper left corner, in the coordinate system XOY can be marked as the position coordinates of the circumscribed minimum rectangle, and the position coordinates of each stray light block can be marked as the coordinates of the circumscribed minimum rectangle corresponding to it. The size of each stray light block can be obtained by calculating the size of the circumscribed minimum rectangle of each stray light block. Specifically, the number of corresponding pixels of the above-mentioned stray light blocks F1, F2, F3, ... can be counted according to the marking sequence, recorded as the area value A1, A2, A3, ... of each stray light block, and the circumscribed minimum rectangle of each stray light block F1, F2, F3, ... is calculated synchronously. In the process of counting the number of pixels of each marked block, the maximum and minimum values of the horizontal and vertical coordinates of the pixels in the marked stray light block in the coordinate system XOY are recorded, wherein the maximum value of the vertical coordinate is the upper side of the circumscribed minimum rectangle in the coordinate system XOY, the minimum value of the vertical coordinate is the lower side of the circumscribed minimum rectangle in the coordinate system XOY, the maximum value of the horizontal coordinate is the right side of the circumscribed minimum rectangle in the coordinate system XOY, and the minimum value of the horizontal coordinate is the left side of the circumscribed minimum rectangle in the coordinate system XOY. The shape framed by the four sides is the circumscribed minimum rectangle of each stray light block, and the multiple circumscribed minimum rectangles can be recorded as R1, R2, R3, ... respectively.
[0072] The size of the circumscribed minimum rectangle calculated according to the values of the positions of the four sides of each circumscribed minimum rectangle R1, R2, R3... can be used as the size of the corresponding stray light block. The coordinates of the upper left corner of each circumscribed minimum rectangle R1, R2, R3..., for example, can be recorded as the position coordinates of the corresponding stray light block. The energy value of each stray light block can also be obtained by a double discrete integration method.
[0073] The stray light quantification method provided in the present application is beneficial to improving the detection speed of the camera module to be tested and the accuracy of the detection results, avoiding the subjectivity of human eye observation. The quantified value of the stray light also helps to eliminate the stray light more accurately and improve the imaging quality of the camera module.
[0074] The present application also provides a stray light quantization detection system, including: a superposition device, a segmentation device and a quantization device.
[0075] The superposition device can be used to obtain the light source images captured by the camera module to be detected, splice and superimpose the light source images, and convert the spliced and superimposed light source images into grayscale images. The superposition device may include a superposition module and a conversion module. Among them, the superposition module can be used to normalize the light source images, and splice and superimpose the normalized light source images; the conversion module can be used to convert the spliced and superimposed light source images into grayscale images.
[0076] Specifically, the conversion module can be used to convert the composite image P formed after the splicing and superposition is completed into a grayscale image. First, the conversion module can perform RGB interpolation on the composite image P according to the corresponding Bayer arrangement format, and then convert the interpolated RGB image into a grayscale image for calculating a unified stray light component to avoid the influence of different channels on the response values of the light signal. Furthermore, the conversion module also sets the pixel points in the grayscale image whose pixel values are greater than or equal to 1023 (that is, the pixel values of the pixel points are greater than or equal to 1023 in the original data Raw10) to 0, excludes the overexposed stray light signal and the light source signal, and finally obtains a grayscale image recorded as Pgray.
[0077] The segmentation device can be used to mark the stray light area in the grayscale image Pgray, and use the morphological watershed algorithm to segment each stray light area to form a plurality of connected stray light blocks. The segmentation device may include: a boundary extraction module and a segmentation module. The boundary extraction module can be used to obtain a plurality of stray light areas with complete and continuous boundaries in the grayscale image, and mark the stray light areas according to the pixel values in the stray light areas; and the segmentation module can use the morphological watershed algorithm to segment each stray light area to form a plurality of connected stray light blocks.
[0078] Specifically, in order to more easily locate the position and shape of stray light in the grayscale image Pgray, the boundary extraction module needs to enhance the features of the grayscale image Pgray. The value of each pixel in the grayscale image Pgray is squared to obtain a square grayscale image, which is denoted as P 2 gray. Image P 2 gray uses the edge detection operator Canny to obtain the image boundary information. The boundary information is connected to obtain multiple stray light areas with complete and continuous boundary contours, which are recorded as B1, B2, B3... On this basis, different stray light areas B1, B2, B3... are marked according to the pixel value level in the stray light area and combined with the marked boundaries. The morphological watershed algorithm is used to segment each stray light area B1, B2, B3..., and the grayscale image Pgray is converted into a gradient image. In the gradient image, the local minimum values of multiple pixel values are used as the "centroid" to form multiple connected stray light blocks with the surrounding neighborhoods. The stray light blocks can be recorded as F1, F2, F3...
[0079] The quantization device can quantize the segmented stray light blocks. The quantization device may include: an area calculation module, a size calculation module, a marking module, and an energy value calculation module. Among them, the area calculation module can be used to calculate the number of pixels included in each stray light block to obtain the area value of each stray light block; the size calculation module can be used to calculate the size of the circumscribed minimum rectangle of each stray light block to obtain the size of each stray light block; the marking module can mark the position coordinates of each stray light block as the coordinates of its corresponding circumscribed minimum rectangle; and the energy value calculation module can obtain the energy value of each stray light block through a double discrete integration method.
[0080] The stray light quantitative detection system can input the acquired light source image, calculate and output the serial number, position, size, shape, area and energy value data of the stray light block in the light source image through the detection software. It is beneficial to improve the detection speed of the camera module to be tested and the accuracy of the detection results, avoid the subjectivity of human eye observation, and the quantitative value of stray light helps to eliminate stray light more accurately and improve the imaging quality of the camera module.
[0081] The superposition device may also include an image input module for inputting the acquired light source image to the stray light quantification detection system. At the same time, information including the file path and size of the light source image, the Bayer module, and different exposure values (ISO value of the sensitivity of the photosensitive element of the camera module to be tested) may also be input to the stray light quantification detection system.
[0082] The receiving format of the stray light quantification detection system for the light source image can be a BMP (Bitmap, standard image file format in the Windows operating system) single-channel format, that is, receiving the light source image data output by the BMP single-channel image, or it can be a YUV format image Y channel format, that is, receiving the light source image data in the Y image format; in addition, the light source image can also be in the RAW image G channel format, that is, receiving the light source image data of the RAW image G channel collected by the camera module to be tested.
[0083] The light source images can be obtained by iteratively adjusting the ISO value of the photosensitive element of the camera module to be tested. The light source images taken under each ISO value state can include all types of stray light.
[0084] In one embodiment, the light source image can be obtained by adjusting the ISO value En to 5×En-1, and obtaining a light source image Pn taken by the camera module to be tested when the ISO value is En, wherein n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image taken by the camera module to be tested when the ISO value is En.
[0085] The superposition device may also include a bad pixel detection module and a denoising module. Among them, the bad pixel detection module can use a 13×13 neighborhood traversal to screen the bad pixels of the input light source image in multiple channels; the denoising module can use a 5×5 neighborhood mean filter in multiple channels for denoising. As described above, in one embodiment, after determining the first ISO value of the camera module to be tested, an image of a point light source is taken at every 5-fold ISO setting until all types of stray light are reflected in the captured image, and each light source image is recorded as P1, P2, P3, P4...P8. The light source images P1, P2, P3, P4...P8 can be divided into four channels for bad pixel detection, the same abnormal pixels in the 8 images are screened out, and the pixel values of the pixels around the abnormal pixels are used to fill them. Among them, the bad pixel detection method can use a 13×13 neighborhood for traversal screening, while excluding the influence of overexposed pixels at the light source position. Afterwards, the eight images after bad pixels are removed are divided into four channels and denoised using a 5×5 neighborhood mean filter. The eight bad pixel and denoised light source images are normalized and each identical pixel in the eight light source images is accumulated and averaged to finally obtain the composite image P.
[0086] Figure 3 FIG. 1 is a flowchart of a method for using a stray light quantization system based on high dynamic image synthesis according to an embodiment of the present application. Figure 3 As shown, the method of use according to one embodiment of the present application may include the following steps:
[0087] S201, inputting the information of the acquired light source image, such as file path, size information, Bayer module and exposure value (sensitivity value of the photosensitive element of the camera module to be tested) into an image input module;
[0088] S202, extracting light source image data in multiple channels, for example, extracting light source image data in four channels;
[0089] S203, using a neighborhood to screen bad pixels in the light source image, for example, using a 13×13 neighborhood to screen bad pixels in the light source image;
[0090] S204, using a neighborhood mean filter to perform denoising on the light source image after bad pixels are removed in multiple channels. For example, a 5×5 neighborhood mean filter can be used to perform denoising on the light source image after bad pixels are removed in four channels.
[0091] S205, repeating step S203 and step S204 until all channels have completed bad pixel screening and denoising processing;
[0092] S206, performing normalization processing on the image, converting the processed image into an RGB image, and then converting the RGB image into a grayscale image;
[0093] S207, square the acquired grayscale image to enhance the image features, that is, square the pixel value of each pixel in the image.
[0094] S208, using the edge detection operator Canny to calculate and obtain the boundary information of the stray light, marking the boundary information and performing connectivity processing to obtain multiple stray light regions with complete and continuous boundaries, and marking the stray light regions according to the pixel values in the stray light regions.
[0095] S209, using a morphological watershed algorithm to segment each stray light area, and marking the segmented stray light blocks;
[0096] S210, calculating the number of pixels included in each stray light block to obtain the area value of each stray light block; calculating the size of the circumscribed minimum rectangle of each stray light block to obtain the size of each stray light block; marking the position coordinates of each stray light block as the coordinates of the corresponding circumscribed minimum rectangle;
[0097] S211, obtaining the energy value of each stray light block by a double discrete integration method;
[0098] S212, outputting the quantized value of the stray light block of the light source image, such as the sequence number, coordinates, size, shape, area value, energy value, etc. of the stray light block.
[0099] The method of using the stray light quantification system provided in the present application can output the quantified value of the stray light, which can be more intuitively and accurately benchmarked with the objective indicators of the camera module to be tested, thereby improving the efficiency and accuracy of mutual benchmarking and avoiding the waste of resources in the repeated benchmarking process.
[0100] The above description is only an implementation method of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of protection involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in this application.
Claims
1. A stray light quantification method, characterized in that: include: Adjusting the ISO value of the photosensitivity of the photosensitive element of the camera module to be tested, so as to obtain light source images taken by the camera module to be tested under the conditions of multiple different ISO values, and representing all types of stray light in the multiple light source images, wherein the next ISO value among the multiple different ISO values is greater than the previous ISO value; Splicing and superimposing a plurality of the light source images, and converting the spliced and superimposed light source images into grayscale images; Marking stray light areas in the grayscale image, and dividing each stray light area into a plurality of connected stray light blocks; as well as The stray light blocks formed by segmentation are quantized to obtain a quantized value of the stray light.
2. The method according to claim 1, characterized in that Quantizing the stray light blocks formed by segmentation to obtain the quantized value of the stray light includes: Calculating the number of pixels included in each stray light block to obtain an area value of each stray light block; Calculating the size of the minimum circumscribed rectangle of each stray light block to obtain the size of each stray light block; Marking the position coordinates of each stray light block as the coordinates of the corresponding circumscribed minimum rectangle; and The energy value of each stray light block is obtained by an integration method.
3. The method according to claim 1, characterized in that: Marking the stray light regions in the grayscale image and segmenting each of the stray light regions to form a plurality of connected stray light blocks comprises: Acquire a plurality of stray light regions having complete and continuous boundaries in the grayscale image; Marking the stray light region according to pixel values in the stray light region; and Each of the stray light regions is divided to form a plurality of connected stray light blocks.
4. The method according to claim 3, characterized in that Acquiring a plurality of stray light regions having complete and continuous boundaries in the grayscale image includes: Squaring pixel values in the grayscale image to obtain a squared grayscale image; Using the Canny edge detection operator to calculate the square grayscale image to obtain boundary information of the stray light area; and The boundary information is marked and connected to obtain a plurality of stray light regions with complete and continuous boundaries.
5. The method according to claim 1, characterized in that Acquiring the light source image captured by the camera module to be detected also includes: Setting a point light source and the camera module to be tested in a dark environment; and The camera module to be tested is clamped by a module clamp with an adjustable clamping angle.
6. The method according to claim 1, characterized in that Adjusting the ISO value of the light sensitivity of the photosensitive element of the camera module to be tested to obtain the light source image captured by the camera module to be tested under different ISO values includes: Adjusting the camera module to be tested to realize that the image of the point light source captured by the camera module to be tested is a real image, and recording the ISO value E1 when the light source image is a real image and obtaining the light source image P1; and The ISO value is iteratively adjusted to obtain light source images captured by the camera module to be tested under various ISO value states.
7. The method according to claim 6, characterized in that The method of iteratively adjusting the ISO value to obtain light source images captured by the camera module to be tested under various ISO value states includes: The ISO value En is adjusted to 5×En-1, and the light source image Pn captured by the camera module to be tested when the ISO value is En is obtained. Wherein, n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image taken by the camera module to be tested when the ISO value is En.
8. The method according to claim 7, characterized in that The light source images P1 to Pn include all types of stray light.
9. The method according to claim 7, characterized in that: The light source images are spliced and superimposed, and converting the spliced and superimposed light source images into grayscale images comprises: Performing bad pixel detection and denoising on the light source images P1 to Pn; Selecting effective pixels from the light source images P1 to Pn after bad pixel detection and denoising, where n is an integer and n≥2; Normalizing the light source images P1 to Pn after the effective pixel selection according to their respective normalization coefficients; splicing and superimposing the normalized light source images P1 to Pn into a composite image P; and The composite image P is converted into a grayscale image.
10. The method according to claim 9, characterized in that Selecting effective pixels of the light source images P1 to Pn after bad pixel detection and denoising includes: Select the pixel points that meet the following conditions in the light source images P1 to Pn after the bad pixel detection and denoising process, LSB ≥ 128 The LSB is the least significant bit of the pixel value in the raw data RAW10 in the light source images P1 to Pn.
11. The method according to claim 9, characterized in that The normalization coefficients include: The normalization coefficient X1 of the light source image P1 = Enmax / E1; and The normalization coefficient Xn of the light source image Pn=Enmax / En, Wherein, Enmax is the ISO value adjusted for the last time; E1 is the ISO value when the light source image is a real image; and En is the ISO value adjusted for the nth time, and n is an integer and n≥2.
12. The method according to claim 9, characterized in that The normalized light source images P1 to Pn are superimposed and spliced into a composite image P, comprising: Superimposing the pixel values of repeated pixels in the normalized light source images P1 to Pn and calculating the average value; and The average values are concatenated into a composite image P.
13. The method according to claim 9, characterized in that Converting the composite image P into a grayscale image comprises: Performing RGB interpolation on the synthesized image P according to the corresponding Bayer array; Converting the interpolated RGB image into the grayscale image; and The pixels in the grayscale image whose pixel values are greater than or equal to 1023 are set to 0.
14. The method according to any one of claims 1 to 13, characterized in that Dividing each of the stray light areas to form a plurality of connected stray light blocks comprises: A morphological watershed algorithm is used to segment each stray light region to form the plurality of connected stray light blocks.
15. A stray light quantization system, characterized in that: include: A superposition device, which adjusts the ISO value of the sensitivity of the photosensitive element of the camera module to be tested, obtains the light source images taken by the camera module to be tested under the state of multiple different ISO values, and represents all types of stray light in the multiple light source images, wherein the next ISO value among the multiple different ISO values is greater than the previous ISO value, splices and superimposes the multiple light source images, and converts the spliced and superimposed light source images into grayscale images; A segmentation device, marking stray light areas in the grayscale image, and segmenting each stray light area to form a plurality of connected stray light blocks; as well as A quantization device quantizes the stray light blocks formed by segmentation.
16. The system according to claim 15, characterized in that The quantization device comprises: An area calculation module, used for calculating the number of pixels included in each stray light block to obtain an area value of each stray light block; A size calculation module, used for calculating the size of the circumscribed minimum rectangle of each stray light block to obtain the size of each stray light block; a marking module, marking the position coordinates of each stray light block as the coordinates of the corresponding circumscribed minimum rectangle; and The energy value calculation module obtains the energy value of each stray light block by an integration method.
17. The system according to claim 15, characterized in that The superposition device comprises: a superposition module, used for normalizing the light source images, and splicing and superimposing the normalized light source images; and The conversion module is used to convert the light source image after splicing and superposition into a grayscale image.
18. The system according to claim 15, characterized in that The segmentation device comprises: a boundary extraction module, configured to obtain a plurality of stray light regions with complete and continuous boundaries in the grayscale image, and mark the stray light regions according to pixel values in the stray light regions; and The segmentation module uses a morphological watershed algorithm to segment each of the stray light areas to form a plurality of connected stray light blocks.
19. The system according to claim 18, characterized in that The method for obtaining a plurality of stray light regions with complete and continuous boundaries in the grayscale image includes: Used to square the pixel values in the grayscale image to obtain a squared grayscale image; Using the Canny edge detection operator to calculate the square grayscale image to obtain boundary information of the stray light area; and The boundary information is marked and connected to obtain a plurality of stray light regions with complete and continuous boundaries.
20. The system according to claim 15, characterized in that The format of the light source image is at least one of BMP, YUV and RAW.
21. The system according to claim 15, characterized in that The system further comprises: By iteratively adjusting the ISO value of the photosensitive element, the light source images captured by the camera module to be tested under various ISO value states are obtained respectively.
22. The system according to claim 21, characterized in that The system further comprises: By adjusting the ISO value En to 5×En-1, the light source image Pn captured by the camera module to be tested when the ISO value is En is obtained. Wherein, n is the number of times the ISO value is adjusted, n is an integer and n≥2; En is the ISO value adjusted for the nth time; En-1 is the ISO value adjusted for the n-1th time; and Pn is the light source image taken by the camera module to be tested when the ISO value is En.
23. The system according to claim 21, characterized in that The light source image includes all stray light types.
24. The system according to claim 15, characterized in that The superposition device also includes: The bad pixel detection module uses multiple channels to screen bad pixels using neighborhood traversal; and The denoising module uses a neighborhood mean filter to denoise multiple channels.
25. A method of using the stray light quantification system according to any one of claims 15 to 24, characterized in that: The method comprises: Inputting the acquired light source image information and the sensitivity value of the photosensitive element of the camera module to be tested into the system; Screening the bad pixels of the light source image by using neighborhood traversal on the light source image in multiple channels; Using a neighborhood mean filter to perform denoising on the light source image after bad pixel removal in multiple channels; Normalizing the light source images that have undergone bad pixel screening and denoising, and splicing and superimposing the light source images that have undergone normalization to convert them into grayscale images; Acquiring boundary information of stray light, marking the boundary information and performing connectivity processing to obtain a plurality of stray light regions with complete and continuous boundaries, and marking the stray light regions according to pixel values in the stray light regions; Using a morphological watershed algorithm to segment each stray light region to form a plurality of connected stray light blocks; Calculating the number of pixels included in each stray light block to obtain an area value of each stray light block; Calculating the size of the minimum circumscribed rectangle of each stray light block to obtain the size of each stray light block; Marking the position coordinates of each stray light block as the coordinates of the corresponding circumscribed minimum rectangle; Obtaining the energy value of each stray light block by an integration method; and The sequence number, coordinates, size, shape, area value and energy value of the stray light block of the light source image are output.
