Method, device and electronic device for determining device dirt specification

Abstract

The present application discloses a method, device and electronic device for determining the dirt specification of a device. A preset optical imaging model is constructed in advance according to the imaging module. By simulating dirt areas with a variety of different reference dirt specifications on the target surface in the preset optical imaging model and generating corresponding optical spectrum data, and then based on the optical spectrum data, generating an imaging image under the action of the dirt area with the corresponding reference dirt specification, so as to determine the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets the preset qualified conditions, which can effectively improve the efficiency of determining the dirt specification of optical components and reduce costs.

Description

Technical Field

[0001] This application relates to the field of optical detection technologies, and in particular, to a method, apparatus, and electronic device for determining the dirt specification of a device. Background Art

[0002] With more and more electronic products having camera functions, such as smartphones and tablet computers, the imaging quality of electronic products has also become an important factor affecting their market competitiveness. The imaging quality is often determined by the camera module in the electronic device. Therefore, the production quality of the camera module is particularly important.

[0003] The degree of dirt on optical components is one of the important factors affecting the imaging quality of the camera module. At present, for the dirt specification of optical components in the camera module, first, the manufacturer is required to select different dirt specification gradient products on the surface of the optical components, then use the selected dirt specification gradient products of the optical components to form an actual module, and then through actual imaging shooting, determine the imaging interval of the gradient products, so as to obtain the imaging specification of the optical components. Then the manufacturer or internal production produces according to the specification. This method has low efficiency and will also cause cost waste. Summary of the Invention

[0004] Embodiments of this application can effectively improve the efficiency of determining the dirt specification of optical components and reduce costs by providing a method, apparatus, and electronic device for determining the dirt specification of a device.

[0005] In a first aspect, embodiments of this application provide a method for determining the dirt specification of a device, which is used to determine the dirt specification on the surface of an optical component in a camera module. The method includes:

[0006] Obtain multiple sets of optical spectrum data corresponding to each target surface in a preset optical imaging model, where the preset optical imaging model is constructed according to the camera module, the multiple sets of optical spectrum data are image plane illuminance distribution data obtained after simulating dirt regions on the corresponding target surface according to a preset dirt specification sequence, the preset dirt specification sequence includes multiple reference dirt specifications arranged from small to large, and each set of optical spectrum data of the same target surface corresponds to a reference dirt specification;

[0007] For each set of optical spectrum data of each target surface, generate an imaging image under the action of a dirt region with the corresponding reference dirt specification based on the optical spectrum data;

[0008] For each target surface, determine the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets a preset qualification condition.

[0009] Further, each set of optical spectrum data corresponding to each target surface is generated according to the following steps:

[0010] Simulate the contaminated areas of the same reference contamination specification at multiple different half-image height positions on the target surface;

[0011] By performing ray tracing on the preset optical imaging model, generate the illuminance distribution data at the corresponding positions on the image plane under the action of the simulated contaminated areas, and use the generated illuminance distribution data as a set of the optical spectrum data of the corresponding target surface.

[0012] Further, generating an imaging image under the action of the contaminated areas of the corresponding reference contamination specification based on the optical spectrum data includes:

[0013] Convert the optical spectrum data into a picture containing the original image information;

[0014] Based on the pixel arrangement of the image sensor in the camera module, perform color filtering and color analysis processing on the picture to obtain an imaging image under the action of the contaminated areas of the corresponding reference contamination specification.

[0015] Further, the sizes of the reference contamination specifications in the preset contamination specification sequence are arranged in equal gradients, where the minimum size is 10 μm, the maximum size is 100 μm, and the gradient is 10 μm.

[0016] Further, detecting whether the imaging image meets the preset qualified conditions includes:

[0017] By comparing the brightness differences between each pixel point in the imaging image and the pixel points in the preset neighborhood, detect the contaminated pixel groups and bright pixel groups in the imaging image. The contaminated pixel groups include multiple adjacent contaminated pixels, the bright pixel groups include multiple adjacent bright pixels, and both the contaminated pixels and the bright pixels are abnormal pixel points caused by the influence of the contaminated areas;

[0018] Based on the number of detected contaminated pixel groups, the number of bright pixel groups, the number of contaminated pixels included in each contaminated pixel group, and the number of bright pixels included in each bright pixel group, determine whether the imaging image meets the preset qualified conditions.

[0019] Further, the preset qualified conditions include:

[0020] The number of contaminated pixel groups is less than or equal to a first preset threshold, and the number of contaminated pixels included in each contaminated pixel group is less than or equal to a second preset threshold; and

[0021] The number of bright pixel groups is less than or equal to a third preset threshold, and the number of bright pixels included in each bright pixel group is less than or equal to a fourth preset threshold.

[0022] Further, determining the contamination specification of the target surface from the preset contamination specification sequence includes:

[0023] Arrange the reference dirt specifications corresponding to the last one in the preset dirt specification sequence in ascending order, and determine the dirt specification of the target surface as the reference dirt specification for which the corresponding imaging image meets the preset qualification conditions.

[0024] Further, before obtaining multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model, it further includes:

[0025] Obtain the reference optical spectrum data of the preset optical imaging model. The reference optical spectrum data is the image plane illuminance distribution data obtained when there are no dirty areas in each optical component in the preset optical imaging model.

[0026] Generate a reference imaging image based on the reference optical spectrum data.

[0027] If the reference imaging image meets the preset calibration conditions, then execute the step of obtaining multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model, and determine the dirt specification of each target surface.

[0028] If the reference imaging image does not meet the preset calibration conditions, then determine that the preset optical imaging model is abnormal, and stop determining the dirt specification of the target surface.

[0029] In a second aspect, an embodiment of the present application provides a device dirt specification determination device for determining the dirt specification of the surface of an optical component in a camera module. The device includes:

[0030] A data acquisition module for acquiring multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model. The preset optical imaging model is constructed according to the camera module. The multiple sets of optical spectrum data are image plane illuminance distribution data obtained after simulating dirty areas on the corresponding target surface according to a preset dirt specification sequence. The preset dirt specification sequence includes multiple reference dirt specifications arranged from small to large, and each set of optical spectrum data of the same target surface corresponds to a reference dirt specification.

[0031] A generation module for generating an imaging image under the action of a dirty area of the corresponding reference dirt specification for each set of optical spectrum data of each target surface based on the optical spectrum data.

[0032] A determination module for determining the dirt specification of the target surface from the preset dirt specification sequence for each target surface by detecting whether the imaging image meets the preset qualification conditions.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, the steps of the device dirt specification determination method provided in the first aspect are implemented.

[0034] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0035] For the device dirt specification determination method provided in the embodiment of the present application, a preset optical imaging model is pre-constructed according to the camera module. By simulating dirt regions with a variety of different reference dirt specifications on the target surface in the preset optical imaging model and generating corresponding optical spectrum data, and then generating imaging images under the action of dirt regions with corresponding reference dirt specifications based on the optical spectrum data, so as to determine the dirt specification of the target surface from a preset dirt specification sequence by detecting whether the imaging image meets the preset qualified conditions. This not only reduces the labor and cost, but also can efficiently and relatively accurately complete the determination of the dirt specification on the surface of the optical component, which is beneficial to improving the quality of the optical component and thus improving the production yield of the camera module. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a schematic structural diagram of a camera module in an embodiment of the present application;

[0038] Figure 2 It is a flowchart of a device dirt specification determination method in an embodiment of the present application;

[0039] Figure 3 It is a schematic structural diagram of a preset optical imaging model in an embodiment of the present application;

[0040] Figure 4 It is a set of exemplary relative illuminance optical spectrum data diagrams in an embodiment of the present application;

[0041] Figure 5 It is a schematic diagram of an exemplary dirt region layout in an embodiment of the present application;

[0042] Figure 6 It is a picture containing original image information in an embodiment of the present application;

[0043] Figure 7 It is the imaging image in an embodiment of the present application;

[0044] Figure 8 This is a block diagram of a device dirt specification determination device in an embodiment of the present application;

[0045] Figure 9 This is a schematic structural diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0046] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0047] Schematic structural diagrams according to embodiments of the present application are shown in the accompanying drawings. These figures are not drawn to scale, and for the purpose of clear expression, some details are enlarged and some details may be omitted. The shapes of the various structures shown in the figures and their relative sizes and positional relationships are only exemplary, and in practice, there may be deviations due to manufacturing tolerances or technical limitations, and those skilled in the art can design structures with different shapes, sizes, and relative positions according to actual needs.

[0048] Generally speaking, as Figure 1 shown, the camera module 1 includes structures such as a lens 10, an infrared cut-off filter 20, and an image sensor 30. Among them, the dirt conditions of optical components such as the infrared cut-off filter 20 and the lenses in the lens 10 will affect the imaging quality of the assembled camera module 1. Therefore, it is necessary to strictly control the dirt specifications of these optical components.

[0049] Considering the current methods for determining dirt specifications, the cost is relatively high, and it also requires a large amount of human resources and time. Moreover, it is difficult to select a relatively comprehensive gradient product, resulting in incomplete verification and the reliability of the determined dirt specification results cannot be guaranteed.

[0050] In view of this, the embodiment of the present application provides a method for determining the dirt specification of a device. First, a preset optical imaging model is constructed according to the camera module. By simulating dirt areas with a variety of different reference dirt specifications on the target surface in the preset optical imaging model and generating corresponding optical spectrum data, and then based on the optical spectrum data, generating an imaging image under the action of the dirt area with the corresponding reference dirt specification, so as to determine the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets the preset qualified conditions. This not only reduces the human and cost, but also can efficiently and accurately complete the determination of the dirt specification of the optical component surface, which is beneficial to improving the quality of the optical component and thus improving the production yield of the camera module.

[0051] The following will separately elaborate on the device dirt specification determination method, device, and electronic device provided in the embodiments of the present application. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of the present application, rather than limitations on the technical solutions of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0052] In a first aspect, an embodiment of the present application provides a method for determining a device dirt specification, which is used to determine the dirt specification on the surface of an optical component in a camera module. As Figure 2 shown, this method may include the following steps S101 to S103.

[0053] Step S101: Obtain multiple groups of optical spectrum data corresponding to each target surface in a preset optical imaging model. The multiple groups of optical spectrum data are image plane illuminance distribution data obtained after simulating a dirt area on the corresponding target surface according to a preset dirt specification sequence. Each group of optical spectrum data for the same target surface corresponds to a reference dirt specification.

[0054] In this embodiment, the preset optical imaging model is constructed according to the optical imaging system in an actual camera module. For example, Figure 3 shows an exemplary preset optical imaging model, including a lens cover glass CG of the camera module, a lens group (such as including Figure 3 the lenses L1 to L5 shown in it), an infrared cut-off filter IR, and an image plane IMG. The incident light passes through the lens cover glass CG, the lens group L1 to L5, and the infrared cut-off filter IR in sequence, and then forms an image on the image plane IMG.

[0055] In specific implementation, the target surface for which the dirt specification needs to be determined can be selected in the constructed optical imaging model according to actual needs. In an optional implementation manner, in order to reduce the amount of data processing and save the specification test time, one lens surface closest to the image sensor in the lens, and two surfaces of the infrared cut-off filter IR can be used as the target surfaces. For example, in Figure 3 the exemplary preset optical imaging model shown, the surface S1 of the lens L5 close to the infrared cut-off filter IR, and the two surfaces S2 and S3 of the infrared cut-off filter IR can be determined as the target surfaces. Of course, in other application scenarios, other optical component surfaces in the preset optical imaging model can also be used as the target surfaces. For example, each surface of each optical component in the optical path can be used as the target surface. This embodiment does not limit this.

[0056] In addition, it is necessary to determine multiple different reference dirt specifications in advance according to actual experience, and arrange these reference dirt specifications in ascending order to form a preset dirt specification sequence. The shape of the simulated dirt area can be configured according to the needs of the actual application scenario. For example, it can be circular or square, etc. The reference dirt specification includes the size of the simulated dirt area. For example, taking the shape of the dirt area as a circle, the size of the dirt area can be the diameter of the circular dirt area. For example, the sizes of the reference dirt specifications in the preset dirt specification sequence can be arranged in equal gradients. For example, the minimum size is 10μm, the maximum size is 100μm, and the gradient is 10μm. Of course, other size ranges and gradients can also be set according to actual needs, and this embodiment does not limit this.

[0057] Further, for each target surface, simulate the dirt area on the target surface according to the preset dirt specification sequence. Of course, each time a dirt area of a reference dirt specification is simulated on a target surface, a collection is made to generate a set of optical spectrum data of the target surface under the reference dirt specification. It should be noted that the simulation of the dirt area can be achieved by setting a light-shielding area of the corresponding specification on the surface of the component.

[0058] For example, Figure 4 shows a set of exemplary relative illuminance optical spectrum data graphs, where the abscissa is the half-image height field of view and the ordinate is the relative illuminance. This set of optical spectrum data is generated when a circular dirt area with a specification size of 40μm is simulated on the S3 surface of the above-mentioned infrared cut-off filter IR and the imaging distance is infinity. Among them, the S3 surface is the surface close to the image sensor side in the camera module. It can be understood that the farther the imaging distance is, the closer the lens of the camera module is to the image sensor. When the imaging distance is infinity, the lens is closest to the image sensor. If the generated imaging image meets the preset qualified conditions at this time, the imaging images generated at other imaging distances also meet the preset qualified conditions. Therefore, in an optional implementation manner, the multiple sets of optical spectrum data corresponding to each target surface can all be generated when the imaging distance is infinity.

[0059] Specifically, for each target surface, the process of obtaining the optical spectrum data may include: simulating the dirt area of the same reference dirt specification at multiple different half-image height positions on the target surface, and then generating the illuminance distribution data at the corresponding positions of the image plane IMG under the action of the simulated dirt area by performing ray tracing on the above-mentioned preset optical imaging model, and taking the generated illuminance distribution data as a set of optical spectrum data of the target surface.

[0060] Furthermore, according to the above preset dirt specification sequence, by changing the size of the dirt area simulated on the target surface, multiple groups of optical spectrum data corresponding to the target surface can be obtained, that is, the optical spectrum data under different dirt specifications. Different groups of optical spectrum data correspond to different reference dirt specifications in the preset dirt specification sequence. For example, successively simulate a dirt area with a specification of 10 μm on the S3 surface of the infrared cut-off filter IR, and obtain the optical spectrum data generated under the influence of the dirt area with a specification of 10 μm, which is recorded as the first group of optical spectrum data corresponding to the S3 surface; simulate a dirt area with a specification of 20 μm on the S3 surface of the infrared cut-off filter IR, and obtain the optical spectrum data generated under the influence of the dirt area with a specification of 20 μm, which is recorded as the second group of optical spectrum data corresponding to the S3 surface; and so on, obtain the optical spectrum data generated under the influence of the dirt area with a specification of 100 μm, which is recorded as the tenth group of optical spectrum data corresponding to the S3 surface.

[0061] For example, when simulating a dirt area on the target surface, as Figure 5 shown, based on the semi-image height of the effective imaging area AIMG on the image plane IMG, multiple different semi-image height positions can be determined on the target surface 100, such as Figure 5 P1 to P6 shown in. Among them, the semi-image height is half of the diagonal length of the effective imaging area AIMG. It should be noted that Figure 5 the six semi-image height positions P1 to P6 shown in are only for illustration and not for limitation. The specific division of the semi-image height positions can be determined according to actual needs. For example, the semi-image height position range can be normalized to 0 to 1 and evenly divided into 100 parts, and then a dirt area is set every 0.01. Considering that the relative illumination value of imaging at the same semi-image height position on the image plane IMG is the same, only one dirt area needs to be simulated at the same semi-image height position. It should be noted that the same specification of dirt areas set at different semi-image height positions on the same surface have different effects on imaging. The closer the dirt is to the center position of the component, the smaller the impact on imaging, and vice versa, the closer to the edge position of the component, the greater the impact on imaging.

[0062] Of course, in other embodiments of the present invention, it is also possible to simulate a dirt area only at a specified position on one target surface. The specific simulation position of the dirt area on the target surface can be determined according to the needs of the actual scenario, and this embodiment does not limit this.

[0063] In an alternative embodiment, in order to further improve the accuracy of the determined dirt specification result, first, according to the ideal situation, that is, when all optical components in the optical imaging model meet the design specifications of the camera module and have no dirt, the preset calibration conditions for the imaging quality to be achieved are configured to calibrate the constructed optical imaging model. Then, before performing the above step S101, the imaging calibration step is first performed.

[0064] Specifically, the above imaging calibration steps may include: obtaining reference optical spectrum data of a preset optical imaging model, where the reference optical spectrum data is image plane illuminance distribution data obtained when there are no dirty areas in each optical component in the preset optical imaging model; generating a reference imaging image based on the reference optical spectrum data; if the reference imaging image meets the preset calibration conditions, indicating that the preset optical imaging model is normal, then steps S101 to S103 can be executed on the basis of this preset optical imaging model to determine the dirt specifications of each target surface; if the reference imaging image does not meet the preset calibration conditions, it is determined that the preset optical imaging model is abnormal and the determination of the dirt specifications of the target surface needs to be stopped. For example, a prompt message can be sent to prompt that the optical imaging model is abnormal and the abnormality existing in the model itself needs to be resolved first. It should be noted that in the above process, the implementation process of generating a reference imaging image based on the reference optical spectrum data is similar to the implementation process of generating an imaging image based on the optical spectrum data in step S102 below, and the specific implementation process will be described in the relevant part of step S102 below.

[0065] This can avoid the problems existing in the optical imaging model itself from affecting the imaging quality, so as to ensure that the imaging abnormalities in the imaging images obtained in the subsequent steps are all caused by the simulated dirty areas, which is beneficial to improving the accuracy of the determination result of the dirt specifications.

[0066] Step S102: For each group of optical spectrum data of each target surface, generate an imaging image under the action of a dirty area with a corresponding reference dirt specification based on the optical spectrum data.

[0067] Specifically, in an optional implementation manner, each group of optical spectrum data obtained in step S101 can be respectively converted into a picture containing original image information; then, based on the pixel arrangement of the image sensor in the camera module, color filtering and color analysis processing are performed on the picture to obtain an imaging image under the action of a dirty area with a corresponding reference dirt specification.

[0068] Since at the same semi-image height position on the target surface (such as Figure 5The relative illumination values of the images formed on the image plane IMG at the same elliptical dotted line positions shown are the same. Therefore, for each target surface, by obtaining a set of optical spectrum data (such as the relative illumination data at corresponding positions on the image plane IMG when dirty areas with the same reference dirt specification are set at P1 to P6), the relative illumination values at corresponding positions on the image plane IMG can be obtained when the dirty areas with this reference dirt specification are set at other positions on the target surface. In this way, the relative illumination distribution of the effective imaging area on the image plane IMG can be obtained when the dirty areas with this reference dirt specification are arranged on the entire target surface, and thus a picture containing the original image information can be obtained, such as Figure 6 shown. It should be noted that in practical applications, the picture is in color, Figure 6 and what is shown is the picture after grayscale processing.

[0069] It should be noted that the original picture converted from the optical spectrum data can be understood as the image picture that has not undergone color filtering and subsequent color analysis processing by the image sensor when the actual camera module is in use. Further, it is also necessary to refer to the pixel arrangement in the image sensor of the camera module in the actual application scenario, that is, the arrangement of the color filter array (Color Filter Array, abbreviated as CFA), to perform corresponding color filtering on the picture, and then perform color analysis to obtain the tricolor component data to obtain the color imaging data, that is, the imaging image, such as Figure 7 shown. It should be noted that in practical applications, the imaging image is in color, Figure 7 and what is shown is the imaging image after grayscale processing. For example, if the odd-row filters of the CFA in the image sensor are arranged in a cycle of RGRG, and the even-row filters are arranged in a cycle of GBGB, then the weight of green (G) in each pixel is 2, and the weights of red (R) and blue (B) are 1. The specific process can refer to the color filtering and color analysis principles of the image sensor, and will not be elaborated here.

[0070] In this way, for each target surface, for each reference dirt specification in the preset dirt specification sequence, a set of imaging data can be obtained, and thus an imaging image can be obtained. For example, taking the target surface including the S1 surface of the above-mentioned lens L5, and the S2 and S3 surfaces of the infrared cut-off filter IR, and the preset dirt specification sequence including 10μm, 20μm,..., 100μm as an example, through the above steps S101 and S102, 10 imaging images corresponding to the S1 surface, 10 imaging images corresponding to the S2 surface, and 10 imaging images corresponding to the S3 surface can be obtained respectively, and one imaging image corresponds to one reference dirt specification.

[0071] Further, for the obtained imaging image, the following step S103 can be executed to determine the dirt specification.

[0072] Step S103: For each target surface, determine the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets the preset qualified conditions.

[0073] Before determining the dirt specification, it is necessary to pre-configure the qualified conditions according to the imaging quality requirements of the actual camera module to measure the impact of dirt areas of different specifications on imaging.

[0074] For example, the imaging requirements of the camera module are as follows: It is allowed to have a dirt point clusters, where each dirt point cluster is composed of multiple adjacent dirt points, and each dirt point cluster is allowed to have N dirt points; it is allowed to have b bright point clusters, where each bright point cluster is composed of multiple adjacent bright points, and each bright point cluster is allowed to have M bright points. Here, a, b, N, and M are all determined according to the requirements of the actual application scenario. It can be understood that both dirt points and bright points are abnormally bright pixels. Among them, dirt points are pixels with too low brightness, and bright points are pixels with too high brightness. Then, the preset qualified conditions can include: a dirt point sub-condition and a bright point sub-condition. The dirt point cluster number sub-condition is that the number of dirt point clusters in the imaging image is less than or equal to the first preset threshold, and the number of dirt points included in each dirt point cluster is less than or equal to the second preset threshold. The bright point cluster number sub-condition is that the number of bright point clusters in the imaging image is less than or equal to the third preset threshold, and the number of bright points included in each bright point cluster is less than or equal to the fourth preset threshold. Among them, the first preset threshold, the second preset threshold, the third preset threshold, and the fourth preset threshold can be set according to the above a, N, b, M, and the actual specification test requirements.

[0075] Therefore, the process of detecting whether the imaging image meets the preset qualified conditions can include: detecting the dirt point clusters and bright point clusters in the imaging image, and determining whether the imaging image meets the preset qualified conditions based on the detected number of dirt point clusters, the number of bright point clusters, the number of dirt points included in each dirt point cluster, and the number of bright points included in each bright point cluster. Here, both dirt points and bright points are considered as abnormal pixels affected by the dirt area.

[0076] Taking the above-mentioned qualified conditions as an example, if the number of detected dirt point clusters is less than or equal to the first preset threshold, and the number of dirt points in each dirt point cluster is less than or equal to the second preset threshold; the number of bright point clusters is less than or equal to the third preset threshold, and the number of bright points in each bright point cluster is less than or equal to the fourth preset threshold, then it is determined that the imaging image meets the preset qualified conditions; otherwise, if the number of detected dirt point clusters is greater than the first preset threshold, or there is a dirt point cluster with the number of dirt points greater than the second preset threshold, or the number of detected bright point clusters is greater than the third preset threshold, or there is a bright point cluster with the number of bright points greater than the fourth preset threshold, then it is determined that the imaging image does not meet the preset qualified conditions. For example, Figure 7 the imaging image shown in (a) in Figure 7 does not meet the preset qualified conditions, and there is a dirt point cluster with the number of dirt points greater than the second preset threshold at the position marked by the ellipse, while

[0077] the imaging image shown in (b) in

[0078] meets the preset qualified conditions. Specifically, the dirt point clusters and bright point clusters in the imaging image can be detected by comparing the brightness differences between each pixel point in the imaging image and the pixel points in the preset neighborhood. For example, the sub-images of each color channel in each imaging image can be extracted first, such as the R-channel sub-image, the G-channel sub-image, and the B-channel sub-image; then, the dirt point clusters and bright point clusters are detected respectively for the sub-images of each color channel. For example, the brightness of the i-th pixel point in the sub-image is denoted as Pi, and the average brightness of the 8 pixel points around this pixel point is denoted as Yi. If (Yi - Pi) / Yi * 100 > the dirt comparison threshold, then the i-th pixel point is recorded as a dirt point, and if (Pi - Yi) / Yi * 100 > the bright point comparison threshold, then the i-th pixel point is recorded as a bright point. Further, all adjacent dirt points are recorded as a dirt point cluster, and all adjacent bright points are recorded as a bright point cluster.

[0079] Alternatively, in other embodiments, the captured image may also be compared with the reference captured image that meets the preset calibration conditions to detect the dirty and bright spots in the captured image. For example, the sub-images of each color channel in each captured image and the sub-images of each color channel in the reference captured image may be extracted. For each color channel of the captured image, the brightness of the i-th pixel is denoted as Pi, and the brightness of the i-th pixel in the sub-image of the corresponding color channel of the reference captured image is denoted as Yi'. If (Yi' - Pi) / Yi' * 100 > the dirty spot comparison threshold, then the i-th pixel is marked as a dirty spot. If (Pi - Yi') / Yi' * 100 > the bright spot comparison threshold, then the i-th pixel is marked as a bright spot.

[0080] It should be noted that there are multiple ways to determine whether the captured image is qualified, not limited to the embodiments listed above. For details, reference may be made to related technologies.

[0081] For each target surface, each captured image corresponds to a reference dirt specification. By detecting whether the captured image meets the preset qualified conditions, it is possible to determine whether the dirt area with the corresponding reference dirt specification on the target surface will affect the imaging quality.

[0082] During specific implementation, the detection order of the captured images corresponding to the preset dirt specification sequence can be determined according to actual needs, and this embodiment does not limit this.

[0083] For example, for each target surface, in ascending order, the reference dirt specification corresponding to the last captured image that meets the preset qualified conditions in the preset dirt specification sequence can be determined as the dirt specification of the target surface. For example, taking the S1 surface of the above lens L5 as an example, the captured images corresponding to the reference dirt specifications of 10μm, 20μm, 30μm, 40μm,..., 100μm are sequentially named as the 1st - 10th captured images. Each captured image is detected in the order of the 1st - 10th captured images. The 1st - 4th captured images all meet the preset qualified conditions, while the 5th captured image does not meet the preset qualified conditions. Then, the reference dirt specification corresponding to the last captured image that meets the preset qualified conditions is 40μm, and 40μm is determined as the dirt specification of the S1 surface.

[0084] Alternatively, in descending order, the reference dirt specification corresponding to the first captured image that meets the preset qualified conditions in the preset dirt specification sequence can be determined as the dirt specification of the target surface. For example, in the above example, when each captured image is detected in the order of the 10th - 1st captured images, the reference dirt specification corresponding to the first captured image that meets the preset qualified conditions is also 40μm.

[0085] In this way, the dirt specifications of each target surface can be determined. It should be noted that if the target surface includes a lens surface closest to the image sensor in the lens and two surfaces of the infrared cut-off filter IR, such as Figure 3 the S1 surface, S2 surface, and S3 surface shown in

[0086] then the dirt specifications of the S1 surface, S2 surface, and S3 surface can be determined respectively. Compared with the S1 surface, other lens surfaces in the lens are farther from the image sensor. The dirt specification of the S1 surface can also be used as the dirt specification of other lens surfaces. When the dirt specification is met, the imaging requirements of the camera module can be satisfied.

[0087] After that, when manufacturing the camera module, the dirt detection can be carried out on each optical component required for assembling the camera module, such as each lens included in the lens and the infrared cut-off filter IR, according to the determined dirt specifications, and the optical components exceeding the dirt specifications can be removed in time to avoid waste of materials caused by dirty products entering the subsequent production process and ensure the imaging quality of the product.

[0088] In a second aspect, based on the same inventive concept, an embodiment of the present application further provides a device dirt specification determination device for determining the dirt specifications of the surfaces of optical components in a camera module. As Figure 8 shown, the device dirt specification determination device 80 includes:

[0089] A data acquisition module 801, configured to acquire multiple sets of optical spectrum data corresponding to each target surface in a preset optical imaging model, where the preset optical imaging model is constructed according to the camera module, and the multiple sets of optical spectrum data are image plane illuminance distribution data obtained after simulating dirt regions on the corresponding target surfaces according to a preset dirt specification sequence. The preset dirt specification sequence includes multiple reference dirt specifications arranged from small to large, and each set of optical spectrum data of the same target surface corresponds to a reference dirt specification;

[0090] A generation module 802, configured to generate an imaging image under the action of a dirt region with a corresponding reference dirt specification for each set of optical spectrum data of each target surface based on the optical spectrum data;

[0091] A determination module 803, configured to determine the dirt specification of each target surface from the preset dirt specification sequence by detecting whether the imaging image meets a preset qualification condition.

[0092] In an optional implementation manner, the above data acquisition module 801 is specifically configured to:

[0093] Simulate dirt areas of the same reference dirt specification at multiple different half-image height positions of the target surface;

[0094] Generate illuminance distribution data at corresponding positions on the image plane under the action of the simulated dirt areas by performing ray tracing on the preset optical imaging model, and use the generated illuminance distribution data as a set of the optical spectrum data of the corresponding target surface.

[0095] In an optional implementation manner, the above generation module 802 is specifically configured to:

[0096] Convert the optical spectrum data into a picture containing original image information;

[0097] Based on the pixel arrangement of the image sensor in the camera module, perform color filtering and color analysis processing on the picture to obtain an imaging image under the action of the dirt area of the corresponding reference dirt specification.

[0098] In an optional implementation manner, the sizes of the reference dirt specifications in the preset dirt specification sequence are arranged in equal gradients, where the minimum size is 10 μm, the maximum size is 100 μm, and the gradient is 10 μm.

[0099] In an optional implementation manner, the above determination module 803 is specifically configured to:

[0100] Detect dirt point groups and bright point groups in the imaging image by comparing the brightness differences between each pixel point in the imaging image and the pixel points in a preset neighborhood. The dirt point groups include multiple adjacent dirt points, the bright point groups include multiple adjacent bright points, and both the dirt points and the bright points are abnormal pixel points caused by the influence of the dirt areas;

[0101] Determine whether the imaging image meets the preset qualification condition based on the detected number of dirt point groups, the number of bright point groups, the number of dirt points included in each dirt point group, and the number of bright points included in each bright point group.

[0102] In an optional implementation manner, the above preset qualification condition includes:

[0103] The number of dirt point groups is less than or equal to a first preset threshold, and the number of dirt points included in each dirt point group is less than or equal to a second preset threshold; and

[0104] The number of the highlight groups is less than or equal to a third preset threshold, and the number of highlights included in each of the highlight groups is less than or equal to a fourth preset threshold.

[0105] In an alternative embodiment, the determining module 803 is specifically configured to:

[0106] Determine, in ascending order, the reference dirt specification corresponding to the last one in the preset dirt specification sequence, for which the corresponding imaging image satisfies the preset qualification condition, as the dirt specification of the target surface.

[0107] In an alternative embodiment, the device dirt specification determining apparatus 80 further includes: a calibration module, configured to:

[0108] Obtain reference optical spectrum data of the preset optical imaging model, where the reference optical spectrum data is image plane illuminance distribution data obtained when there are no dirt regions in each optical component in the preset optical imaging model;

[0109] Generate a reference imaging image based on the reference optical spectrum data;

[0110] If the reference imaging image satisfies a preset calibration condition, execute the step of obtaining multiple groups of optical spectrum data corresponding to each target surface in the preset optical imaging model, and determine the dirt specification of each target surface;

[0111] If the reference imaging image does not satisfy the preset calibration condition, determine that the preset optical imaging model is abnormal, and stop determining the dirt specification of the target surface.

[0112] It should be noted that the above modules may be implemented by software code or by hardware such as an integrated circuit chip.

[0113] It should also be noted that for the specific processes of the above modules to implement their respective functions, please refer to the specific content described in the above method embodiments, and details are not described herein again.

[0114] In a third aspect, based on the same inventive concept, an embodiment of the present application further provides an electronic device. For example, the electronic device may be a terminal device or a server. As Figure 9 shown, the electronic device includes a memory 904, one or more processors 902, and a computer program stored in the memory 904 and executable on the processor 902. When the processor 902 executes the program, it implements the steps of any one of the embodiments of the device dirt specification determining method provided in the first aspect above.

[0115] Among them, in Figure 9Among them, a bus architecture (represented by bus 900), bus 900 may include any number of interconnected buses and bridges. Bus 900 links together various circuits of one or more processors represented by processor 902 and a memory represented by memory 904. Bus 900 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and thus will not be further described herein. Bus interface 905 provides an interface between bus 900 and receiver 901 and transmitter 903. Receiver 901 and transmitter 903 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 902 is responsible for managing bus 900 and general processing, while memory 904 may be used to store data used by processor 902 when performing operations.

[0116] It can be understood that Figure 9 The structure shown is only schematic. The electronic device provided by the embodiments of the present invention may also include more or fewer components than those shown Figure 9 in, or have a different configuration from that shown Figure 9 in. Figure 9 Each component shown in can be implemented by hardware, software, or a combination thereof.

[0117] In a fourth aspect, based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, it implements the steps of any embodiment of the device dirt specification determination method provided in the first aspect above.

[0118] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0119] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocksFigure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. The term "plurality" means more than two, including two or more than two situations.

[0122] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0123] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a device dirt specification, characterized in that, For determining the dirt specifications on the surface of optical components in an imaging module, the method includes: Obtaining multiple sets of optical spectrum data corresponding to each target surface in a preset optical imaging model, where the preset optical imaging model is constructed based on the imaging module, and the multiple sets of optical spectrum data are image plane illuminance distribution data obtained after simulating dirt regions on the corresponding target surfaces according to a preset dirt specification sequence. The preset dirt specification sequence includes multiple reference dirt specifications arranged from small to large, and each set of optical spectrum data for the same target surface corresponds to one reference dirt specification; For each set of optical spectrum data of each target surface, generating an imaging image under the action of a dirt region with the corresponding reference dirt specification based on the optical spectrum data; For each target surface, determining the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets a preset qualified condition; Among them, each set of optical spectrum data corresponding to each target surface is generated according to the following steps: simulating dirt regions with the same reference dirt specification at multiple different semi-image height positions on the target surface; generating illuminance distribution data at the corresponding positions on the image plane under the action of the simulated dirt regions by performing ray tracing on the preset optical imaging model, and using the generated illuminance distribution data as a set of the optical spectrum data for the corresponding target surface.

2. The method according to claim 1, characterized in that The generating an imaging image under the action of a dirt region with the corresponding reference dirt specification based on the optical spectrum data includes: Converting the optical spectrum data into a picture containing original image information; Based on the pixel arrangement of the image sensor in the imaging module, performing color filtering and color analysis processing on the picture to obtain an imaging image under the action of a dirt region with the corresponding reference dirt specification.

3. The method according to claim 1, characterized in that, The sizes of the reference dirt specifications in the preset dirt specification sequence are arranged in equal gradients, where the minimum size is 10 μm, the maximum size is 100 μm, and the gradient is 10 μm.

4. The method according to claim 1, characterized in that, The detecting whether the imaging image meets a preset qualified condition includes: Detecting dirt point groups and bright point groups in the imaging image by comparing the brightness differences between each pixel point in the imaging image and the pixel points in a preset neighborhood. The dirt point groups include multiple adjacent dirt points, the bright point groups include multiple adjacent bright points, and the dirt points and the bright points are both abnormal pixel points caused by the influence of the dirt regions; Based on the detected number of dirt point groups, the number of bright point groups, the number of dirt points included in each dirt point group, and the number of bright points included in each bright point group, determining whether the imaging image meets a preset qualified condition.

5. The method according to claim 4, wherein The preset qualified condition includes: The number of dirt point groups is less than or equal to a first preset threshold, and the number of dirt points included in each dirt point group is less than or equal to a second preset threshold; and The number of bright point groups is less than or equal to a third preset threshold, and the number of bright points included in each bright point group is less than or equal to a fourth preset threshold.

6. The method according to claim 1, wherein Determining the dirt specification of the target surface from the preset dirt specification sequence includes: Arrange the reference dirt specifications in the preset dirt specification sequence in ascending order, and determine the dirt specification of the target surface as the reference dirt specification corresponding to the last one among them for which the corresponding imaging image meets the preset qualification conditions.

7. The method according to claim 1, characterized in that Before obtaining multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model, it further includes: Obtain the reference optical spectrum data of the preset optical imaging model, where the reference optical spectrum data is the image plane illuminance distribution data obtained when there are no dirt areas in each optical component in the preset optical imaging model; Generate a reference imaging image based on the reference optical spectrum data; If the reference imaging image meets the preset calibration conditions, then execute the step of obtaining multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model, and determine the dirt specification of each target surface; If the reference imaging image does not meet the preset calibration conditions, then determine that the preset optical imaging model is abnormal, and stop determining the dirt specification of the target surface.

8. A device dirt specification determination device, characterized in that Used to determine the dirt specification of the surface of the optical component in the camera module, the device includes: A data acquisition module, configured to acquire multiple sets of optical spectrum data corresponding to each target surface in the preset optical imaging model, where the preset optical imaging model is constructed according to the camera module, and the multiple sets of optical spectrum data are image plane illuminance distribution data obtained after simulating dirt areas on the corresponding target surfaces according to a preset dirt specification sequence, and the preset dirt specification sequence includes multiple reference dirt specifications arranged from small to large, and each set of optical spectrum data of the same target surface corresponds to a reference dirt specification; A generation module, configured to, for each set of optical spectrum data of each target surface, generate an imaging image under the action of a dirt area with the corresponding reference dirt specification based on the optical spectrum data; A determination module, configured to, for each target surface, determine the dirt specification of the target surface from the preset dirt specification sequence by detecting whether the imaging image meets the preset qualification conditions; Among them, the data acquisition module is specifically configured to: simulate dirt areas with the same reference dirt specification at multiple different semi-image height positions on the target surface; generate the illuminance distribution data at the corresponding positions on the image plane under the action of the simulated dirt areas by performing ray tracing on the preset optical imaging model, and use the generated illuminance distribution data as a set of the optical spectrum data of the corresponding target surface.

9. An electronic device, characterized in that, It includes: A processor, a memory, and a computer program stored on the memory, where when the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

Images