Defect optimization method and device for lens module

Abstract

The invention relates to a defect optimization method and device for a lens module, and belongs to the technical field of lens detection.The method comprises the steps that a to-be-detected lens module is obtained; wherein the lens module to be detected comprises N lenses; n is a positive integer; performing defect detection on the to-be-detected lens module; if the defects exist in the to-be-detected lens module, determining whether the defects are the defects at the same positions of two adjacent lenses in the to-be-detected lens module or not; if not, acquiring a target image after imaging based on the to-be-detected lens module; correcting the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used for removing imaging deviation caused by defects; carrying out qualification verification on the corrected image; and if the corrected image qualification verification is passed, determining that the to-be-detected lens module meets a good product requirement. According to the mode, the cost is reduced, meanwhile, the treatment process is saved, and the finished lens module does not need to be disassembled, replaced and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of lens detection, and in particular to a defect optimization method and device for a lens module. Background Art

[0002] In VR (Virtual Reality) or AR (Augmented Reality) devices, the lens module is a key optical component that directly affects the imaging quality of the device, the user's immersion and visual experience. The yield rate of the lens not only determines the overall production efficiency and cost control of the device, but also affects the market competitiveness of the product.

[0003] The lens module in a VR device or AR device is usually composed of multiple lenses with different transmittance or distortion, which together constitute the optical function of the module. The existing yield detection method is to detect and analyze a single lens. When a single lens does not meet the yield specifications, it is directly discarded. From the perspective of production cost, the direct abandonment method brings huge expenses. In addition, this method will also reduce production efficiency and even repeat the phenomenon of reinventing the wheel. After research, it was found that the function of the optical module is realized through multiple single lenses, and the defects of some single lenses will not cause the optical module as a whole to fail to meet the yield specifications. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, the present invention provides a defect optimization method and device for a lens module.

[0005] In a first aspect, an embodiment of the present application provides a defect optimization method for a lens module, comprising: obtaining a lens module to be detected; wherein the lens module to be detected includes N lenses; N is a positive integer; performing defect detection on the lens module to be detected; if there is a defect in the lens module to be detected, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be detected; if not, obtaining a target image after imaging based on the lens module to be detected; correcting the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used to remove imaging deviations caused by defects; performing qualification verification on the corrected image; if the qualification verification of the corrected image passes, determining that the lens module to be detected meets the requirements for good products.

[0006] Optionally, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected includes: calculating the first lens light beam exit angle based on the refractive index of the first lens, the first lens light beam incident angle, and the medium refractive index; calculating the displacement of the second lens light beam incident angle and the first lens light beam incident angle based on the first lens light beam exit angle and the placement distance between the first lens and the second lens; with the displacement as the radius, the position of the defect of the first lens is vertically projected to the position of the second lens to construct a determination circle; and determining whether the first lens and the second lens have defects at the same position based on whether the defect in the second lens is located in the determination circle; wherein the first lens and the second lens are any two adjacent lenses in the lens module to be inspected.

[0007] Optionally, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected includes: dividing each lens in the lens module to be inspected into four areas; and determining whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be inspected.

[0008] Optionally, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be detected includes: dividing each lens in the lens module to be detected into four areas; determining whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be detected; if there is a defect in the same area of ​​the adjacent first lens and the second lens, calculating the first lens light beam exit angle based on the refractive index of the first lens, the first lens light beam incident angle, and the medium refractive index; calculating the displacement of the second lens light beam incident angle and the first lens light beam incident angle based on the first lens light beam exit angle and the placement distance of the first lens and the second lens; with the displacement as the radius, the position of the defect of the first lens is vertically projected to the position of the second lens to construct a determination circle; and determining whether the first lens and the second lens have defects at the same position based on whether the defect in the second lens is located in the determination circle.

[0009] Optionally, the preset correction model is constructed by the following steps, including: constructing a point spread function library; wherein the point spread function library is a high-dimensional matrix, and the point spread function library includes multiple point spread functions; performing matrix decomposition on the point spread function to obtain a basis set B i ; Based on the input image and sample image, construct the basis set B i wherein the sample image is an image formed by a defect-free lens module; the degradation model characterizes the input image as a degraded image of the sample image; based on the degradation model, multiple components of the input image are determined and input into the U-net model supplementary basis set Bi The aberration caused by the encoder part is taken as the decomposed component each time, and a joint loss function is constructed to train the U-net model; wherein the trained U-net model is the preset correction model.

[0010] Optionally, the acquiring of a target image after imaging based on the lens module to be detected comprises: using a standard checkerboard as a test object, mounting the lens module to be detected on an imaging device, and acquiring a target image after imaging the test object based on the lens module to be detected; wherein the target image includes the standard checkerboard.

[0011] Optionally, the qualification verification of the corrected image includes: calculating the average spacing value between multiple grid points of a standard chessboard in the corrected image; if the difference between the average spacing value and the set standard spacing value is less than a first preset threshold, performing image clarity evaluation on the corrected image to obtain a quality score of the corrected image; if the quality score is higher than a second preset threshold, determining that the qualification verification of the corrected image has passed.

[0012] Optionally, performing image clarity assessment on the corrected image to obtain a quality score of the corrected image includes: inputting the corrected image into a deep neural network model, performing image clarity assessment on the corrected image based on the deep neural network model, and obtaining a quality score of the corrected image; wherein the deep neural network model includes a backbone network module, a gated local pooling module, a self-attention module, and a multilayer perceptron connected in sequence.

[0013] Optionally, the defect detection on the lens module to be detected includes: acquiring N lens images of the lens module to be detected; wherein each lens image includes a lens in the lens module to be detected; and using an EfficientViT-based target detection network to perform defect detection on the N lens images.

[0014] In a second aspect, an embodiment of the present application provides a defect optimization device for a lens module, comprising: an acquisition module, used to acquire a lens module to be detected; wherein the lens module to be detected includes N lenses; N is a positive integer; a defect detection module, used to perform defect detection on the lens module to be detected; if there is a defect in the lens module to be detected, determine whether the defect is a defect at the same position of two adjacent lenses in the lens module to be detected; an imaging module, used to, if not, obtain a target image after imaging based on the lens module to be detected; a correction module, used to correct the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used to remove imaging deviations caused by defects; a verification module, used to verify the conformity of the corrected image; if the corrected image passes the conformity line verification, it is determined that the lens module to be detected meets the requirements of good products.

[0015] The beneficial effects of the present invention include: since it is found in research that the function of the lens module in the VR device or AR device is realized by multiple single lenses, the defects of some single lenses will not cause the optical module as a whole to fail to meet the good product specifications. Based on the structural characteristics of the device, this application adopts the method of defect detection and position matching of the lens lens to analyze the good product potential of defective lenses (i.e. lens modules that can be corrected by algorithms, and the defects are not in the same position of adjacent lenses), correct the lens modules with good product potential, and judge whether they are good products based on their imaging performance. This method is different from the existing method in that the defective lenses are reused, that is, the defective imaging is compensated by the post-algorithm method. The algorithm cost is much lower than the cost of the lens itself. Compared with directly discarding and replacing the defective lenses, the cost is reduced, and the processing flow is also saved. There is no need to disassemble and replace the finished lens module. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of the steps of a defect optimization method for a lens module provided by an embodiment of the present invention;

[0017] Figure 2 A module block diagram of a defect optimization device for a lens module provided by an embodiment of the present invention;

[0018] Figure 3 A module block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0020] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0021] At present, the lens module in VR devices or AR devices is usually composed of multiple lenses. These lenses have different transmittances or distortions, and they together constitute the optical function of the module. The existing yield detection method is to detect and analyze a single lens. When a single lens does not meet the good specifications, it is directly discarded. From the perspective of production costs, the direct abandonment method brings huge expenses. In addition, this method will also reduce production efficiency and even repeat the phenomenon of reinventing the wheel. After research, it was found that the function of the optical module is realized through multiple single lenses, and the defects of some single lenses will not cause the optical module as a whole to fail to meet the good specifications.

[0022] In view of the above problems, the present application proposes the following embodiments to solve the above technical problems.

[0023] See also Figure 1 , an embodiment of the present application provides a defect optimization method for a lens module, including: steps 101 to 107.

[0024] Step 101: Obtain a lens module to be inspected.

[0025] The lens module to be tested includes N lenses, and N is a positive integer. The lens module to be tested is a produced lens module corresponding to a VR device or an AR device.

[0026] Step 102: Perform defect detection on the lens module to be inspected.

[0027] Here, it can be understood that defect detection is performed on each lens in the lens module to be inspected, and then it is determined whether there is a defect in each single lens.

[0028] If there is a defect in the lens module to be inspected, that is, as long as a defect is detected in the lens module to be inspected, step 103 is executed.

[0029] If no defects occur in the lens module to be inspected, it is determined that the lens module to be inspected meets the requirements of good products.

[0030] Step 103: Determine whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected.

[0031] That is, here the defect position is matched, and the purpose of defect position matching is to filter out lens matching pairs that have defects in adjacent areas at the same position. The reason is that in the lens module, a lens has a defect at a certain place while other lenses do not have defects. When this single-layer defect is not serious, research has found that the effect of the defect on imaging can be eliminated through the later image correction algorithm. However, when other lenses in the area also have defects, the imaging error at that place will double. Therefore, in the defect optimization method of the lens module proposed in the embodiment of the present application, this situation needs to be avoided. That is, the solution is mainly applicable to lens modules to be detected that have defects that are not in the same position.

[0032] Furthermore, when it is determined that the defect in the lens module to be inspected is not a defect at the same position of two adjacent lenses in the lens module to be inspected, step 104 is executed.

[0033] When it is determined that the defect in the lens module to be inspected is a defect at the same position of two adjacent lenses in the lens module to be inspected, two processes can be used for processing.

[0034] The first process is to replace any one of the two adjacent lenses with a corresponding lens without defects, and then continue to execute step 104.

[0035] The second process is to replace the two adjacent lenses with corresponding lenses without defects. After the two lenses are replaced, the lens module to be inspected can be determined to meet the requirements of good products.

[0036] Step 104: obtaining a target image after imaging based on the lens module to be detected; and correcting the target image based on a preset correction model to obtain a corrected image.

[0037] The corrected image is used to remove the imaging deviation caused by the defect. That is, the preset correction model is used here to use the algorithm to correct the impact of the defect on the imaging.

[0038] Step 105: Verify the corrected image for quality.

[0039] After the corrected image is obtained, the corrected image is verified for quality. If the corrected image passes the qualification verification, step 106 is performed. If the corrected image fails the qualification verification, it means that the defect cannot be corrected by the algorithm to achieve a good imaging effect, and a corresponding lens without defects is used for replacement.

[0040] Step 106: Determine whether the lens module to be inspected meets the requirements of good products.

[0041] In summary, the defect optimization method of a lens module provided in the embodiment of the present application has the following beneficial effects:

[0042] Since it is found in the research that the function of the lens module in the VR device or AR device is realized by multiple single lenses, the defects of some single lenses will not cause the optical module as a whole to fail to meet the specifications of good products. Based on the structural characteristics of the device, this application adopts the method of defect detection and position matching of lens lenses to analyze the good product potential of defective lenses (i.e. lens modules that can be corrected by algorithms, and the defects are not in the same position of adjacent lenses), correct the lens modules with good product potential, and judge whether they are good products based on their imaging performance. This method is different from the existing method in that the defective lenses are reused, that is, the defective imaging is compensated by the post-algorithm method. The algorithm cost is much lower than the cost of the lens itself. Compared with directly discarding and replacing defective lenses, the cost is reduced, and the processing flow is also saved. There is no need to disassemble and replace the finished lens module.

[0043] Optionally, defect detection is performed on the lens module to be inspected, including: acquiring N lens images of the lens module to be inspected; wherein each lens image includes a lens in the lens module to be inspected; and using an EfficientViT-based target detection network to perform defect detection on the N lens images.

[0044] That is, firstly, an image of the lens module is acquired, and then defect detection is performed using an object detection network based on EfficientViT. The reason why EfficientViT is selected in the embodiment of the present application is that the features of lens defects are not obvious in the image, and EfficientViT has multi-scale feature extraction capabilities, so it has better detection effects on weak targets.

[0045] Optionally, determining whether a defect is a defect at the same position of two adjacent lenses in a lens module to be detected includes: calculating the emission angle of the first lens beam based on the refractive index of the first lens, the incident angle of the first lens beam, and the refractive index of the medium; calculating the displacement of the incident angle of the second lens beam and the incident angle of the first lens beam based on the emission angle of the first lens beam and the placement distance between the first lens and the second lens; constructing a determination circle by vertically projecting the position of the defect of the first lens to the position of the second lens with the displacement as the radius; and determining whether the first lens and the second lens have defects at the same position based on whether the defect in the second lens is located in the determination circle; wherein the first lens and the second lens are any two adjacent lenses in the lens module to be detected.

[0046] Specifically, the purpose of the above steps is to filter out lens matching pairs that have defects in adjacent areas at the same position. The reason is that in the lens module, a lens has a defect at a certain place while other lenses do not have defects. This single-layer defect can be eliminated through the subsequent image correction algorithm to eliminate the impact of the defect on imaging when the defect is not serious. However, when other lenses in the area also have defects, the imaging error at that place will be doubled, so this situation should be avoided.

[0047] The matching logic is that, given the refractive index of the lens installed at each position in the lens module and the distance between each lens, the detection result of the defect detection module is obtained, and Snell's law can be used to obtain:

[0048] n1sin(θ i )=n2sin(θ r );

[0049] Where n1 is the refractive index of the first lens, θ i is the incident angle of the first lens beam, n2 is the refractive index of the medium, θ r is the exit angle of the first lens, when θ i When θ is known, r Then, according to θ r We can get:

[0050] Δy=d·tan(θ r );

[0051] Where d is the placement distance of the first lens relative to the second lens (known). At this time, Δy is the displacement of the incident light of the second lens compared to the incident light of the first lens. Finally, a judgment circle is constructed with the displacement as the radius and the position of the defect of the first lens vertically projected to the position of the second lens as the center. If there is a defect in the judgment circle, it is considered that the second lens and the first lens have a defect match, and both have defects at the same position. The pair of lenses cannot appear in the same lens module at the same time. If there is no defect in the judgment circle, it is considered that there is no defect match between the second lens and the first lens.

[0052] It should be noted that, in this application, by constructing a judgment circle and projecting and judging the defect position, the defect correlation judgment process is simplified, the complex manual or point-by-point analysis work is reduced, and the detection efficiency is improved. That is, through calculation and geometric methods, an efficient and accurate defect positioning and judgment method is provided, which helps to improve the detection accuracy of the optical module.

[0053] Optionally, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected includes: dividing each lens in the lens module to be inspected into four areas; and determining whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be inspected.

[0054] It should be noted that in this embodiment, each lens is divided into four regions, such as the four quadrants in the corresponding coordinate system, so that the process of determining the position of defects of adjacent lenses only requires determining whether they are in the same region. This method reduces complex calculation steps and improves detection efficiency. In other words, the regional division provides a clear spatial reference for fast and accurate positioning judgment.

[0055] Optionally, determining whether a defect is a defect at the same position of two adjacent lenses in a lens module to be detected includes: dividing each lens in the lens module to be detected into four areas; determining whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be detected; if there is a defect in the same area of ​​the adjacent first lens and the second lens, calculating the first lens light beam exit angle based on the refractive index of the first lens, the first lens light beam incident angle, and the medium refractive index; calculating the displacement of the second lens light beam incident angle and the first lens light beam incident angle based on the first lens light beam exit angle and the placement distance between the first lens and the second lens; with the displacement as the radius, the position of the defect of the first lens is vertically projected to the position of the second lens to construct a determination circle; and determining whether the first lens and the second lens have defects at the same position based on whether the defect in the second lens is located in the determination circle.

[0056] That is, in this embodiment, regional division and geometric projection judgment are combined. First, preliminary screening is performed through regional division, and then secondary matching is performed in combination with geometric projection. This method can save certain process steps while ensuring accuracy. In other words, compared with only regional division, the judgment accuracy is improved, and compared with only geometric projection judgment, the process is saved.

[0057] Optionally, obtaining a target image after imaging based on the lens module to be detected includes: using a standard checkerboard as a test object, installing the lens module to be detected on an imaging device, and obtaining a target image after imaging the test object based on the lens module to be detected; wherein the target image includes a standard checkerboard.

[0058] Optionally, the preset correction model is constructed by the following steps, including: constructing a point spread function library; wherein the point spread function library is a high-dimensional matrix, and the point spread function library includes multiple point spread functions; performing matrix decomposition on the point spread function to obtain a basis set B i ; Based on the input image and sample image, construct the basis set B i The degradation model is a degradation model; wherein the sample image is an image formed by imaging through a defect-free lens module; the degradation model characterizes the input image as a degraded image of the sample image; based on the degradation model, multiple components of the input image are determined and input into the U-net model supplementary basis set B iThe aberration caused by the encoder part is taken as the decomposed component each time, and a joint loss function is constructed to train the U-net model; wherein the trained U-net model is the preset correction model.

[0059] First, common point spread functions are constructed as a point spread function library (a high-dimensional matrix), such as Gaussian point spread function, annular point spread function, etc., and then matrix decomposition is performed to obtain a basis set {B i}, this basis set can be used to obtain any point spread function through a finite number of linear combinations:

[0060] k=∑ i α i ·B i ;

[0061] Where k is the point spread function obtained by combining the basis sets; α i Represents the weight coefficient.

[0062] Take the image input image Y as the degraded image of the potential high-quality image X and build a degradation model:

[0063]

[0064] in, is 2D convolution, n is white noise;

[0065] Assume that Y can be decomposed into multiple component, using the pre-trained U-net model to compensate for B i The aberration caused by the encoder is used as the decomposed component to construct the joint loss function.

[0066]

[0067] X is obtainable by simply using a lens module composed of completely defect-free lenses for image acquisition;

[0068] Finally, the corrected image Y can be output * (Output of U-net model).

[0069] Optionally, the corrected image is verified for quality, including: calculating the average spacing value between multiple grid points of a standard chessboard in the corrected image; if the difference between the average spacing value and the set standard spacing value is less than a first preset threshold, performing an image clarity evaluation on the corrected image to obtain a quality score of the corrected image; if the quality score is higher than a second preset threshold, determining that the corrected image has passed the qualification verification.

[0070] That is, the quality of the corrected image is evaluated, and the evaluation is divided into two stages. The first stage is to calculate the average spacing between the chessboard points of the image. If the average value is different from the average spacing value of the set standard spacing value within a certain threshold, the lens module is considered to be preliminarily qualified and the second stage evaluation is carried out. The second stage evaluation can be carried out through an image clarity evaluation network.

[0071] In the embodiment of the present application, the dual indicators (spacing and clarity) are used to make the judgment criteria more comprehensive, which not only ensures the accuracy of the corrected image, but also ensures that its clarity reaches a qualified level, thereby improving the reliability of the results.

[0072] Optionally, performing image clarity evaluation on the corrected image to obtain a quality score of the corrected image includes: inputting the corrected image into a deep neural network model, performing image clarity evaluation on the corrected image based on the deep neural network model, and obtaining a quality score of the corrected image; wherein the deep neural network model includes a backbone network module, a gated local pooling module, a self-attention module, and a multi-layer perceptron connected in sequence.

[0073] That is, the corrected image is input into the backbone network to extract the multi-scale features of the image, and then the multi-scale features are reduced to the same spatial size using gated local pooling, and enhanced using self-attention. Finally, the semantically perceived distortion features are aggregated through a multi-layer perceptron and regressed to a quality score. If the quality score meets a certain threshold, the lens module is considered to meet the requirements of good quality.

[0074] See also Figure 2 Based on the same inventive concept, the embodiment of the present application provides a defect optimization device 200 for a lens module, comprising: an acquisition module 201, used to acquire a lens module to be detected; wherein the lens module to be detected includes N lenses; N is a positive integer; a defect detection module 202, used to perform defect detection on the lens module to be detected; if there is a defect in the lens module to be detected, determine whether the defect is a defect at the same position of two adjacent lenses in the lens module to be detected; an imaging module 203, used to obtain a target image after imaging based on the lens module to be detected if not; a correction module 204, used to correct the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used to remove the imaging deviation caused by the defect; a verification module 205, used to verify the conformity of the corrected image; if the corrected image passes the conformity line verification, it is determined that the lens module to be detected meets the good product requirements.

[0075] See also Figure 3 Based on the same inventive concept, the present application embodiment provides a module frame of an electronic device 300 applying the above method. The electronic device 300 includes: at least one processor 301 ( Figure 3 Only one is shown), a memory 302, a computer program 303 stored in the memory 302 and executable on at least one processor 301, and when the processor 301 executes the computer program 303, the steps of the method in any of the foregoing embodiments are implemented.

[0076] Those skilled in the art will understand that Figure 3 The electronic device 300 is merely an example and does not constitute a limitation on the electronic device 300 , and may include more or less components than those shown in the figure, or a combination of certain components, or different components.

[0077] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0078] In some embodiments, the memory 302 may be an internal storage unit of the electronic device 300, such as a hard disk or memory of the electronic device 300. In other embodiments, the memory 302 may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 400. Further, the memory 302 may also include both an internal storage unit of the electronic device 300 and an external storage device.

[0079] It should be noted that the above-mentioned systems, devices, etc. are based on the same concept as the method embodiments of the present application. The modules designed for the systems, the steps performed by the devices, and the technical effects brought about can all be found in the method embodiment section and will not be repeated here.

[0080] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0081] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0082] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0084] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0086] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A defect optimization method for a lens module, characterized in that: include: Obtain a lens module to be detected; wherein the lens module to be detected includes N lenses; N is a positive integer; Performing defect detection on the lens module to be inspected; If there is a defect in the lens module to be inspected, determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected; If not, obtaining a target image after imaging based on the lens module to be detected; Correcting the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used to remove imaging deviations caused by defects; Performing conformity verification on the corrected image; If the corrected image passes the qualification verification, it is determined that the lens module to be inspected meets the requirements of good products.

2. The defect optimization method of the lens module according to claim 1, characterized in that: The step of determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected comprises: Calculating the light beam exit angle of the first lens based on the refractive index of the first lens, the light beam incident angle of the first lens, and the refractive index of the medium; Based on the light beam exit angle of the first lens and the placement distance between the first lens and the second lens, calculate the displacement between the light beam incident angle of the second lens and the light beam incident angle of the first lens; Taking the displacement as a radius, the position of the defect of the first lens is vertically projected to the position of the second lens to construct a determination circle; determining whether the first lens and the second lens have defects at the same position according to whether the defect in the second lens is located in the determination circle; Wherein, the first lens and the second lens are any two adjacent lenses in the lens module to be detected.

3. The defect optimization method of the lens module according to claim 1, characterized in that: The step of determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected comprises: Divide each lens in the lens module to be inspected into four areas; Determine whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be inspected.

4. The defect optimization method of the lens module according to claim 1, characterized in that: The step of determining whether the defect is a defect at the same position of two adjacent lenses in the lens module to be inspected comprises: Divide each lens in the lens module to be inspected into four areas; Determine whether there is a defect in the same area of ​​two adjacent lenses in the lens module to be inspected; If there is a defect in the same area of ​​the adjacent first lens and the second lens, the first lens light beam exit angle is calculated based on the refractive index of the first lens, the first lens light beam incident angle, and the medium refractive index; Based on the emission angle of the light beam of the first lens and the placement distance between the first lens and the second lens, calculating the displacement of the incident angle of the light beam of the second lens and the incident angle of the light beam of the first lens; Taking the displacement as a radius, the position of the defect of the first lens is vertically projected to the position of the second lens to construct a determination circle; According to whether the defect in the second lens is located in the determination circle, it is determined whether the first lens and the second lens have defects at the same position.

5. The defect optimization method of the lens module according to claim 1, characterized in that: The preset correction model is constructed by the following steps, including: Constructing a point spread function library; wherein the point spread function library is a high-dimensional matrix, and the point spread function library includes a plurality of point spread functions; Perform matrix decomposition on the point spread function to obtain the basis set B i ; Based on the input image and the sample image, we construct a basis set B i wherein the sample image is an image formed by imaging through a defect-free lens module; the degradation model characterizes the input image as a degraded image of the sample image; Based on the degradation model, multiple components of the input image are determined and input into the U-net model supplementary basis set B i The aberration caused by the encoder is taken as the decomposed component each time, and a joint loss function is constructed to train the U-net model; Among them, the trained U-net model is the preset correction model.

6. The defect optimization method of the lens module according to claim 1, characterized in that: The step of acquiring a target image after imaging based on the lens module to be detected comprises: Using a standard chessboard as a test object, installing the lens module to be detected on an imaging device, and acquiring a target image after imaging the test object based on the lens module to be detected; Wherein, the target image includes the standard chessboard.

7. The defect optimization method of the lens module according to claim 6, characterized in that: The performing qualification verification on the corrected image comprises: Calculating the average spacing value between multiple grid points of the standard chessboard in the corrected image; If the difference between the average spacing value and the set standard spacing value is less than a first preset threshold, performing image clarity evaluation on the corrected image to obtain a quality score of the corrected image; If the quality score is higher than a second preset threshold, it is determined that the corrected image passes the qualification verification.

8. The defect optimization method of the lens module according to claim 7, characterized in that: The performing image clarity evaluation on the corrected image to obtain a quality score of the corrected image includes: Inputting the corrected image into a deep neural network model, performing image clarity evaluation on the corrected image based on the deep neural network model, and obtaining a quality score of the corrected image; The deep neural network model includes a backbone network module, a gated local pooling module, a self-attention module and a multi-layer perceptron connected in sequence.

9. The defect optimization method of the lens module according to claim 1, characterized in that: The defect detection of the lens module to be detected includes: Acquire N lens images of the lens module to be detected; wherein each lens image includes one lens in the lens module to be detected; An object detection network based on EfficientViT is used to perform defect detection on the N lens images.

10. A defect optimization device for a lens module, characterized in that: include: An acquisition module is used to acquire a lens module to be detected; wherein the lens module to be detected includes N lenses; N is a positive integer; A defect detection module is used to perform defect detection on the lens module to be detected; if there is a defect in the lens module to be detected, determine whether the defect is a defect at the same position of two adjacent lenses in the lens module to be detected; An imaging module, for obtaining a target image after imaging based on the lens module to be detected if no; A correction module, used to correct the target image based on a preset correction model to obtain a corrected image; wherein the corrected image is used to remove imaging deviations caused by defects; The verification module is used to verify the conformity of the corrected image; if the corrected image passes the conformity line verification, it is determined that the lens module to be inspected meets the requirements of good products.

Images