A non-planar lens defect detection method and system

Abstract

The present invention relates to a non-planar lens defect detection method and system, and belongs to the field of lens detection technology. The method comprises: performing contour detection on a first image, and determining a contour-matched complement lens set from a lens library based on the contour detection result of the non-planar lens to be detected; performing refractive index simulation calculation based on the complement lens set and the non-planar lens to be detected, and determining a first target lens and a second target lens for complementing the non-planar lens to be detected; obtaining an equivalent lens image collected by using the complemented target lens as a lens lens; inputting the second image into a defect detection model to obtain the defect detection result of the complemented target lens; the defect detection result of the complemented target lens represents the defect detection result of the non-planar lens to be detected. The present application provides a defect detection method for non-planar lenses, which can be applied to non-planar lenses of various curvatures and has a high defect detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of lens detection, and in particular to a method and system for detecting defects of a non-planar lens. Background Art

[0002] Non-planar lenses are widely used in optical systems, such as cameras, microscopes, and lasers. Their unique curved surface design can effectively improve imaging quality and optical performance. However, non-planar lenses are prone to various defects during manufacturing and use, such as bubbles, scratches, and optical distortion. These defects not only affect the optical properties of the lens, but may also cause the performance of the entire optical system to deteriorate or even fail. Therefore, it is necessary to detect and repair defects in non-planar lenses in a timely manner to improve the stability and reliability of the optical system and ensure its effectiveness in high-precision applications.

[0003] However, research has found that most of the existing defect detection methods for lenses are for plane lenses. For example, a defect detection method proposes to obtain defect characteristics generated under various conditions by simulating the heating and cooling, pressure changes, and material flow characteristics of the material during the lens production process, and build a defect database. Finally, the lens image features are extracted and compared with the defect database features to obtain the detection results. The above method can achieve good results under the premise that the database is large enough, but in actual production, companies usually produce a variety of lenses, including non-plane mirrors. Non-plane mirrors have curved or irregular surfaces compared to plane mirrors, which makes the propagation path of light inside the lens complicated. Therefore, when defects are detected through lens images, the same defect will present different image features under lenses with different curvatures, which increases the difficulty of detection. It also means that if existing methods want to achieve better detection results, a huge database needs to be built. In terms of both cost and efficiency, existing methods cannot meet the detection needs of enterprises. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, the present invention provides a method and system for detecting defects in a non-planar lens.

[0005] In a first aspect, the present application provides a non-planar lens defect detection method, comprising: acquiring a first image; wherein the first image is an image obtained by photographing a non-planar lens to be detected; performing contour detection on the first image, and determining a contour-matched complement lens set from a lens library based on the contour detection result of the non-planar lens to be detected; performing refractive index simulation calculation based on the complement lens set and the non-planar lens to be detected, and determining a first target lens and a second target lens for complementing the non-planar lens to be detected; wherein, when the first target lens and the second target lens are used to complement the non-planar lens to be detected, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected; the first target lens and the second target lens are two non-planar lenses in the complement lens set; acquiring a second image; wherein the second image is an equivalent lens image collected by using the complemented target lens as a lens lens; inputting the second image into a defect detection model to obtain a defect detection result of the complemented target lens; wherein the defect detection result of the complemented target lens represents the defect detection result of the non-planar lens to be detected.

[0006] Optionally, the refractive index simulation calculation of the lenses in the complementary lens set and the non-planar lens to be detected is performed through the following steps, including: obtaining the transmission matrix of the first lens, the transmission matrix of the second lens and the transmission matrix of the non-planar lens to be detected; wherein the first lens and the second lens are both lenses in the complementary lens set; based on the transmission matrix of the first lens, the transmission matrix of the second lens and the transmission matrix of the non-planar lens to be detected, obtaining the equivalent transmission matrix of the complementary lens; and obtaining the equivalent refractive index of the complementary lens through the equivalent transmission matrix of the complementary lens.

[0007] Optionally, performing contour detection on the first image and determining a complementary lens set with contour matching from a lens library based on the contour detection result of the non-planar lens to be detected comprises: performing contour detection on the first image based on a contour detection algorithm of a Canny operator to obtain a contour detection result of the non-planar lens to be detected; wherein the contour detection result of the non-planar lens to be detected comprises an identification contour of a first surface of the non-planar lens to be detected and an identification contour of a second surface of the non-planar lens to be detected; the first surface of the non-planar lens to be detected and the second surface of the non-planar lens to be detected are opposite surfaces; determining a first complementary lens set matching the identification contour of the first surface of the non-planar lens to be detected from the lens library, and determining a second complementary lens set matching the identification contour of the second surface of the non-planar lens to be detected from the lens library; wherein the complementary lens set comprises the first complementary lens set and the second complementary lens set.

[0008] Optionally, the defect detection model includes: a feature enhancement sub-model, a feature extraction sub-model and a detection sub-model; the feature enhancement sub-model, the feature extraction sub-model and the detection sub-model are connected in sequence; the feature enhancement sub-model is used to enhance the features on the second image; the feature extraction sub-model is used to extract features from the feature-enhanced image; the detection sub-model is used to identify defects based on the extracted features and output defect detection results.

[0009] Optionally, inputting the second image into the defect detection model to obtain the defect detection result of the completed target lens includes: inputting the second image into the feature enhancement sub-model, down-sampling the second image through a Gaussian pyramid; wherein the Gaussian pyramid has N layers; N is a positive integer; up-sampling the down-sampled image through a Laplacian pyramid to obtain an up-sampled image; wherein the Laplacian pyramid also has N layers; performing image fusion on the up-sampled image to obtain the feature-enhanced image; inputting the feature-enhanced image into the feature extraction sub-model, and outputting the defect detection result through the detection sub-model.

[0010] Optionally, the Gaussian pyramid has four layers; the Laplacian pyramid has four layers; and the number of up-sampled images is four.

[0011] Optionally, the feature extraction sub-model includes: a deep residual network and a feature pyramid network; the feature maps output by the 3rd, 4th and 5th layers of the deep residual network undergo another deep separable convolution and then enter the first three separable convolution layers of the feature pyramid network through upsampling fusion.

[0012] Optionally, the detection sub-model adopts a YoloX-based target detection model; and the detection head of the detection sub-model includes deformable convolution.

[0013] Optionally, the defect detection result includes a detection frame with a defect detection type; wherein, the defect position of the non-planar lens to be detected is determined by the position of the detection frame on the target lens after the padding.

[0014] In a second aspect, the present application provides a non-planar lens defect detection system, comprising: a first acquisition module, used to acquire a first image; wherein the first image is an image obtained by photographing the non-planar lens to be detected; a contour detection module, used to perform contour detection on the first image, and based on the contour detection result of the non-planar lens to be detected, determine a set of complementary lenses with contour matching from a lens library; a complementary lens determination module, used to perform refractive index simulation calculation based on the complementary lens set and the non-planar lens to be detected, and determine a first target lens and a second target lens for complementing the non-planar lens to be detected; wherein, when the first target lens and the second target lens are used After the two target lenses complement the non-planar lens to be detected, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected; the first target lens and the second target lens are two non-planar lenses in the complementing lens set; a second acquisition module is used to acquire a second image; wherein the second image is an equivalent lens image collected by using the complemented target lens as a lens; a defect detection module is used to input the second image into a defect detection model to acquire the defect detection result of the complemented target lens; wherein the defect detection result of the complemented target lens represents the defect detection result of the non-planar lens to be detected.

[0015] The beneficial effects of the present invention include: the present application takes into account that a non-planar lens may be distorted in imaging due to a difference in refractive index from the medium, so the aforementioned steps modify the non-planar lens to be detected, and fill in the gaps through the first target lens and the second target lens, so that the incident light of the filled in target lens is parallel to the outgoing light, so as to obtain a clear equivalent lens image through the filled in target lens. It should be noted that since the filled in lenses are all defect-free lenses, and the incident light of the filled in target lens is parallel to the outgoing light, the final defect detection result of the equivalent lens image can directly characterize the defect detection result of the non-planar lens to be detected. In other words, the defect detection position and type of the filled in target lens corresponding to the equivalent lens image correspond to the defect detection position and type of the non-planar lens to be detected. It can be seen that the present application provides a defect detection method for non-planar lenses, which can be applied to non-planar lenses of various curvatures and has a high defect detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of the steps of a non-planar lens defect detection method provided by an embodiment of the present invention;

[0017] Figure 2 A flowchart of another non-planar lens defect detection method provided by an embodiment of the present invention;

[0018] Figure 3 A module block diagram of a defect detection model provided by an embodiment of the present invention;

[0019] Figure 4 A module block diagram of another defect detection model provided by an embodiment of the present invention;

[0020] Figure 5 A module block diagram of a non-planar lens defect detection system provided by an embodiment of the present invention;

[0021] Figure 6 A module block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0023] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0024] Research has found that most of the existing defect detection methods for lenses are for plane lenses. For example, a defect detection method proposes to obtain defect characteristics generated under various conditions by simulating the heating and cooling, pressure changes, and material flow characteristics of the material during the lens production process, and build a defect database. Finally, the lens image features are extracted and compared with the defect database features to obtain the detection results. The above method can achieve good results under the premise that the database is large enough, but in actual production, companies usually produce a variety of lenses, including non-plane mirrors. Non-plane mirrors have curved or irregular surfaces compared to plane mirrors, which makes the propagation path of light inside the lens complicated. Therefore, when defects are detected through lens images, the same defect will present different image characteristics under lenses with different curvatures, which increases the difficulty of detection. It also means that if existing methods want to achieve better detection results, a huge database needs to be built. In terms of both cost and efficiency, existing methods cannot meet the detection needs of enterprises.

[0025] In view of the above problems, the present application proposes the following embodiments to solve the above technical problems.

[0026] See also Figure 1, an embodiment of the present application provides a non-planar lens defect detection method, including: steps 101 to 105.

[0027] Step 101: Acquire a first image.

[0028] The first image is an image obtained by photographing the non-planar lens to be inspected.

[0029] Exemplarily, the non-planar lens to be inspected may be placed on a detection platform, and then the non-planar lens to be inspected may be photographed to obtain a first image.

[0030] It should be noted that the first image needs to be a clear image so that the contour of the non-planar lens to be detected can be accurately identified later. When shooting, lighting beads can be added to the detection table to ensure sufficient light.

[0031] Step 102: Perform contour detection on the first image, and determine a set of complementary lenses with contour matching from a lens library based on the contour detection result of the non-planar lens to be detected.

[0032] After acquiring the first image, the target (non-planar lens to be detected) is detected, and contour recognition detection is performed on the target to obtain the contour detection result of the target. Then, a primary screening is performed from the lens library to screen out all the complementary lenses that match the contour detection result of the non-planar lens to be detected, thereby forming a complementary lens set.

[0033] It should be noted that the lens library is a pre-prepared set of defect-free lenses with various curvatures.

[0034] Step 103: Perform refractive index simulation calculation based on the complement lens set and the non-planar lens to be detected, and determine the first target lens and the second target lens for complementing the non-planar lens to be detected.

[0035] When the non-planar lens to be detected is complemented by the first target lens and the second target lens, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected.

[0036] That is, in this step, each lens in the complementary lens set is screened twice, and then the first target lens and the second target lens that meet the above screening requirements are determined. That is, the first target lens and the second target lens are two non-planar lenses in the complementary lens set.

[0037] In other words, the screening requirement is to determine two lenses from the complement lens set so that after complementing the non-planar lens to be detected, their overall equivalent refractive index is equal to the refractive index of the non-planar lens to be detected. In this way, the incident light and the outgoing light can be made parallel to avoid phase changes when the light passes through the complemented target lens, which affects the imaging position and clarity, and thus leads to inconsistent imaging quality.

[0038] Step 104: Acquire a second image.

[0039] Among them, the second image is an equivalent lens image collected by using the filled target lens as the lens.

[0040] Step 105: Input the second image into the defect detection model to obtain the defect detection result of the target lens after the completion.

[0041] The defect detection result of the filled target lens represents the defect detection result of the non-planar lens to be detected.

[0042] It should be noted that the present application takes into account that the non-planar lens may be distorted in imaging due to the difference in refractive index from the medium, so the aforementioned steps modify the non-planar lens to be detected, and fill it in through the first target lens and the second target lens, so that the incident light of the filled target lens is parallel to the outgoing light, so as to obtain a clear equivalent lens image through the filled target lens. It should be noted that since the filled lenses are all defect-free lenses, and the incident light of the filled target lens is parallel to the outgoing light, the final defect detection result of the equivalent lens image can directly characterize the defect detection result of the non-planar lens to be detected. In other words, the defect detection position and type of the filled target lens corresponding to the equivalent lens image correspond to the defect detection position and type of the non-planar lens to be detected. It can be seen that the present application provides a defect detection method for non-planar lenses, which can be applied to non-planar lenses of various curvatures and has a high defect detection accuracy.

[0043] See also Figure 2 Optionally, refractive index simulation calculation is performed on the lenses in the complementary lens set and the non-planar lens to be detected through the following steps, including: step 201 to step 203.

[0044] Step 201: Obtain the transmission matrix of the first lens, the transmission matrix of the second lens, and the transmission matrix of the non-planar lens to be detected.

[0045] The first lens and the second lens are both lenses in the set of complementary lenses.

[0046] The calculation formula of the transfer matrix of each lens is:

[0047] ; ; ;

[0048] in, represents the transfer matrix of the first lens, represents the focal length of the first lens; represents the transfer matrix of the second lens, Indicates the focal length of the second lens; represents the transmission matrix of the non-planar lens to be detected, Indicates the focal length of the non-planar lens to be tested.

[0049] Step 202: Based on the transmission matrix of the first lens, the transmission matrix of the second lens, and the transmission matrix of the non-planar lens to be detected, an equivalent transmission matrix of the completed lens is obtained.

[0050] Among them, the calculation formula of the equivalent transfer matrix of the completed lens is:

[0051] ;

[0052] in, represents the equivalent transfer matrix of the completed lens; , which represents the distance matrix between the second lens and the non-planar lens to be detected, represents the distance (spacing) between the second lens and the non-planar lens to be detected determined by the placement position; , which represents the distance matrix between the first lens and the non-planar lens to be detected, It indicates the distance (spacing) between the first lens and the non-planar lens to be detected, which is determined by the placement position.

[0053] Then, the equivalent transfer matrix of the completed lens is transformed and calculated step by step using matrix multiplication: .

[0054] in, , , , are all matrix elements of the equivalent transfer matrix calculated by matrix multiplication, and are all associated with the focal length and / or spacing of the lens.

[0055] Step 203: Obtain the equivalent refractive index of the completed lens through the equivalent transmission matrix of the completed lens.

[0056] The equivalent focal length of the entire system (i.e. the completed lens) In the embodiment of the present application, the elements in the equivalent transfer matrix of the completed lens are It can be calculated in, the specific calculation formula is: .

[0057] The calculation formula of the equivalent refractive index of the completed lens is: ;

[0058] in, represents the mean curvature; Represents the equivalent refractive index of the filled lens.

[0059] It should be noted that, by using the transmission matrices of the first lens, the second lens and the non-planar lens to be detected, the overall equivalent transmission matrix of the combined lens system is calculated, so that the optical behavior of the entire lens group can be described more accurately. Moreover, the equivalent refractive index of the entire lens group can be simulated more accurately through the above formula.

[0060] Optionally, in the above steps, contour detection is performed on the first image, and based on the contour detection result of the non-planar lens to be detected, a complementary lens set with contour matching is determined from a lens library, including: based on a contour detection algorithm of a Canny operator, contour detection is performed on the first image to obtain a contour detection result of the non-planar lens to be detected; wherein the contour detection result of the non-planar lens to be detected includes an identification contour of a first surface of the non-planar lens to be detected and an identification contour of a second surface of the non-planar lens to be detected; the first surface of the non-planar lens to be detected and the second surface of the non-planar lens to be detected are opposite surfaces; a first complementary lens set matching the identification contour of the first surface of the non-planar lens to be detected is determined from the lens library, and a second complementary lens set matching the identification contour of the second surface of the non-planar lens to be detected is determined from the lens library; wherein the complementary lens set includes the first complementary lens set and the second complementary lens set.

[0061] It should be noted that the contour detection algorithm of the Canny operator can accurately extract the edge contours of the first and second surfaces of the non-planar lens to be detected, reducing the interference of image noise. This operation ensures that the contour features of the two opposite surfaces of the lens are accurately captured, thereby providing reliable basic data for subsequent matching. In addition, the corresponding complementary lens set is found based on the contour matching of the first and second surfaces, and optical property compensation can be performed on both surfaces. The use of this matching method can effectively improve its imaging performance in the optical path and reduce the distortion caused by the non-planar characteristics of the lens.

[0062] See also Figure 3 Optionally, a defect detection model 300 provided in an embodiment of the present application specifically includes: a feature enhancement sub-model 301, a feature extraction sub-model 302 and a detection sub-model 303.

[0063] The feature enhancement sub-model 301, the feature extraction sub-model 302 and the detection sub-model 303 are connected in sequence.

[0064] The feature enhancement sub-model 301 is used to enhance the features on the second image; the feature extraction sub-model 302 is used to extract features from the image after feature enhancement; the detection sub-model 303 is used to identify defects based on the extracted features and output defect detection results.

[0065] Optionally, the above step inputs the second image into the defect detection model to obtain the defect detection result of the completed target lens, including: inputting the second image into the feature enhancement sub-model, down-sampling the second image through a Gaussian pyramid; wherein the Gaussian pyramid has N layers; N is a positive integer; up-sampling the down-sampled image through a Laplacian pyramid to obtain an up-sampled image; wherein the Laplacian pyramid also has N layers; performing image fusion on the up-sampled image to obtain a feature-enhanced image; inputting the feature-enhanced image into the feature extraction sub-model, and outputting the defect detection result through the detection sub-model.

[0066] Optionally, the Gaussian pyramid has four layers; the Laplacian pyramid has four layers; and the number of upsampled images is four.

[0067] It should be noted that, considering that the feature performance of the equivalent lens image on the defect is still not obvious enough, it is necessary to enhance the weak defect features through the feature enhancement sub-model. Specifically: first, downsample the second image through the Gaussian pyramid, which can be referred to the following formula:

[0068] ;

[0069] in, Indicates Layer downsampling layer; Indicates Layer downsampling layer; represents a Gaussian low-pass filter, and Represents the index inside the Gaussian low-pass filter. The Gaussian low-pass filter can be a 4*4 block. and Represents the coordinate index of the image, In the above embodiment, the Gaussian pyramid has four layers, namely , , , , the second input image is .

[0070] Then, upsampling can be performed through the Laplacian pyramid. The upsampling formula is as follows: ; In this formula, Indicates Layer Laplacian upsampling layer; In this embodiment, there are four Laplace upsampling layers, namely , , , .

[0071] Finally, the upsampled images are fused, and the reference formula is: ;in, correspond The fused image of the layer, correspond The fused image of the layer, represents the upsampling function, which is implemented by quadratic linear interpolation. The final feature-enhanced image is obtained after four fusions.

[0072] Of course, in other embodiments, the number N may also be other values, which can be set according to actual conditions and are not limited here.

[0073] See also Figure 4 Optionally, the feature extraction sub-model 302 includes: a deep residual network and a feature pyramid network; the feature maps output by the 3rd layer (convolution layer 3), the 4th layer (convolution layer 4) and the 5th layer (convolution layer 5) of the deep residual network undergo one more deep separable convolution and then enter the first three separable convolution layers (separable convolution 1, separable convolution 2 and separable convolution 3) of the feature pyramid network through upsampling fusion.

[0074] In an embodiment of the present application, the deep residual network may be specifically ResNet18, and the feature pyramid network may be specifically FPN (Feature Pyramid Network).

[0075] That is, the feature maps of the 3rd, 4th, and 5th layers in ResNet18 are output and passed through a depthwise separable convolution respectively, and then fed into the first three separable convolution layers of FPN after upsampling fusion. It should be noted that the feature maps of the 3rd, 4th, and 5th layers of ResNet18 are extracted separately and passed through a depthwise separable convolution. Depthwise separable convolution is an efficient convolution method that can reduce the amount of calculation while retaining feature information, which helps to reduce computational overhead and improve efficiency. It can highlight the locality of feature information and play a role in enhancing feature details. After processing the features of different layers of ResNet18, they are fused through upsampling. The role of upsampling is to increase the scale of the feature maps of the deeper layers (low resolution) and align them with the feature maps of the shallower layers. This fusion of multi-level features enables the network to consider both high-level semantic information and low-level detail information at the same time, improving the integrity of the features. FPN further processes the fused feature maps through a pyramid structure, refines and extracts them layer by layer, and finally obtains multi-scale feature representations. Sending multi-layer fused feature maps into the first three separable convolutional layers of FPN can further extract richer feature information and enhance the network's ability to detect structures of different scales.

[0076] Optionally, the detection sub-model adopts a YoloX-based target detection model; the detection head of the detection sub-model includes deformable convolution.

[0077] It should be noted that although lens defects can be classified, there are differences in the manifestation of defect characteristics under the same category, because the position and direction of the defects are not constrained. For example, in general inspection, the direction of the same object should be roughly the same (people are perpendicular to the ground), and the defects on the lens are not constrained (scratches can occur in all directions).

[0078] Therefore, the detection head adopts a method based on deformable convolution. The present invention uses YoloX's detection head DecoupledHead as a basis and replaces the general convolution in the structure with deformable convolution. The deformable convolution can adaptively adjust the direction of the convolution kernel, thereby improving the accuracy of defect detection.

[0079] Optionally, the defect detection result includes a detection frame with a defect detection type; wherein the defect position of the non-planar lens to be detected is determined by the position of the detection frame in the target lens after the filling.

[0080] It should be noted that the final output is a detection frame containing the defect detection type. The detection frame provides the defect on the equivalent lens. Since the incident light is parallel to the outgoing light, it can be equivalent to the coordinates on the non-planar lens to be detected. coordinate.

[0081] In summary, the present application provides a method for detecting defects in non-planar lenses. Different from the existing methods, the present invention takes into account that the non-planar lens may be distorted in imaging due to the difference in refractive index and medium. Therefore, a complementary lens is used to make the incident light parallel to the outgoing light, thereby obtaining a clear equivalent lens image (the position of the equivalent lens defect is equal to the position of the lens defect to be detected).

[0082] The fused features are then further enhanced using Gaussian pyramids and Laplacian pyramids, and the detection head is optimized using variable convolution in combination with the non-uniform defect directions, further improving the detection accuracy. The benefits include: first, there is no restriction on the curvature of the lens, and second, it has higher detection accuracy.

[0083] Based on the same inventive concept, please refer to Figure 5 The present application provides a non-planar lens defect detection system 500, comprising: a first acquisition module 501, used to acquire a first image; wherein the first image is an image obtained by photographing the non-planar lens to be detected; a contour detection module 502, used to perform contour detection on the first image, and based on the contour detection result of the non-planar lens to be detected, determine a set of complementary lenses with contour matching from a lens library; a complementary lens determination module 503, used to perform refractive index simulation calculation based on the complementary lens set and the non-planar lens to be detected, and determine a first target lens and a second target lens for complementing the non-planar lens to be detected; wherein, when the first target lens and the After the second target lens complements the non-planar lens to be detected, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected; the first target lens and the second target lens are two non-planar lenses in the complementing lens set; the second acquisition module 504 is used to acquire a second image; wherein the second image is an equivalent lens image collected by using the complemented target lens as a lens; the defect detection module 505 is used to input the second image into the defect detection model to obtain the defect detection result of the complemented target lens; wherein the defect detection result of the complemented target lens represents the defect detection result of the non-planar lens to be detected.

[0084] See also Figure 6 Based on the same inventive concept, the present application embodiment provides a module frame of an electronic device 600 that applies the above method. The electronic device 600 includes: at least one processor 601 ( Figure 6 Only one is shown in the figure), a memory 602, a computer program 603 stored in the memory 602 and executable on at least one processor 601, and when the processor 601 executes the computer program 603, the steps of the method in any of the foregoing embodiments are implemented.

[0085] The electronic device 600 may be a server, a personal computer, a notebook computer, etc.

[0086] Those skilled in the art will understand that Figure 6 This is merely an example of the electronic device 600 and does not constitute a limitation on the electronic device 600 , which may include more or less components than those shown in the figure, or a combination of certain components, or different components.

[0087] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0088] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. In other embodiments, the memory 602 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 600. Furthermore, the memory 602 may also include both an internal storage unit of the electronic device 600 and an external storage device.

[0089] It should be noted that the above-mentioned systems, devices, etc. are based on the same concept as the method embodiments of the present application. The modules designed for the systems, the steps performed by the devices, and the technical effects brought about can all be found in the method embodiment section and will not be repeated here.

[0090] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0091] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0092] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0096] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting defects in non-planar lenses, characterized in that: include: Acquire a first image; wherein the first image is an image obtained by photographing the non-planar lens to be detected; Performing contour detection on the first image, and determining a set of complementary lenses with contour matching from a lens library based on the contour detection result of the non-planar lens to be detected; Based on the refractive index simulation calculation of the complement lens set and the non-planar lens to be detected, a first target lens and a second target lens for complementing the non-planar lens to be detected are determined; wherein, when the first target lens and the second target lens are used to complement the non-planar lens to be detected, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected; the first target lens and the second target lens are two non-planar lenses in the complement lens set; Acquire a second image; wherein the second image is an equivalent lens image collected by using the padded target lens as a lens; Inputting the second image into a defect detection model to obtain a defect detection result of the target lens after the complementation; wherein the defect detection result of the target lens after the complementation represents the defect detection result of the non-planar lens to be detected; The performing of contour detection on the first image and determining a complementary lens set with contour matching from a lens library based on the contour detection result of the non-planar lens to be detected comprises: performing contour detection on the first image based on a contour detection algorithm of a Canny operator to obtain a contour detection result of the non-planar lens to be detected; wherein the contour detection result of the non-planar lens to be detected comprises an identification contour of a first surface of the non-planar lens to be detected and an identification contour of a second surface of the non-planar lens to be detected; the first surface of the non-planar lens to be detected and the second surface of the non-planar lens to be detected are opposite surfaces; determining a first complementary lens set matching the identification contour of the first surface of the non-planar lens to be detected from the lens library, and determining a second complementary lens set matching the identification contour of the second surface of the non-planar lens to be detected from the lens library; wherein the complementary lens set comprises the first complementary lens set and the second complementary lens set.

2. The non-planar lens defect detection method according to claim 1, characterized in that: The refractive index simulation calculation is performed on the lenses in the complementary lens set and the non-planar lens to be detected through the following steps, including: Acquire a transmission matrix of a first lens, a transmission matrix of a second lens, and a transmission matrix of the non-planar lens to be detected; wherein the first lens and the second lens are both lenses in the complementary lens set; Based on the transmission matrix of the first lens, the transmission matrix of the second lens, and the transmission matrix of the non-planar lens to be detected, acquiring an equivalent transmission matrix of the completed lens; The equivalent refractive index of the padded lens is obtained through the equivalent transmission matrix of the padded lens.

3. The non-planar lens defect detection method according to claim 1, characterized in that: The defect detection model includes: a feature enhancement sub-model, a feature extraction sub-model and a detection sub-model; The feature enhancement sub-model, the feature extraction sub-model and the detection sub-model are connected in sequence; The feature enhancement sub-model is used to enhance the features on the second image; The feature extraction sub-model is used to extract features from the image after feature enhancement; The detection sub-model is used to perform defect recognition based on the extracted features and output defect detection results.

4. The non-planar lens defect detection method according to claim 3, characterized in that: The step of inputting the second image into a defect detection model to obtain a defect detection result of the filled target lens includes: Inputting the second image into the feature enhancement sub-model, and downsampling the second image through a Gaussian pyramid; wherein the Gaussian pyramid has N layers; N is a positive integer; Upsampling the downsampled image through a Laplacian pyramid to obtain an upsampled image; wherein the Laplacian pyramid also has N layers; Performing image fusion on the upsampled images to obtain the feature-enhanced image; The feature-enhanced image is input into the feature extraction sub-model, and the defect detection result is output through the detection sub-model.

5. The non-planar lens defect detection method according to claim 4, characterized in that: The Gaussian pyramid has four layers; the Laplacian pyramid has four layers; The number of the upsampled images is four.

6. The non-planar lens defect detection method according to claim 3, characterized in that: The feature extraction sub-model includes: a deep residual network and a feature pyramid network; The feature maps output by the 3rd, 4th and 5th layers of the deep residual network undergo another depthwise separable convolution and then enter the first three separable convolutional layers of the feature pyramid network through upsampling fusion.

7. The non-planar lens defect detection method according to claim 3, characterized in that: The detection sub-model adopts a YoloX-based target detection model; the detection head of the detection sub-model includes deformable convolution.

8. The non-planar lens defect detection method according to claim 7, characterized in that: The defect detection result includes a detection frame with a defect detection type; Wherein, the defect position of the non-planar lens to be detected is determined by the position of the detection frame on the target lens after the padding.

9. A non-planar lens defect detection system, characterized in that: include: A first acquisition module, used to acquire a first image; wherein the first image is an image obtained by photographing the non-planar lens to be detected; A contour detection module, configured to perform contour detection on the first image, and determine a set of complementary lenses with contour matching from a lens library based on the contour detection result of the non-planar lens to be detected; a complementing lens determination module, configured to perform refractive index simulation calculation based on the complementing lens set and the non-planar lens to be detected, and determine a first target lens and a second target lens for complementing the non-planar lens to be detected; wherein, when the first target lens and the second target lens are used to complement the non-planar lens to be detected, the equivalent refractive index of the complemented target lens is equal to the refractive index of the non-planar lens to be detected; and the first target lens and the second target lens are two non-planar lenses in the complementing lens set; A second acquisition module is used to acquire a second image; wherein the second image is an equivalent lens image collected by using the padded target lens as a lens; A defect detection module, used for inputting the second image into a defect detection model to obtain a defect detection result of the target lens after the filling; wherein the defect detection result of the target lens after the filling represents the defect detection result of the non-planar lens to be detected; The contour detection module is further specifically configured to perform contour detection on the first image based on a contour detection algorithm of a Canny operator, and obtain a contour detection result of the non-planar lens to be detected; wherein the contour detection result of the non-planar lens to be detected includes an identification contour of a first surface of the non-planar lens to be detected and an identification contour of a second surface of the non-planar lens to be detected; the first surface of the non-planar lens to be detected and the second surface of the non-planar lens to be detected are opposite surfaces; a first complementary lens set matching the identification contour of the first surface of the non-planar lens to be detected is determined from the lens library, and a second complementary lens set matching the identification contour of the second surface of the non-planar lens to be detected is determined from the lens library; wherein the complementary lens set includes the first complementary lens set and the second complementary lens set.

Images