Adjustment method of optical lens

Abstract

The invention discloses a method for adjusting an optical lens, and relates to the technical field of optical lens adjustment. The specific implementation process comprises the following steps: firstly, in combination with Hough linear transformation, a neighborhood fuzzy evaluation method and edge color change, comprehensively analyzing geometric distortion, fuzziness and chromatic aberration generated by a lens imaging result so as to obtain a distortion offset matrix, a fuzzy matrix and a color offset matrix; secondly, proposing a lens adjustment model based on a generative adversarial network, introducing an adversarial loss and cyclic loss optimization model, and further obtaining a lens adjustment pose matrix through a generator so as to adjust the position and angle of the lens; and finally, the adjustment accuracy is designed, so that the cyclic execution of the adjustment process is realized until the adjustment accuracy reaches a preset threshold value, the high definition of the image, the accurate color restoration and the reduction of optical distortion are ensured, and the adjustment precision and the imaging quality of the lens are improved.

Description

Technical Field

[0001] The invention relates to the technical field of optical lens adjustment, and in particular to an optical lens adjustment method. Background Art

[0002] As one of the core technologies in the field of photography, zoom lens has greatly enriched the creative space of photographers with its ability to flexibly adjust focal length. This type of lens achieves continuous zoom from wide angle to telephoto by changing the distance between the optical center and the photosensitive element, allowing the photographer to capture the details of the scene at different distances without moving, greatly improving the flexibility and convenience of photography. However, in the manufacturing process of zoom lenses, the assembly and adjustment of each optical lens is directly related to the clarity, color reproduction and overall image quality of the lens imaging. Therefore, the adjustment method of optical lenses is particularly important, which directly affects the imaging quality and performance of the optical system. Accurate adjustment can ensure high image clarity and accurate color reproduction, while reducing optical distortion and aberration, making the edges of the picture as clear and sharp as the center.

[0003] At present, the existing technology is only limited to the evaluation of the geometric distortion or clarity of the image, and does not consider the influence of the key factor of chromatic aberration, making it impossible to comprehensively evaluate the quality of lens imaging, resulting in low lens adjustment accuracy, which further affects the color accuracy and naturalness of the imaging results.

[0004] In order to solve the above problems, a method for adjusting an optical lens is proposed. Summary of the invention

[0005] The present invention provides an optical lens adjustment method for adjusting a zoom lens module. In order to solve the problems existing in the prior art, the present invention combines Hough linear transformation, neighborhood blur evaluation method and edge color change, comprehensively analyzes the geometric distortion, blur and chromatic aberration generated by the lens imaging result, thereby obtaining a distortion offset matrix, a blur matrix and a color offset matrix; secondly, a lens adjustment model based on a generative adversarial network is proposed, and adversarial loss and cyclic loss optimization models are introduced, and the lens adjustment posture matrix is ​​further obtained through a generator to adjust the lens position and angle; finally, the present invention designs the adjustment accuracy to realize the cyclic execution of the adjustment process until the adjustment accuracy reaches a predetermined threshold, thereby ensuring the high definition of the image, accurate color restoration and reduction of optical distortion, and improving the lens adjustment accuracy and imaging quality.

[0006] A method for adjusting an optical lens, the specific implementation steps comprising:

[0007] Setting the shooting environment and zoom lens to collect the actual image of the target object; performing ideal lens simulation according to the optical design software to obtain the ideal image of the target object;

[0008] According to Hough linear transformation, geometric distortion information of the actual image is obtained to generate a distortion offset matrix;

[0009] Analyze the image clarity of the actual image according to the neighborhood fuzzy evaluation method; combine with the ideal image to obtain a fuzzy matrix;

[0010] By analyzing the edge color change, determining whether the actual image has color difference; and obtaining a color shift matrix according to the ideal image;

[0011] Inputting the distortion offset matrix, the blur matrix and the color offset matrix into a lens calibration model to generate a calibration pose matrix;

[0012] According to the calibration pose matrix, the positions and angles of all lenses are adjusted to obtain a calibration image of the target object; after color space mapping, the mapping images of the calibration image and the ideal image are obtained respectively, and then the calibration color loss is obtained according to the two-norm function; through the structural similarity function, the overall similarity between the calibration image and the ideal image is generated; and the calibration accuracy is obtained by combining the overall similarity and the calibration color loss;

[0013] If the calibration accuracy is greater than or equal to a predetermined accuracy threshold, the calibration is stopped; otherwise, the distortion, blur and color difference information is re-acquired according to the calibrated image and the calibration is continued.

[0014] Preferably, the zoom lens comprises a first lens group, a second lens group, a third lens group and a fourth lens group; wherein the first lens group obtains the reflected light of the target object and initially focuses it; the second lens group changes the imaging focal length by moving the lens back and forth; the third lens group adjusts the focal position after zooming; and the fourth lens group performs final adjustment and focusing on the reflected light.

[0015] Preferably, the specific implementation process of obtaining the geometric distortion information of the actual image according to the Hough linear transform and generating the distortion offset matrix includes:

[0016] An edge image of the actual image is obtained through an edge detection algorithm; according to the Hough line transform, edge pixel points of the edge image are mapped to the Hough space to obtain edge points; through a voting mechanism, the number of different straight lines passing through the edge points is obtained respectively; if the number is greater than a predetermined threshold, the corresponding straight line is the actual straight line detected in the actual image, and M groups of first straight line parameters are obtained; according to the ideal image and the Hough line transform, the corresponding M groups of second straight line parameters are obtained; and the distortion offset matrix is ​​generated by combining the first straight line parameters and the second straight line parameters.

[0017] Preferably, according to the neighborhood fuzzy evaluation method, the specific implementation process of analyzing the image clarity of the actual image includes:

[0018] The actual image is mapped to the HSV color space to generate a chromaticity map, a saturation map and a brightness map; the average difference of a 3×3 size neighborhood around a pixel is calculated to obtain local difference data; according to the adjustment parameters, the local difference data is weightedly fused to obtain a local difference degree; wherein the local difference data includes a chromaticity neighborhood difference degree, a saturation neighborhood difference degree and a brightness neighborhood difference degree; the adjustment parameters include a chromaticity adjustment parameter α, a saturation adjustment parameter β and a brightness adjustment parameter γ, and 0<α<β<γ<1, α+β+γ=1;

[0019] According to the local difference, global difference data is obtained; the global discrete degree of the actual image pixel difference is analyzed by global relative calculation to obtain image clarity; the global difference data includes a global maximum difference, a global minimum difference and a global average difference.

[0020] Preferably, the implementation process of obtaining the fuzzy matrix in combination with the ideal image includes:

[0021] When the image clarity is less than a predetermined clarity threshold, the actual image and the ideal image are grayed, and N pairs of matching feature points are obtained according to a feature point matching algorithm; a fuzzy matrix is ​​generated through the pixel positions and gray values ​​of the matching feature points; wherein the fuzzy matrix includes a fuzzy offset and a fuzzy amount.

[0022] Preferably, the process of determining whether the actual image has color difference through the edge color change analysis includes:

[0023] According to the edge detection algorithm, edge information of the actual image on three color channels is obtained respectively to obtain a first edge image, a second edge image and a third edge image; K groups of edge point data are obtained, and edge offset is obtained through coordinate relative offset calculation; the edge point data includes edge point coordinates of the first edge image, edge point coordinates of the second edge image and edge point coordinates of the third edge image;

[0024] If the edge offset is greater than a predetermined offset threshold, it indicates that the actual image has lateral chromatic aberration;

[0025] According to the first edge image, the second edge image and the third edge image, a number of neighboring pixel points around the edge pixel point are obtained to obtain a first edge neighborhood map, a second edge neighborhood map and a third edge neighborhood map; after calculating the neighborhood pixel value difference, a first average difference and a second average difference are obtained; the first average difference and the second average difference are compared to obtain an edge color difference; wherein the first average difference is the average pixel difference between the first edge neighborhood map and the second edge neighborhood map; and the second average difference is the average pixel difference between the second edge neighborhood map and the third edge neighborhood map;

[0026] If the edge color difference is greater than a predetermined color difference threshold, it indicates that the actual image has longitudinal color difference; if the actual image has the lateral color difference and / or the longitudinal color difference, it indicates that the actual image has color difference.

[0027] Preferably, according to the ideal image, the process of obtaining the color shift matrix includes:

[0028] The actual image and the ideal image are projected into the HSV color space to obtain an actual chromaticity map, an ideal chromaticity map, an actual saturation map, and an ideal saturation map, respectively; after corresponding pixel difference calculations, chromaticity differences and saturation differences are obtained; the chromaticity differences are mapped into color differences, and combined with the saturation differences, the color shift matrix is ​​obtained.

[0029] Preferably, the implementation process of the lens adjustment model includes:

[0030] In the training stage, several groups of lens sample data are collected, including a first pose matrix, a first actual image and a first ideal image; according to the first actual image and the first ideal image, the corresponding distortion offset matrix, the blur matrix and the color offset matrix are obtained, and input into the first generator with the first pose matrix to obtain a second pose matrix; the second pose matrix is ​​input into the second generator to generate a second ideal image; and the matching degree between the second ideal image and the first ideal image is determined according to the discriminator;

[0031] In the inference stage, the distortion offset matrix, the blur matrix, the color offset matrix and the lens initial pose matrix are input into the trained first generator to obtain the calibrated pose matrix.

[0032] Preferably, the implementation process of the training phase of the lens adjustment model further includes:

[0033] The first ideal image is passed through the discriminator to obtain a first discrimination result; the second ideal image is input into the discriminator to obtain a second discrimination result; an adversarial loss function is generated according to the cross entropy loss function, the first discrimination result and the second discrimination result;

[0034] The first ideal image is replaced according to the second ideal image to obtain the updated distortion offset matrix, the blur matrix and the color offset matrix; a pseudo pose matrix is ​​generated through the first generator; a cosine similarity function is introduced to respectively calculate the similarity between different lens pose vectors in the second pose matrix and the pseudo pose matrix to obtain vector similarity; according to the projection of the unit vector, the probability distribution of the lens pseudo pose vector on the lens predicted pose vector is obtained; the first generator is constrained in combination with the vector similarity and the corresponding probability distribution to obtain a cyclic loss function.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The present invention proposes a neighborhood blur evaluation method to accurately measure the blur degree of the imaging result. The method maps the image to the HSV color space, calculates the local differences of the chromaticity, saturation and brightness channels respectively, and focuses on the structural differences of the imaging results through parameter adjustment, so as to more comprehensively evaluate the clarity of the image. Compared with different color channels with similar structures, it can more accurately capture the image blur information, which helps to more accurately correct the position of the lens during the calibration process, thereby improving the calibration accuracy of the lens.

[0037] 2. The present invention analyzes the edge color change to evaluate the color difference of the imaging result. The edge information and color distribution differences of the actual image on different color channels are analyzed, not only the offset of the edge pixels, but also the subtle changes in the color around the edge are considered, making the color difference detection more sensitive and accurate, thereby accurately detecting the lateral and longitudinal chromatic aberrations caused by the lens. This analysis method helps to detect chromatic aberrations in time during the adjustment process, further improves the adjustment accuracy of the lens, and improves the color reproduction and visual quality of the imaging results.

[0038] 3. The present invention proposes a lens calibration model based on a generative adversarial network to quickly obtain the calibration posture of the lens. The model continuously optimizes the generator and the discriminator during the training phase, so that the generated calibration posture matrix can more accurately reflect the actual calibration requirements of the lens; at the same time, the introduction of adversarial loss and cyclic loss further constrains the output of the generator, thereby ensuring the consistency and stability of the calibration process. In addition, the model can adapt to the calibration characteristics of different lenses through automatic learning, significantly improving the calibration accuracy and efficiency of the lens. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of a method for adjusting an optical lens provided by an embodiment of the present invention;

[0040] Figure 2 An example diagram of a zoom lens assembly provided by an embodiment of the present invention;

[0041] Figure 3 An example diagram of neighborhood fuzzy difference calculation provided by an embodiment of the present invention;

[0042] Figure 4 A flow chart of edge color change analysis provided by an embodiment of the present invention;

[0043] Figure 5 An exemplary diagram of different apertures provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Zoom lenses can achieve different viewing angles and compositions by changing the focal length without moving the camera, thus adapting to various shooting scenarios and needs. However, due to the complexity of the lens group inside the zoom lens, it is very necessary to perform lens adjustment on the zoom lens in order to ensure clear and sharp image quality at all focal lengths from wide angle to telephoto. Lens adjustment can correct problems such as geometric distortion, blur and chromatic aberration that may occur during the zoom process, thereby ensuring good image quality throughout the entire zoom range.

[0046] The present invention provides an optical lens calibration method to calibrate the lens module of a zoom lens to improve the accuracy of lens calibration. To illustrate the effectiveness of the method of the present invention, it will be specifically described in conjunction with the accompanying drawings of this embodiment and the following two embodiments.

[0047] Embodiment 1

[0048] The embodiment of the present application discloses a method for adjusting an optical lens to achieve lens adjustment during the automatic zooming process of a zoom lens A. Figure 1The specific steps of the method proposed in the present invention include: S1. Collecting the actual image of the target object and obtaining the corresponding ideal image; S2. Obtaining geometric distortion information according to Hough linear transformation and generating a distortion offset matrix; S3. Analyzing the clarity through the neighborhood fuzzy evaluation method to obtain a fuzzy matrix; S4. Obtaining the color offset matrix according to the edge color change analysis; S5. Generating a calibration posture matrix through a lens calibration model; S6. Obtaining a calibration image, calculating the calibration accuracy, and determining whether to continue the calibration.

[0049] Further, a shooting environment and a zoom lens A are set to collect an actual image of the target object; an ideal lens simulation is performed according to the optical design software to obtain an ideal image of the target object, corresponding to the above step S1; refer to Figure 2 The zoom lens includes a first lens group, a second lens group, a third lens group and a fourth lens group; wherein the first lens group obtains the reflected light of the target object and initially focuses it; the second lens group changes the imaging focal length by moving the lens back and forth; the third lens group adjusts the focal position after zooming; and the fourth lens group performs final adjustment and focusing on the reflected light.

[0050] Specifically, in order to perform effective A zoom lens adjustment, the shooting environment should have a stable light source and a clear background to reduce the impact of external interference on the image quality. In order to obtain the accurate posture of different lenses of the A zoom lens in the process from wide angle to telephoto, the embodiment of the present application sets several groups of different focal lengths to collect actual images of the target object respectively, and ensure that each part of the lens group can effectively capture and process light. In the ideal lens simulation process, the optical design software simulates the ideal imaging condition of the lens according to the preset optical parameters to obtain the ideal image of the target object. In the lens imaging process, the A zoom lens works together through its internal lens group: the first lens group captures and preliminarily focuses the reflected light of the target object; the second lens group adjusts the imaging focal length by moving back and forth, changing the viewing angle and composition; the third lens group accurately adjusts the focal position after zooming; the fourth lens group finally adjusts and focuses the light to ensure clear imaging.

[0051] The embodiment of the present application reduces the impact of environmental interference on the lens adjustment process through the setting of the shooting environment and the A zoom lens; through precise ideal lens simulation, it provides a precise ideal target image and also provides data support for the subsequent lens adjustment process.

[0052] Furthermore, according to the Hough linear transformation, the geometric distortion information of the actual image is obtained to generate the distortion offset matrix, which corresponds to the above step S2; the specific implementation process includes:

[0053] By using an edge detection algorithm, an edge image of the actual image is obtained; according to the Hough line transform, edge pixel points of the edge image are mapped to the Hough space to obtain edge points; by using a voting mechanism, the number of different straight lines passing through the edge points is obtained respectively; if the number is greater than a predetermined threshold, the corresponding straight line is the actual straight line detected in the actual image, and M groups of first straight line parameters are obtained; according to the ideal image and the Hough line transform, the corresponding M groups of second straight line parameters are obtained; the distortion offset matrix is ​​generated by combining the first straight line parameters and the second straight line parameters, and the specific calculation formula is:

[0054]

[0055] in, Represent the offset angle and offset distance vector respectively; An angle and position vector representing the first line parameter; an angle and position vector representing the second line parameter; represents the distortion offset matrix.

[0056] The embodiment of the present application improves the accurate identification of geometric distortion in lens calibration through Hough linear transformation; at the same time, it calculates an accurate distortion offset matrix in combination with the linear parameters in the ideal image, thereby comprehensively describing the degree and direction of geometric distortion. This process improves the accuracy of distortion detection and also provides accurate distortion parameters for the subsequent lens calibration process, further enhancing the geometric accuracy of lens imaging.

[0057] Furthermore, according to the neighborhood fuzzy evaluation method, the image clarity of the actual image is analyzed, corresponding to the above step S3; the specific implementation process includes:

[0058] Map the actual image to the HSV color space to generate a chromaticity map, a saturation map, and a brightness map; refer to Figure 3 , according to the 3×3 size neighboring area around the pixel point, the local difference is obtained; the specific formula of the local difference is:

[0059]

[0060] ΔD=α·ΔD H +β·ΔD S +γ·ΔD V ;

[0061] Where, ΔD c I represents the neighborhood difference of channel c and c = {H, S, V}; c Represents the pixel value of the pixel point of channel c of the actual image; I c The nth neighboring pixel around; ΔDH ,ΔD S ,ΔD V Respectively represent the neighborhood differences of hue, saturation and brightness channels; ΔD represents the local difference; α, β, γ represent adjustment parameters and 0<α<β<γ<1, α+β+γ=1;

[0062] According to the local difference, the image clarity is obtained, and the specific formula is expressed as:

[0063]

[0064] in, Indicates the image clarity; Max() indicates a maximum value function; Min() indicates a minimum value function; Avg() indicates an average value function.

[0065] Specifically, local difference reflects the complexity of the local structure of the image and the sharpness of the edges, while image clarity measures the relative changes in the overall content of the image. Usually, the difference in pixel value between a blurred image pixel and its neighboring pixels is small, and the corresponding local difference value range is narrow, so the calculated image clarity is often relatively low. A clear image has a sharp structure and distinct colors, and the corresponding pixel neighborhood has a large difference, which makes the local difference value range wider, so the corresponding image clarity is often relatively high.

[0066] The embodiment of the present application maps the actual image to the HSV color space through a neighborhood blur evaluation method to calculate the local differences of the chromaticity, saturation, and brightness channels respectively. Not only the color and brightness information of the image is taken into account, but also the structural differences of the imaging results are focused on through parameter adjustment, thereby achieving a comprehensive and accurate evaluation of the image clarity. Compared with RGB images with similar structures, the HSV color space can capture image blur information more comprehensively, which helps to more accurately locate the blur source of the lens during the calibration process and adjust the position of the lens in a targeted manner, thereby significantly improving the calibration accuracy of the lens.

[0067] Furthermore, in combination with the ideal image, the implementation process of obtaining the fuzzy matrix includes:

[0068] When the image clarity is less than a predetermined clarity threshold, the actual image and the ideal image are grayed, and N pairs of matching feature points are obtained according to a feature point matching algorithm; a fuzzy matrix is ​​generated through the pixel positions and gray values ​​of the matching feature points; wherein the fuzzy matrix includes a fuzzy offset and a fuzzy amount; the specific formula of the fuzzy matrix is:

[0069]

[0070] ΔMV i=|g' i -g i |;

[0071] Among them, ΔMP i represents the fuzzy offset of the i-th pair of matching feature points; ΔMV i The blur value of the i-th pair of matching feature points; (x' i ,y' i ) represents the horizontal and vertical coordinates of the ith pair of matching feature points in the actual image; (x i ,y i ) represents the horizontal and vertical coordinates of the i-th pair of matching feature points of the ideal image; g' i ,g i Respectively represent the grayscale values ​​of the i-th pair of matching feature points of the actual image and the ideal image.

[0072] The embodiment of the present application generates a blur matrix in combination with an ideal image, thereby achieving an accurate assessment of the blur degree of the image. Based on the pixel position and gray value difference of the matching feature points, a blur matrix containing blur offset and blur amount is generated to guide the lens adjustment process; this process not only improves the assessment accuracy of image clarity, but also ensures that the specific blur area can be accurately corrected during the adjustment process, thereby improving the accuracy and efficiency of lens adjustment.

[0073] Furthermore, by analyzing the edge color change, it is determined whether the actual image has lateral chromatic aberration, corresponding to Figure 1 S4 step of Figure 4 The specific implementation process includes:

[0074] According to the edge detection algorithm, the edge information of the actual image on the three color channels is obtained respectively to obtain the first edge image, the second edge image and the third edge image; by comparing the edge coordinates, the edge offset is obtained, and the specific formula is:

[0075]

[0076] Wherein, ΔEP represents the edge offset of the actual image; represent the coordinate vectors of the jth edge point of the first edge image, the second edge image, and the third edge image respectively; K represents the total number of edge points; |||| represents a vector modulo operation;

[0077] If the edge offset is greater than a predetermined offset threshold, it indicates that lateral chromatic aberration exists in the actual image.

[0078] The embodiment of the present application analyzes the edge color pixel offset and can accurately and efficiently identify whether there is a lateral chromatic aberration problem in the actual image. By extracting the edge information of the image on the three color channels respectively and comparing the coordinate differences of these edges to calculate the edge offset, not only the sensitivity of color difference detection is enhanced, but also the accuracy of the detection results is ensured. This method not only simplifies the complexity of the traditional color difference detection process, but also improves the detection efficiency, so that the lens adjustment process can more timely discover and correct the lateral chromatic aberration caused by the lens.

[0079] Furthermore, by analyzing the edge color change, it is determined whether the actual image has longitudinal chromatic aberration; the specific implementation process includes:

[0080] According to the first edge image, the second edge image and the third edge image, the edge pixel points of the actual image and the k neighboring pixel points around each edge pixel point are obtained to obtain the first edge neighborhood data set, the second edge neighborhood data set and the third edge neighborhood data set; the edge color difference is obtained by comparing the edge color distribution difference; wherein the edge neighborhood data set includes the pixel values ​​of (k+1)·K pixel points, K represents the total number of edge points; the specific formula of the edge color difference is:

[0081]

[0082] Wherein, ΔCD represents the edge color difference of the actual image; ε1, ε2, ε3 represent the first edge neighborhood data set, the second edge neighborhood data set and the third edge neighborhood data set respectively; Indicates the subtraction operation of pixel values ​​of pixels at the same position; A data set representing the difference in pixel values ​​between the first color channel and the second color channel of the actual image in the edge neighborhood area; Avg() represents an average function; Max() represents a maximum function;

[0083] If the edge color difference is greater than a predetermined color difference threshold, it indicates that the actual image has longitudinal color difference; if the actual image has the lateral color difference and / or the longitudinal color difference, it indicates that the actual image has color difference.

[0084] The embodiment of the present application introduces edge neighborhood color change analysis to detect longitudinal chromatic aberration in actual images. By constructing a neighborhood map around edge pixels and carefully comparing the color distribution differences in these neighborhoods, it is possible to accurately capture the subtle chromatic aberration that may exist in the longitudinal direction. This analysis process not only enhances the accuracy of chromatic aberration detection, but also ensures the comprehensiveness of the detection results, so that the lens adjustment process can more timely discover and correct the longitudinal chromatic aberration caused by the lens.

[0085] Specifically, lateral chromatic aberration usually refers to the obvious misalignment between the edge positions of different color channels in the horizontal direction of the image, which often presents obvious color fringing. Longitudinal chromatic aberration refers to the significant change or inconsistency in the color distribution near the edge of the image in the vertical direction, which generally causes color distortion and presents purple or green color fringing at the edge. Therefore, it is necessary to evaluate lateral chromatic aberration and longitudinal chromatic aberration separately to more accurately determine whether there is color aberration in the imaging result.

[0086] Furthermore, according to the ideal image, the process of obtaining the color shift matrix includes:

[0087] The actual image and the ideal image are projected into the HSV color space to obtain a chromaticity map and a saturation map respectively; the color shift matrix is ​​generated according to the chromaticity difference and the saturation difference, and the specific formula is:

[0088] ΔCS(h,w)=1-cos(|ΔH(h,w)|)+|ΔS(h,w)|;

[0089] Among them, ΔCS(h,w) represents the pixel offset at the position (h,w) of the color offset matrix; ΔH represents the chromaticity difference between the actual image and the ideal image; ΔS represents the saturation difference between the actual image and the ideal image.

[0090] Specifically, the color shift matrix quantifies the overall deviation in color between the actual image and the ideal image, and the deviation is not limited to a specific direction of the image (such as horizontal or vertical). Therefore, when calculating the color shift matrix, it is usually not necessary to perform independent calculations for the horizontal and vertical directions respectively. In addition, the chromaticity diagram and saturation diagram provide the distribution information of the image in the color space, which can reflect the overall change of the color attributes of the image, rather than just being limited to a certain direction. By comparing the differences in chromaticity and saturation between the actual image and the ideal image, the color deviation between the two can be fully reflected.

[0091] The embodiment of the present application projects the actual image and the ideal image into the HSV color space. Considering that the image color difference will not produce significant changes in the brightness channel, the color shift matrix is ​​generated only based on the chromaticity difference and the saturation difference. This process fully considers the color shift on the chromaticity and saturation channels, thereby ensuring that the adjusted image reaches a higher standard in color reproduction and naturalness, and improves the overall accuracy of lens adjustment.

[0092] Furthermore, the distortion offset matrix, the blur matrix and the color offset matrix are input into the lens calibration model to generate a calibration pose matrix corresponding to Figure 1 The implementation process of the lens calibration model includes:

[0093] In the training phase, several groups of lens sample data are collected, including a first pose matrix, a first actual image, and a first ideal image; the distortion offset matrix, the blur matrix, and the color offset matrix corresponding to the first actual image are obtained, and are input into a first generator together with the first pose matrix to obtain a second pose matrix; the second pose matrix is ​​input into a second generator to generate a second ideal image; and the matching degree between the second ideal image and the first ideal image is determined according to the discriminator;

[0094] In the inference stage, the distortion offset matrix, the blur matrix, the color offset matrix and the lens initial pose matrix are input into the trained first generator to obtain the calibrated pose matrix.

[0095] The lens calibration model proposed in the embodiment of the present application realizes a comprehensive evaluation and precise calibration of the lens imaging quality by fusing the distortion offset matrix, blur matrix and color offset matrix. In the training phase, the accuracy and reliability of the calibration pose matrix are ensured by collecting a large amount of lens sample data and continuously optimizing the generator and discriminator; in the inference phase, the calibration pose matrix is ​​quickly generated using the trained model, which significantly improves the calibration efficiency. This model not only makes the lens calibration process more automated and intelligent, but also significantly improves the calibration accuracy of the lens by comprehensively considering key factors such as distortion, blur and chromatic aberration.

[0096] Furthermore, the implementation process of the training phase of the lens calibration model also includes:

[0097] According to the first ideal image and the second ideal image, an adversarial loss function is generated, and the specific formula is:

[0098]

[0099] Among them, L D represents the adversarial loss function; Q1 and Q2 represent the first ideal image and the second ideal image respectively; represents the discriminator; log() represents a logarithmic function;

[0100] The first generator is constrained by the cyclic loss function, and the specific formula is expressed as:

[0101]

[0102] Among them, L G represents the cycle loss function; R v represents the pose vector of the vth lens in the second pose matrix; O represents the pseudo pose matrix of the second pose matrix after the second generator and the first generator; O urepresents the pose vector of the u-th lens in the pseudo pose matrix; Indicates O u In R v The probability distribution on ; G1(), G2() represent the first generator and the second generator respectively; Represents vector R v The transpose of ; |||| represents the vector modulo operation.

[0103] The embodiment of the present application introduces an adversarial loss function and a cyclic loss function, which significantly improves the accuracy and stability of lens calibration. Among them, the adversarial loss function enables the model to continuously optimize the calibration pose matrix of the generator by comparing the first ideal image and the second ideal image, ensuring that the generated second ideal image can be highly close to the first ideal image, thereby enhancing the calibration accuracy of the model; the cyclic loss function further constrains the output of the generator, ensuring the consistency and stability of the calibration process, avoiding the accumulation of deviations in the generated pose matrix, thereby improving the accuracy and efficiency of lens calibration.

[0104] Further, according to the calibration pose matrix, the positions and angles of all lenses are adjusted to obtain the calibration image of the target object; after color space mapping, the mapping images of the calibration image and the ideal image are obtained respectively, and then the calibration color loss is obtained according to the two-norm function; through the structural similarity function, the overall similarity between the calibration image and the ideal image is generated; combining the overall similarity and the calibration color loss, the calibration accuracy is obtained, corresponding to Figure 1 Step S6; the specific calculation formula for the adjustment accuracy is:

[0105]

[0106] in, represents the calibration accuracy; Z, X represent the calibration image and the ideal image respectively; SSIM() represents the structural similarity function; represents the color space mapping function; ||||2 represents the two-norm function; ∈1,∈2 represents the adjustment parameters;

[0107] If the calibration accuracy is greater than or equal to a predetermined accuracy threshold, the calibration is stopped; otherwise, the distortion, blur and color difference information is re-acquired according to the calibrated image and the calibration is continued.

[0108] Specifically, in order to further illustrate the role of the adjustment accuracy proposed in the present invention, the adjustment accuracy of the lens at different focal lengths of the A zoom lens is given as an example for comparison. The adjustment accuracy of the lens at different focal lengths is listed in Table 1.

[0109] Table 1. Adjustment accuracy of the lens at different focal lengths of A zoom lens

[0110]

[0111] In the embodiment of the present application, different adjustment parameters are set for different focal lengths. According to the data in Table 1, it can be seen that the same adjustment parameters are used for the 25mm and 35mm focal lengths and the proportion of the adjusted color loss is larger. This is because the focal length of a wide-angle lens is generally between 24mm and 35mm and has higher requirements for color reproduction and accuracy; and 15mm is usually an ultra-wide-angle lens, so the adjustment parameter for adjusting the color loss is larger; on the contrary, the focal length of a telephoto lens is generally between 150mm and 300mm, and the focal length of an ultra-telephoto lens is above 300mm. These lenses have higher requirements for imaging clarity, so the adjustment parameters for the overall similarity are higher.

[0112] Specifically, during the automatic zooming process of zoom lens A, the adjustment posture matrices at different focal lengths can be obtained through the above process, and several groups of lens focal lengths and corresponding lens postures can be obtained; further, a corresponding mapping model is selected, and these data are used to fit the model to obtain a mapping relationship between the lens focal length and different lens postures, so as to automatically adjust the lens posture during the zooming process and improve the imaging quality of zoom lens A; the mapping model includes a linear model, a polynomial model and a nonlinear model.

[0113] The embodiment of the present application achieves efficient adjustment of the zoom lens module by comprehensively evaluating the influencing factors such as geometric distortion, blur and chromatic aberration of the imaging results. The specific implementation process mainly includes the following steps: (1) blur degree analysis; (2) chromatic aberration analysis; (3) determination of lens adjustment posture. The present invention proposes a neighborhood blur evaluation method, an edge color change analysis and a lens adjustment model for the above three processes respectively; the neighborhood blur evaluation method comprehensively captures the structure and blur information of the image, providing a reliable basis for the precise correction of the lens posture; the edge color change analysis deeply analyzes the edge color change, combines the edge pixel offset with the subtle difference of the surrounding color, and realizes the sensitive detection of the lateral and longitudinal chromatic aberration caused by the lens; the lens adjustment model, through automatic learning and iterative training, quickly generates a posture matrix that meets the actual lens adjustment requirements, while ensuring the consistency and stability of the adjustment process, greatly improving the accuracy of lens adjustment.

[0114] Embodiment 2

[0115] In the first embodiment, the method of the present invention realizes the lens adjustment for the automatic zooming process of the A zoom lens. In the embodiment of the present application, the method proposed by the present invention will be described again to realize the lens adjustment for the B zoom lens with different apertures. The specific implementation process is as follows:

[0116] A shooting environment and a B zoom lens are set to collect an actual image of a target object; an ideal lens simulation is performed according to optical design software to obtain an ideal image of the target object; the zoom lens includes a first lens group, a second lens group, a third lens group, a fourth lens group and an aperture.

[0117] In the embodiments of this application, see Figure 5 , the aperture controls the amount of light allowed to pass through the B zoom lens, which directly affects the exposure, depth of field and image quality of the photo. If the position of the lens group is not corrected in time after replacing the aperture, background blur and color distortion may occur, affecting the overall clarity and tonal accuracy. Considering that geometric distortion is related to the design of the lens, and the aperture mainly affects the amount of light entering and the depth of field, adjusting the aperture alone has little effect on the degree of distortion. Therefore, when calibrating the B zoom lens with different apertures, the embodiment of the present application only evaluates the clarity and color difference of the imaging results.

[0118] Further, the actual image is mapped to the HSV color space to generate a chromaticity map, a saturation map and a brightness map; the average difference of the 3×3 size neighborhood around the pixel point is calculated to obtain local difference data; according to the adjustment parameter, the local difference data is weightedly fused to obtain the local difference degree; wherein the local difference data includes the chromaticity neighborhood difference degree, the saturation neighborhood difference degree and the brightness neighborhood difference degree; the adjustment parameter includes the chromaticity adjustment parameter α, the saturation adjustment parameter β and the brightness adjustment parameter γ and 0<α<β<γ<1, α+β+γ=1;

[0119] According to the local difference, global difference data is obtained; the global discrete degree of the actual image pixel difference is analyzed by global relative calculation to obtain image clarity; the global difference data includes a global maximum difference, a global minimum difference and a global average difference;

[0120] When the image clarity is less than a predetermined clarity threshold, the actual image and the ideal image are grayed, and N pairs of matching feature points are obtained according to a feature point matching algorithm; a fuzzy matrix is ​​generated through the pixel positions and gray values ​​of the matching feature points; wherein the fuzzy matrix includes a fuzzy offset and a fuzzy amount.

[0121] Further, edge information of the actual image on three color channels is obtained respectively according to an edge detection algorithm to obtain a first edge image, a second edge image and a third edge image; K groups of edge point data are obtained, and an edge offset is obtained through coordinate relative offset calculation;

[0122] If the edge offset is greater than a predetermined offset threshold, it indicates that the actual image has lateral chromatic aberration;

[0123] According to the first edge image, the second edge image and the third edge image, a plurality of neighboring pixel points around the edge pixel point are obtained to obtain a first edge neighborhood map, a second edge neighborhood map and a third edge neighborhood map; after calculating the difference of neighborhood pixel values, a first average difference and a second average difference are obtained; and the first average difference and the second average difference are compared to obtain an edge color difference;

[0124] If the edge color difference is greater than a predetermined color difference threshold, it indicates that the actual image has longitudinal color difference; if the actual image has the lateral color difference and / or the longitudinal color difference, it indicates that the actual image has color difference;

[0125] When there is color difference in the actual image, the actual image and the ideal image are projected into the HSV color space to obtain the actual chromaticity map, the ideal chromaticity map, the actual saturation map and the ideal saturation map respectively; after the corresponding pixel difference calculation, the chromaticity difference and the saturation difference are obtained; the chromaticity difference is mapped into the color difference, and combined with the saturation difference, the color offset matrix is ​​obtained.

[0126] In the embodiment of the present application, since the aperture change of the B zoom lens directly affects the depth of field of the lens and the incident angle of light, the lateral and longitudinal chromatic aberration performance of the B zoom lens at different apertures may be significantly different. By evaluating these two chromatic aberrations separately, the optical characteristics of the lens at different aperture settings can be more comprehensively understood, so as to more accurately adjust the lens position or optimize the lens design to reduce chromatic aberration and improve image quality.

[0127] Furthermore, the blur matrix and the color offset matrix are input into a lens calibration model to generate a calibration pose matrix; the implementation process of the lens calibration model includes:

[0128] In the training phase, several groups of lens sample data are collected, including a first pose matrix, a first actual image, and a first ideal image; according to the first actual image and the first ideal image, the corresponding blur matrix and the color offset matrix are obtained, and are input into the first generator together with the first pose matrix to obtain a second pose matrix; the second pose matrix is ​​input into the second generator to generate a second ideal image; the matching degree between the second ideal image and the first ideal image is judged according to the discriminator; the discrimination result is constrained by the adversarial loss function, and the cyclic loss function is introduced to optimize the generator, so that the first generator can output an accurate lens pose matrix;

[0129] In the inference stage, the blur matrix, the color offset matrix and the initial lens pose matrix are input into the trained first generator to obtain the calibrated pose matrix.

[0130] Further, according to the calibration pose matrix, the positions and angles of all lenses are adjusted to obtain a calibration image of the target object; after color space mapping, the mapping images of the calibration image and the ideal image are obtained respectively, and then the calibration color loss is obtained according to the two-norm function; through the structural similarity function, the overall similarity between the calibration image and the ideal image is generated; and the calibration accuracy is obtained by combining the overall similarity and the calibration color loss;

[0131] If the calibration accuracy is greater than or equal to a predetermined accuracy threshold, the calibration is stopped; otherwise, blur and color difference information is reacquired according to the calibrated image and the calibration is continued.

[0132] In the embodiment of the present application, the lens adjustment posture matrix of the B zoom lens at different apertures can be obtained through the above process, and different apertures and corresponding lens postures can be obtained, which helps the photographer to better utilize the characteristics of the lens. For example, when shooting a specific scene, the appropriate aperture and corresponding lens posture can be selected as needed to obtain the best imaging effect.

[0133] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for adjusting an optical lens, characterized in that: include: Set up the shooting environment and zoom lens to collect the actual image of the target object; Perform ideal lens simulation according to optical design software to obtain an ideal image of the target object; According to Hough linear transformation, geometric distortion information of the actual image is obtained to generate a distortion offset matrix; Analyze the image clarity of the actual image according to the neighborhood fuzzy evaluation method; combine with the ideal image to obtain a fuzzy matrix; By analyzing the edge color change, determining whether the actual image has color difference; According to the ideal image, a color shift matrix is ​​obtained; Inputting the distortion offset matrix, the blur matrix and the color offset matrix into a lens calibration model to generate a calibration pose matrix; According to the calibration pose matrix, the positions and angles of all lenses are adjusted to obtain a calibration image of the target object; Through color space mapping, mapping images of the calibrated image and the ideal image are obtained respectively, and then the calibrated color loss is obtained according to the two-norm function; the overall similarity between the calibrated image and the ideal image is generated through the structural similarity function; the calibration accuracy is obtained by combining the overall similarity and the calibrated color loss; If the calibration accuracy is greater than or equal to a predetermined accuracy threshold, the calibration is stopped; otherwise, the distortion, blur and color difference information is re-acquired according to the calibrated image and the calibration is continued.

2. The method for adjusting an optical lens according to claim 1, characterized in that: The zoom lens includes a first lens group, a second lens group, a third lens group and a fourth lens group; wherein the first lens group obtains the reflected light of the target object and initially focuses it; the second lens group changes the imaging focal length by moving the lens back and forth; the third lens group adjusts the focal position after zooming; and the fourth lens group performs final adjustment and focusing on the reflected light.

3. The method for adjusting an optical lens according to claim 1, characterized in that: The specific implementation process of obtaining the geometric distortion information of the actual image according to the Hough linear transformation and generating the distortion offset matrix includes: An edge image of the actual image is obtained through an edge detection algorithm; according to the Hough line transform, edge pixel points of the edge image are mapped to the Hough space to obtain edge points; through a voting mechanism, the number of different straight lines passing through the edge points is obtained respectively; if the number is greater than a predetermined threshold, the corresponding straight line is the actual straight line detected in the actual image, and M groups of first straight line parameters are obtained; according to the ideal image and the Hough line transform, the corresponding M groups of second straight line parameters are obtained; and the distortion offset matrix is ​​generated by combining the first straight line parameters and the second straight line parameters.

4. The method for adjusting an optical lens according to claim 1, characterized in that: According to the neighborhood fuzzy evaluation method, the specific implementation process of analyzing the image clarity of the actual image includes: The actual image is mapped to the HSV color space to generate a chromaticity map, a saturation map and a brightness map; the average difference of a 3×3 size neighborhood around a pixel is calculated to obtain local difference data; according to the adjustment parameters, the local difference data is weightedly fused to obtain a local difference degree; wherein the local difference data includes a chromaticity neighborhood difference degree, a saturation neighborhood difference degree and a brightness neighborhood difference degree; the adjustment parameters include a chromaticity adjustment parameter α, a saturation adjustment parameter β and a brightness adjustment parameter γ, and 0<α<β<γ<1, α+β+γ=1; According to the local difference, global difference data is obtained; the global discrete degree of the actual image pixel difference is analyzed by global relative calculation to obtain image clarity; the global difference data includes a global maximum difference, a global minimum difference and a global average difference.

5. The method for adjusting an optical lens according to claim 1, characterized in that: The implementation process of obtaining the fuzzy matrix in combination with the ideal image includes: When the image clarity is less than a predetermined clarity threshold, the actual image and the ideal image are grayed, and N pairs of matching feature points are obtained according to a feature point matching algorithm; a fuzzy matrix is ​​generated through the pixel positions and gray values ​​of the matching feature points; wherein the fuzzy matrix includes a fuzzy offset and a fuzzy amount.

6. The method for adjusting an optical lens according to claim 1, characterized in that: The implementation process of judging whether the actual image has color difference through the edge color change analysis includes: According to the edge detection algorithm, edge information of the actual image on three color channels is obtained respectively to obtain a first edge image, a second edge image and a third edge image; K groups of edge point data are obtained, and edge offset is obtained through coordinate relative offset calculation; the edge point data includes edge point coordinates of the first edge image, edge point coordinates of the second edge image and edge point coordinates of the third edge image; If the edge offset is greater than a predetermined offset threshold, it indicates that lateral chromatic aberration exists in the actual image.

7. The method for adjusting an optical lens according to claim 6, characterized in that: The implementation process of judging whether the actual image has color difference through the edge color change analysis further includes: According to the first edge image, the second edge image and the third edge image, a number of neighboring pixel points around the edge pixel point are obtained to obtain a first edge neighborhood map, a second edge neighborhood map and a third edge neighborhood map; after calculating the neighborhood pixel value difference, a first average difference and a second average difference are obtained; the first average difference and the second average difference are compared to obtain an edge color difference; wherein the first average difference is the average pixel difference between the first edge neighborhood map and the second edge neighborhood map; and the second average difference is the average pixel difference between the second edge neighborhood map and the third edge neighborhood map; If the edge color difference is greater than a predetermined color difference threshold, it indicates that the actual image has longitudinal color difference; if the actual image has the lateral color difference and / or the longitudinal color difference, it indicates that the actual image has color difference.

8. The method for adjusting an optical lens according to claim 1, characterized in that: According to the ideal image, the process of obtaining the color shift matrix includes: The actual image and the ideal image are projected into the HSV color space to obtain an actual chromaticity map, an ideal chromaticity map, an actual saturation map, and an ideal saturation map, respectively; after corresponding pixel difference calculations, chromaticity differences and saturation differences are obtained; the chromaticity differences are mapped into color differences, and combined with the saturation differences, the color shift matrix is ​​obtained.

9. The method for adjusting an optical lens according to claim 1, characterized in that: The implementation process of the lens adjustment model includes: In the training stage, several groups of lens sample data are collected, including a first pose matrix, a first actual image and a first ideal image; according to the first actual image and the first ideal image, the corresponding distortion offset matrix, the blur matrix and the color offset matrix are obtained, and input into the first generator with the first pose matrix to obtain a second pose matrix; the second pose matrix is ​​input into the second generator to generate a second ideal image; and the matching degree between the second ideal image and the first ideal image is determined according to the discriminator; In the inference stage, the distortion offset matrix, the blur matrix, the color offset matrix and the lens initial pose matrix are input into the trained first generator to obtain the calibrated pose matrix.

10. The method for adjusting an optical lens according to claim 9, characterized in that: The implementation process of the lens calibration model in the training stage also includes: The first ideal image is passed through the discriminator to obtain a first discrimination result; the second ideal image is input into the discriminator to obtain a second discrimination result; an adversarial loss function is generated according to the cross entropy loss function, the first discrimination result and the second discrimination result; The first ideal image is replaced according to the second ideal image to obtain the updated distortion offset matrix, the blur matrix and the color offset matrix; a pseudo pose matrix is ​​generated through the first generator; a cosine similarity function is introduced to respectively calculate the similarity between different lens pose vectors in the second pose matrix and the pseudo pose matrix to obtain vector similarity; according to the projection of the unit vector, the probability distribution of the lens pseudo pose vector on the lens predicted pose vector is obtained; the first generator is constrained in combination with the vector similarity and the corresponding probability distribution to obtain a cyclic loss function.

Images