Optical correction camera lens module and imaging method thereof

Abstract

The invention provides an optical correction camera lens module and an imaging method thereof. The method comprises the following steps: extracting a radial distortion parameter and a tangential distortion parameter from an original imaging image; performing target identification on the original imaging image to obtain a plurality of identification targets, performing regularized screening to obtain a regularized identification target, extracting a pixel reduction matrix of the regularized identification target based on a regularized target library, performing region reconstruction on a corresponding region of the original imaging image according to the pixel reduction matrix, and performing region reconstruction on a corresponding region of the original imaging image; extracting a radial distortion correction parameter and a tangential distortion correction parameter according to the reconstructed image area; and constructing an optical correction model based on the radial distortion correction parameter and the tangential distortion correction parameter, performing distortion correction on the original imaging image through the optical correction model to obtain a target imaging image, extracting a regularized identification target according to the original imaging image, and extracting a dynamic distortion correction parameter through the regularized identification target to obtain the target imaging image. And the real-time performance of imaging distortion correction is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of imaging processing, and more particularly, to an optical correction camera lens module and an imaging method thereof. BACKGROUND

[0002] Lens distortion refers to the image deformation phenomenon that the imaging picture deviates from the ideal position due to the inconsistency of the magnification of the central area and the edge area of the lens, and the degree of lens distortion increases from the picture center to the picture edge, and the picture edge reflects more obviously. Due to the natural structure of the optical lens, the object on the imaging of the optical camera is geometrically distorted, especially radially distorted, which easily brings unavoidable errors to the later various applications.

[0003] The existing geometric distortion correction method mainly obtains radial and axial distortion parameters by placing a calibration template in the scene, so as to realize distortion correction, but placing a calibration template in the scene is limited by the application scene and cannot be corrected at all times, so that the real-time performance of the imaging distortion correction is not high. SUMMARY

[0004] The present application provides an optical correction camera lens module and an imaging method thereof, which can extract a regularized recognition target according to an original imaging image, and extract dynamic distortion correction parameters through the regularized recognition target, thereby improving the real-time performance of the imaging distortion correction.

[0005] In a first aspect, the present application provides an optical correction camera lens imaging method, which can be executed by a network device, or can also be executed by a chip configured in the network device, and the present application does not limit this.

[0006] Specifically, the method comprises: obtaining an original imaging image of an optical correction camera; extracting distortion features from the original imaging image, wherein the distortion features comprise radial distortion parameters and tangential distortion parameters; when the radial distortion parameters and the tangential distortion parameters meet a preset distortion correction condition, performing target recognition on the original imaging image to obtain a plurality of recognition targets; performing regularized screening on the plurality of recognition targets to obtain regularized recognition targets, extracting a pixel restoration matrix of the regularized recognition targets based on a regularized target library, performing regional reconstruction on a corresponding region of the original imaging image according to the pixel restoration matrix, and extracting radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region; constructing an optical correction model based on the radial distortion correction parameters and the tangential distortion correction parameters, performing distortion correction on the original imaging image through the optical correction model, and obtaining a target imaging image.

[0007] In some implementations of the first aspect, the raw image data is acquired by an image sensor corresponding to the optical correction camera, and the raw image data is output to the image processing module through a camera driving interface to form a raw imaging image.

[0008] In some implementations of the first aspect, the target recognition on the raw imaging image includes: performing multi-level binarization processing on the raw imaging image to obtain a plurality of binarization images; extracting a plurality of boundary contour features based on each binarization image, respectively, and performing target recognition on each boundary contour feature using a pre-trained target detection model to obtain a plurality of recognition targets.

[0009] In some implementations of the first aspect, the target recognition on the raw imaging image includes:

[0010] In some implementations of the first aspect, the rule-based screening on the plurality of recognition targets includes: extracting a geometric shape feature corresponding to each recognition target; obtaining a target category corresponding to the recognition target, obtaining a standard geometric shape parameter corresponding to the target category, and extracting a feature contrast based on the standard geometric shape parameter and the geometric shape feature corresponding to the recognition target; iterating through each recognition target and selecting a recognition target with the lowest feature contrast as the rule-based recognition target.

[0011] In some implementations of the first aspect, the pixel restoration matrix of the rule-based recognition target is extracted based on a rule-based target library, and the pixel restoration matrix includes: obtaining a rule-based target library, matching the boundary contour information of the rule-based recognition target with a standard geometric template of a corresponding category in the rule-based target library; mapping the position of each pixel of the recognition target in the raw image to a standard template coordinate position according to the matching result to generate the pixel restoration matrix of the rule-based recognition target.

[0012] In some implementations of the first aspect, the region reconstruction of the corresponding region of the raw imaging image based on the pixel restoration matrix includes: obtaining a corresponding region pixel of the rule-based recognition target in the raw imaging image; Map each pixel to a standard template coordinate position according to the offset in the pixel restoration matrix, wherein for non-integer coordinate positions, the pixel value is calculated by an interpolation method, and all the mapped pixels form a reconstruction region; Integrate the reconstruction region into the original imaging image to obtain a reconstructed image region.

[0013] In a second aspect, the present application provides an optical correction camera lens module, which comprises an imaging processing unit, and the imaging processing unit comprises: An imaging data extraction module is configured to acquire an original imaging image of the optical correction camera, and extract distortion features from the original imaging image, wherein the distortion features comprise radial distortion parameters and tangential distortion parameters. An imaging data processing module is configured to perform target recognition on the original imaging image when the radial distortion parameters and the tangential distortion parameters satisfy a preset distortion correction condition, and obtain a plurality of recognized targets. The imaging data processing module is further configured to perform regularized screening on the plurality of recognized targets to obtain regularized recognized targets, extract a pixel restoration matrix of the regularized recognized targets based on a regularized target library, perform regional reconstruction on a corresponding region of the original imaging image according to the pixel restoration matrix, and extract radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region. An imaging distortion correction module is configured to construct an optical correction model by using the radial distortion correction parameters and the tangential distortion correction parameters, and perform distortion correction on the original imaging image by using the optical correction model to obtain a target imaging image.

[0014] In a third aspect, the present application provides a computer terminal device, which comprises a memory and a processor, the memory stores a code, and the processor is configured to acquire the code and perform the optical correction camera lens imaging method described above.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, which stores at least one computer program, the computer program is loaded and executed by a processor to implement the operations performed by the optical correction camera lens imaging method described above.

[0016] The technical scheme provided by the embodiments of the present application has the following beneficial effects: The application provides a camera lens module and an imaging method thereof. The method comprises the following steps: obtaining an original imaging image of the camera lens module; extracting distortion features of the original imaging image, wherein the distortion features comprise radial distortion parameters and tangential distortion parameters; when the radial distortion parameters and the tangential distortion parameters satisfy a preset distortion correction condition, performing target recognition on the original imaging image to obtain a plurality of recognized targets; performing regularized screening on the plurality of recognized targets to obtain a regularized recognized target; extracting a pixel restoration matrix of the regularized recognized target based on a regularized target library; performing regional reconstruction on a corresponding region of the original imaging image according to the pixel restoration matrix; extracting radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region; constructing an optical correction model based on the radial distortion correction parameters and the tangential distortion correction parameters; and performing distortion correction on the original imaging image through the optical correction model to obtain a target imaging image.

[0017] It can be seen that the application extracts radial distortion parameters and tangential distortion parameters through distortion feature extraction on the original imaging image, reflects the distortion state of the lens under the current acquisition condition, and when the distortion parameters are within the preset correctable range, that is, the distortion correction condition is satisfied, the image features are stable enough for target recognition, thereby ensuring the accuracy and repeatability of the recognition result. On this basis, regularized screening is performed on the plurality of recognized targets, the target with the closest shape to the standard template and the most regular geometric features is selected from the original image as the regularized recognized target, the actual pixels are corresponded to the standard template pixels through the pixel restoration matrix extracted from the regularized target library, the ideal geometric shape of the target region can be accurately restored after regional reconstruction, thereby eliminating the interference of local distortion on parameter calculation, the radial distortion correction parameters and the tangential distortion correction parameters are further extracted based on the reconstructed image region, the obtained distortion parameters can reflect the dynamic distortion features under the current acquisition condition, and the real-time and self-adaptability are higher than those of the traditional preset parameters or fixed calibration method. The distortion correction model can be directly constructed and the original imaging image can be corrected. Finally, the original image is pixel-level mapped and corrected through the optical correction model to obtain a target imaging image, and real-time distortion correction is realized.

[0018] In summary, the application can extract a regularized recognized target according to the original imaging image, and extract dynamic distortion correction parameters through the regularized recognized target, thereby improving the real-time performance of imaging distortion correction. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is an exemplary flowchart of a camera lens imaging method according to some embodiments of the application; Figure 2 is a structural schematic diagram of an imaging processing unit according to some embodiments of the application; Figure 3is a structural schematic diagram of a computer terminal device for implementing an optical correction camera lens imaging method according to some embodiments of the present application. DETAILED DESCRIPTION

[0020] The present application obtains an original imaging image of an optical correction camera, extracts distortion features from the original imaging image, the distortion features including radial distortion parameters and tangential distortion parameters, performs target recognition on the original imaging image when the radial distortion parameters and the tangential distortion parameters meet preset distortion correction conditions, obtains a plurality of recognized targets, performs regularized screening on the plurality of recognized targets, obtains a regularized recognized target, extracts a pixel restoration matrix of the regularized recognized target based on a regularized target library, performs regional reconstruction on a corresponding region of the original imaging image according to the pixel restoration matrix, extracts radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region, constructs an optical correction model based on the radial distortion correction parameters and the tangential distortion correction parameters, and performs distortion correction on the original imaging image through the optical correction model to obtain a target imaging image. The regularized recognized target can be extracted according to the original imaging image, and dynamic distortion correction parameters can be extracted through the regularized recognized target, thereby improving the real-time performance of imaging distortion correction.

[0021] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings in the specification and specific embodiments; refer to Figure 1 The figure is an exemplary flowchart of an optical correction camera lens imaging method 100 according to some embodiments of the present application, which mainly includes the following steps: In step S101, an original imaging image of an optical correction camera is obtained, and distortion features are extracted from the original imaging image, the distortion features including radial distortion parameters and tangential distortion parameters.

[0022] It should be noted that the optical correction camera refers to a camera device provided with a data processing module according to the present application, the data processing module can construct an optical correction model based on distortion parameters and perform a geometric correction operation on a collected image, thereby reducing the distortion influence caused by the lens structure, wherein the original imaging image refers to image data collected and output by a target camera lens without optical correction or image preprocessing operation, and the original imaging image has inherent distortion features of the camera lens, which will not be described herein.

[0023] Optionally, in some embodiments, obtaining the original imaging image of the optical correction camera specifically includes: collecting original image data through an image sensor corresponding to the optical correction camera, wherein the original image data can be output in a Bayer mode or a RAW format, and the original image data is output to an image processing module through a camera driving interface.

[0024] It should be noted that the radial distortion parameter is used to represent the distortion degree of the image in the radial direction of the optical axis, and the tangential distortion parameter is used to represent the distortion degree of the image in the asymmetric direction of the optical axis. Preferably, in some embodiments, the distortion feature extraction on the original imaging image specifically comprises: identifying a preset feature point in the original imaging image, and calculating the offset between the feature point and an ideal geometric position; substituting the offset into a preset distortion model, and solving the radial distortion parameter and the tangential distortion parameter by using a nonlinear optimization algorithm.

[0025] In a specific implementation, the image processing module built-in or externally connected to the optical calibration camera can be used to automatically identify the preset feature point in the original imaging image. The feature point can be a corner point of a regular calibration board, a center point of a circular dot array, or a position with high stability such as a corner point or an edge intersection point in a natural scene. The pixel coordinates of the feature point are obtained by using a feature point detection algorithm, preferably Harris corner detection, Shi-Tomasi algorithm, sub-pixel corner positioning, and the like. Then, based on a preset geometric model, such as an ideal geometric relationship of a checkerboard point array, the offset between the actual coordinates of the feature point in the original imaging image and the geometric position of the feature point on an ideal imaging plane is calculated. The offset is used to reflect the geometric deviation caused by the lens distortion of the camera. The offset is substituted into the distortion model for fitting and solving. The distortion model is preferably a combination of a radial distortion model and a tangential distortion model, wherein the radial distortion parameter is used to describe the nonlinear deformation of the image center expanding or contracting outward, and the tangential distortion parameter is used to describe the transverse displacement caused by the lens assembly error or tilt.

[0026] In some specific embodiments of the present application, a nonlinear optimization algorithm, preferably a least squares method, a Levenberg-Marquardt algorithm, or a gradient descent algorithm, can be used to iteratively solve the unknown parameters in the distortion model to minimize the reprojection error between the actual imaging position and the ideal position of all feature points, so as to obtain the radial distortion parameter and the tangential distortion parameter of the target camera lens, and complete the extraction of the distortion feature.

[0027] In step S102, when the radial distortion parameter and the tangential distortion parameter satisfy a preset distortion correction condition, target recognition is performed on the original imaging image to obtain a plurality of recognition targets.

[0028] It should be noted that, considering that when the parameter value of the distortion feature is within the preset range, it indicates that the lens distortion has reached the condition for target recognition (in the actual image acquisition process, too large distortion may lead to unstable feature recognition), the target recognition step is triggered, preferably, in some embodiments, the distortion correction condition is that the radial distortion parameter and the tangential distortion parameter are both within a preset interval range, further, the interval range can be set according to different types of camera lenses, focal lengths and shooting environments, when the actually measured radial distortion parameter and tangential distortion parameter fall within the corresponding preset interval range at the same time, the system determines that the lens distortion has been in an acceptable range, thereby starting the image target recognition module.

[0029] Preferably, in some embodiments, the target recognition of the original imaging image to obtain a plurality of recognition targets specifically includes: performing multi-level binarization processing on the original imaging image to obtain a plurality of binarization images; respectively performing boundary contour extraction based on each binarization image to obtain a plurality of boundary contour features, and respectively performing target recognition on each boundary contour feature using a pre-trained target detection model to obtain a plurality of recognition targets.

[0030] In specific implementation, a multi-level threshold segmentation method is used, and the present application preferably uses existing threshold segmentation algorithms such as Otsu algorithm, multi-threshold segmentation, adaptive threshold method, etc. to binarize the gray-scale image, thereby obtaining a plurality of binarization images, each binarization image corresponding to an image region of different gray-scale range. For any one binarization image, the Canny edge detection algorithm is used to extract the boundary contour feature of the binarization image, wherein the boundary contour feature includes a set of contour coordinate points of the target, boundary box information, geometric features (area, aspect ratio, perimeter, etc.), and then the YOLO target detection algorithm commonly used in the prior art for target recognition and classification is used for target recognition. The model predicts the target class and position information according to the boundary contour feature, and outputs the boundary box coordinates, confidence score and class label of each recognition target. For the plurality of detected recognition targets, non-maximum suppression processing can be further performed to remove overlapping or repeatedly detected targets to obtain the final plurality of recognition target information; each recognition target contains its class, boundary contour information, boundary box coordinates and related geometric features.

[0031] Optionally, in some embodiments, before the target recognition of the original imaging image to obtain a plurality of recognition targets, the original imaging image is further subjected to grayscale processing. Optionally, in some embodiments, the recognition target contains target class, position information and boundary contour information.

[0032] In step S103, the multiple recognition targets are subjected to regularized screening to obtain regularized recognition targets, a pixel restoration matrix of the regularized recognition targets is extracted based on a regularized target library, a corresponding region of the original imaging image is subjected to regional reconstruction according to the pixel restoration matrix, and a radial distortion correction parameter and a tangential distortion correction parameter are extracted according to the reconstructed image region.

[0033] Preferably, in some embodiments, the regularized screening of the multiple recognition targets to obtain regularized recognition targets specifically includes: For any one recognition target, a geometric shape feature corresponding to the recognition target is extracted; A target category corresponding to the recognition target is obtained, a standard geometric shape parameter corresponding to the target category is obtained, and a feature contrast is extracted based on the standard geometric shape parameter and the geometric shape feature corresponding to the recognition target; Each recognition target is traversed, and a recognition target with the lowest feature contrast is taken as the regularized recognition target.

[0034] In a specific implementation, for each recognition target, its geometric shape feature is calculated through boundary contour information, the geometric shape feature has multiple feature dimensions, including area, perimeter, aspect ratio, contour convexity, circularity, and rectangularity, wherein the rectangularity is calculated by a ratio of a contour area to a minimum circumscribed rectangle area, and the convexity is calculated by a ratio of a contour area to a convex hull area, which will not be described in detail herein, and then a standard geometric shape parameter corresponding to a category to which the recognition target belongs is obtained from a regularized target library, for example, a circularity of a standard circle is 1, an aspect ratio of a standard rectangle is a preset range, and convexity and rectangularity of a standard polygon are preset values, the actual geometric shape feature of the recognition target is compared with the standard geometric shape parameter of the corresponding category, a feature deviation is calculated as a feature contrast, the feature contrast can be a weighted sum of multiple geometric feature errors, and a weight of each geometric feature dimension can be set according to actual application requirements, in some other embodiments, after feature normalization, a feature vector corresponding to the standard geometric shape parameter and the geometric shape feature corresponding to the recognition target can be constructed, and a cosine similarity between the feature vectors is obtained as the feature contrast, which will not be limited herein, and a recognition target with the lowest feature contrast, that is, a target with a shape closest to a standard geometric shape, is selected as a regularized recognition target.

[0035] It should be noted that the pixel restoration matrix records offset information of each pixel point in the regularized recognition target, and is used for regional reconstruction of a corresponding region of the original imaging image, and preferably, in some embodiments, the pixel restoration matrix of the regularized recognition target extracted based on the regularized target library specifically includes: acquire a regularized target library, match the boundary profile information of the regularized recognition target with a standard geometric template of a corresponding category in the regularized target library; According to the matching result, the position of each pixel of the recognition target in the original image is mapped to the coordinate position of the standard template, and a pixel restoration matrix of the regularized recognition target is generated.

[0036] In a specific implementation, a mapping relationship of the regularized target library is established. In the process of establishing the mapping relationship of the regularized target library, the acquisition and storage of the regularized target can be implemented in various ways. Specifically, on the one hand, the regularized target information can be collected by experimental determination. For example, in a controlled experimental environment, a high-precision calibrated imaging device is used to image standard geometric shape objects (such as circular targets, rectangular targets, and polygon reference plates corresponding to different object types), the standard geometric shape parameters and pixel distribution information of various regularized targets are measured through distortion correction and geometric analysis of the collected images, and a corresponding pixel mapping table is constructed, so that the reference information in the regularized target library has high precision and high consistency, and is suitable for factory calibration of imaging systems or precise application in professional scenarios.

[0037] On the other hand, the system can also support user-side uploading of standard images of commonly used objects, and the user can directly provide reference images of the target category during use. After receiving the standard images uploaded by the user, the system can automatically generate a standard geometric shape template of the target category through image preprocessing and geometric feature extraction algorithms (such as contour detection, shape fitting, and normalization transformation), and extract the pixel coordinate mapping relationship. This makes the regularized target library have stronger adaptability and expandability, and meets the individual needs of different users in different application scenarios.

[0038] The implementation process of the process of extracting the pixel coordinate mapping relationship is as follows: after the user-side uploads the standard image and completes the preprocessing, the system performs grayscale and binarization processing on the standard image, extracts the target contour through an edge detection algorithm, uniformly samples or extracts key feature points on the contour, including corner points, convex hull vertices, and endpoints of contour segments, forms a contour feature point set, for the contour feature points, the system performs shape normalization processing through scale normalization, so that they conform to the preset standard geometric shape size and direction in the coordinate system, for example, aligning the longest side to the horizontal, moving the centroid to the coordinate origin, and scaling according to the standard size, based on the normalized contour feature points, the system one-to-one corresponds them with the standard template pixel coordinates, constructs a preliminary pixel mapping table, and for each pixel point inside the contour, adopts bilinear interpolation to calculate its corresponding coordinates in the standard template, thereby forming the pixel coordinate mapping relationship.

[0039] In some other embodiments, in order to improve the mapping accuracy, the system can also divide the pixels in the inner region of the contour into grid units, perform mapping calculation for each unit respectively, and perform smoothing or interpolation processing on the continuous image region to eliminate the error caused by discrete pixels. Finally, the system stores the generated pixel coordinate mapping relationship into the regularized target library, so that the subsequent reconstruction and distortion correction of the imaging region can accurately adjust the position of each pixel according to the mapping relationship, and high-precision geometric correction and image recovery can be realized.

[0040] In specific implementation, the system can be designed to support a double-channel construction mechanism, that is, both the existing standard target library information loaded from experimental determination and the new target image uploaded by the user end can be automatically standardized and updated, and finally a unified mapping storage format is formed in the regularized target library; so that the regularized target library not only can maintain high-precision standard reference, but also can be dynamically expanded to meet the needs of diversified regularized targets in the real-time distortion correction process, for example, the user needs to regularize the common screw cap on the production line; the user uploads a clear standard image of the screw cap through the client, and the system first performs grayscale and edge detection on the image after receiving the image, extracts the contour features of the screw cap; calculates the geometric parameters of the target through the data processing chip, including the diameter of the circumscribed circle, the roundness, the edge concave-convex degree, etc., and performs normalization processing to generate a standard geometric template of the screw cap category; based on the registration relationship between the screw cap contour and the standard template, a pixel coordinate mapping table is established, and finally a pixel restoration matrix of the screw cap is generated and stored in the regularized target library; in the subsequent imaging process, when the system detects that the target category is a screw cap target, the mapping relationship in the regularized target library can be called for quick pixel restoration and region reconstruction, so as to realize efficient distortion correction.

[0041] The regularized target library stores the standard geometric shape template of each target category, including the ideal position coordinates of each pixel in the standard target; for each target, a pixel mapping table is constructed in advance, the actual image coordinates and the standard target template coordinates are one-to-one corresponding to form a pixel-level mapping relationship, then the boundary contour information of the regularized recognition target is matched with the corresponding standard template in the regularized target library, and the actual contour coordinates of the recognition target are mapped to the standard template coordinates through affine transformation or thin plate spline transformation method, so as to realize the spatial correction of the target pixel position. Based on the matching result, for each pixel point of the regularized recognition target, a pixel restoration matrix is generated according to its corresponding coordinates in the standard template, wherein each element in the matrix represents the mapping relationship from the actual pixel position to the standard template position. The pixel restoration matrix can be represented as a two-dimensional matrix or a coordinate mapping table, and the element value in the matrix preferably saves the offset information of each pixel point (X, Y axes correspond to the pixel offset respectively).

[0042] Optionally, in some embodiments, during the process of generating the pixel restoration matrix of the regularized recognition target, the pixel mapping is interpolated, for example, using bilinear interpolation or bicubic interpolation, to enhance the mapping accuracy and reduce the error caused by discrete pixels. For a large size or irregular shape of the recognition target, the area within the contour can be divided into a grid, and the pixel mapping is calculated for each grid element, thereby obtaining a higher resolution pixel restoration matrix and improving the accuracy of subsequent area reconstruction and optical correction.

[0043] Optionally, in some embodiments, the area reconstruction of the corresponding area of the original imaging image according to the pixel restoration matrix comprises: Obtaining the corresponding area pixels of the regularized recognition target in the original imaging image; Mapping each pixel to the standard template coordinate position according to the offset in the pixel restoration matrix, wherein for non-integer coordinate positions, the pixel value is calculated by interpolation method, and all the mapped pixels form a reconstructed area; Integrating the reconstructed area into the original imaging image to obtain a reconstructed image area.

[0044] In specific implementation, the corresponding area of the original image is obtained, that is, the image area where the regularized recognition target is located is cut from the original imaging image, including the target contour and the internal pixel points, and the pixel coordinates of the area are corresponded to the pixel restoration matrix to form an input set of the area to be reconstructed. For each input pixel, the original coordinates are mapped to the target coordinate position of the standard template according to the offset recorded in the pixel restoration matrix. For non-integer coordinate positions, the gray value or color value of the corresponding pixel can be calculated by interpolation method, for example, bilinear interpolation or bicubic interpolation, and all the mapped pixels are filled into the reconstructed area of the standard geometric shape to obtain the image segment after geometric correction. Optionally, the reconstructed area is subjected to smoothing or edge enhancement processing to correct the edge discontinuity or artifacts generated in the interpolation or mapping process, thereby improving the image quality. Finally, the reconstructed image area is backfilled to the corresponding position of the original imaging image. For the reconstructed areas of multiple regularized recognition targets, they can be integrated in turn according to the position order or priority to form a complete area reconstruction result.

[0045] Preferably, in some embodiments, the process of extracting the radial distortion correction parameter and the tangential distortion correction parameter according to the reconstructed image area is consistent with the process of extracting the distortion feature of the original imaging image in the present application, which will not be described in detail herein.

[0046] In step S104, an optical correction model is constructed based on the radial distortion correction parameter and the tangential distortion correction parameter, and the original imaging image is subjected to distortion correction through the optical correction model to obtain a target imaging image.

[0047] It should be noted that, by the rule-based screening of multiple identification targets, the present application can select a target with the closest shape to the standard template and the most regular geometric feature from the original image as the rule-based identification target. The rule-based identification target itself has a predictable geometric shape and stable pixel distribution feature, which makes it a reliable basis for accurately calculating pixel mapping and distortion parameters. By extracting the pixel restoration matrix based on the rule-based target library, the actual pixels are corresponded to the standard template pixels, which can accurately restore the ideal geometric shape of the target area after the area reconstruction, thereby eliminating the interference of local distortion on parameter calculation, and further extracting the radial distortion correction parameter and the tangential distortion correction parameter based on the reconstructed image area, so that the obtained distortion parameters can reflect the dynamic distortion characteristics under the current acquisition condition, which is more real-time and adaptive than the traditional preset parameters or fixed calibration method, and can be directly used to construct an optical correction model and correct the original imaging image. Finally, the original image is pixel-level mapped and corrected through the optical correction model to obtain a target imaging image, realizing real-time distortion correction. The present application uses the rule-based identification target as the reference for distortion parameter extraction, which can not only automatically adapt to the lens distortion under different acquisition conditions, but also perform real-time optical correction through the dynamically extracted correction parameters, thereby significantly improving the real-time performance and accuracy of the imaging system.

[0048] Preferably, in some embodiments, the process of constructing an optical correction model based on the radial distortion correction parameter and the tangential distortion correction parameter can be achieved by substituting the radial distortion correction parameter and the tangential distortion correction parameter into a preset camera distortion model to construct a pixel-level optical correction function.

[0049] In specific implementation, the radial distortion correction parameter and the tangential distortion correction parameter are substituted into the distortion equation corresponding to the optical correction camera lens, the radial distortion variable and the tangential distortion variable corresponding to each pixel point are determined based on the radial distance of each pixel point relative to the optical center, the correction position corresponding to each pixel point is obtained, and the mapping relationship is saved as a pixel-level optical correction function, which can be stored in the form of a two-dimensional matrix in specific implementation.

[0050] Preferably, in some embodiments, the optical correction model is applied to the original imaging image to obtain a corrected target imaging image. In the pixel mapping process, the corrected pixel coordinates are usually non-integer values. To this end, the present application uses an interpolation method (such as bilinear interpolation and bicubic interpolation) to calculate the gray value, so that the corrected image is smooth and continuous, avoiding the presence of faults, sawtooth and other phenomena in the target imaging image during the correction process.

[0051] Optionally, in some embodiments, after the distortion correction of the original imaging image by the optical correction model to obtain a target imaging image, the present application further comprises outputting the target imaging image.

[0052] Preferably, in some embodiments, outputting the target imaging image includes multiple implementation manners; the corrected target imaging image can be saved as a common image format such as JPEG, PNG, TIFF or RAW file, and the image can be selected to be saved to a local memory, an SD card or a remote server for subsequent processing or archiving; the corrected target imaging image can be displayed through a display device including a liquid crystal display screen, an OLED screen or a monitoring terminal, and in a real-time imaging application, the corrected image can be displayed synchronously, and the real purpose is for an operator to observe and quality detect.

[0053] In addition, the target imaging image can be output to an image processing module, a machine vision system or an artificial intelligence recognition module for further image analysis, feature extraction and target recognition, and image data transmission can be realized through memory sharing, an image buffer or a standard communication interface (such as USB, Ethernet, CameraLink); before output, additional processing can be performed on the corrected image, including grayscale normalization, contrast enhancement, denoising or edge enhancement, to improve image quality and subsequent recognition effect; for a multi-frame image sequence, each frame of image can be corrected and output to form a continuous video stream or an image sequence.

[0054] In addition, another aspect of the present application provides an optical correction camera lens module, which includes an imaging processing unit, as described above with reference to Figure 2 The figure is a structural schematic diagram of an exemplary hardware and / or software of an imaging processing unit according to some embodiments of the present application, which includes an imaging data extraction module 201, an imaging data processing module 202 and an imaging distortion correction module 203, which are described as follows: The imaging data extraction module 201 is configured to acquire an original imaging image of an optical correction camera; and extract distortion features from the original imaging image, wherein the distortion features include a radial distortion parameter and a tangential distortion parameter. The imaging data processing module 202 is configured to perform target recognition on the original imaging image when the radial distortion parameter and the tangential distortion parameter satisfy a preset distortion correction condition, to obtain a plurality of recognized targets. The imaging data processing module 202 is further configured to perform regularized screening on the plurality of recognized targets to obtain regularized recognized targets, extract a pixel restoration matrix of the regularized recognized targets based on a regularized target library, perform regional reconstruction on a corresponding region of the original imaging image according to the pixel restoration matrix, and extract a radial distortion correction parameter and a tangential distortion correction parameter according to the reconstructed image region. The imaging distortion correction module 203 is configured to construct an optical correction model by the radial distortion correction parameter and the tangential distortion correction parameter, and correct the original imaging image by the optical correction model to obtain a target imaging image.

[0055] The above describes an example of the camera lens module and the imaging method in the embodiments of the present application in detail. It can be understood that the corresponding device includes the hardware structure and / or software module for executing the respective functions in order to realize the above functions.

[0056] Those skilled in the art should easily understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function in the present application is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution, and therefore a person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0057] In addition, the present application also provides a computer terminal device, which comprises a memory and a processor, the memory stores a code, and the processor is configured to acquire the code and execute the above-mentioned optical correction camera lens imaging method.

[0058] In some embodiments, with reference to Figure 3 The figure is a structural schematic diagram of a computer terminal device for implementing an optical correction camera lens imaging method according to some embodiments of the present application; the optical correction camera lens imaging method in the above-mentioned embodiments can be implemented by the computer terminal device shown in the figure, which comprises at least one communication bus 301, a communication interface 302, a processor 303 and a memory 304. Figure 3

[0059] The processor 303 can be a general central processing unit (CPU), an application-specific integrated circuit (ASIC) or one or more processors for controlling the execution of the optical correction camera lens imaging method in the present application.

[0060] The communication bus 301 can include a path for transmitting information between the above-mentioned components.

[0061] ​The memory 304 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk or other magnetic storage device, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto; the memory 304 can exist independently, and is connected to the processor 303 through the communication bus 301; the memory 304 can also be integrated with the processor 303.

[0062] The memory 304 is configured to store program code for implementing the scheme of the present application, and the processor 303 is configured to control the execution of the program code stored in the memory 304; the program code can include one or more software modules; the determination of the distortion feature in the above embodiments can be implemented by one or more software modules in the program code of the processor 303 and the memory 304.

[0063] The communication interface 302 is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., using any transceiver-like device.

[0064] Optionally, the computer terminal device 300 can further include a power supply 305 configured to provide power supply for various devices or circuits in the real-time computer terminal device.

[0065] In a specific implementation, as an embodiment, the computer terminal device can include a plurality of processors, each of which can be a single-CPU processor or a multi-CPU processor; the processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0066] The computer terminal device described above can be a general-purpose computer terminal device or a special-purpose computer terminal device; in specific implementation, the computer terminal device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device; the embodiments of the present application do not limit the type of computer terminal device.

[0067] In addition, the computer readable storage medium of other aspects of the present application stores at least one computer program, which is loaded and executed by the processor to implement the operations performed by the above-mentioned optical correction camera lens imaging method.

[0068] In summary, the optical correction camera lens module and the imaging method thereof disclosed in the embodiments of the present application first acquire an original imaging image of the optical correction camera; extract distortion features from the original imaging image, including radial distortion parameters and tangential distortion parameters; when the radial distortion parameters and the tangential distortion parameters meet the preset distortion correction condition, perform target recognition on the original imaging image to obtain a plurality of recognized targets; perform regularized screening on the plurality of recognized targets to obtain regularized recognized targets, extract a pixel restoration matrix of the regularized recognized targets based on a regularized target library, reconstruct a region of the original imaging image according to the pixel restoration matrix, and extract radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region; construct an optical correction model based on the radial distortion correction parameters and the tangential distortion correction parameters, and perform distortion correction on the original imaging image through the optical correction model to obtain a target imaging image; the regularized recognized targets can be extracted from the original imaging image, and dynamic distortion correction parameters can be extracted through the regularized recognized targets, thereby improving the real-time performance of imaging distortion correction.

[0069] The above is only an embodiment of the present application, and common knowledge such as specific technical solutions or characteristics in the scheme is not described in detail here; it should be noted that, for those skilled in the art, without departing from the technical solutions of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application, and these will not affect the effect and practicality of the patent implementation.

[0070] The scope of protection of the present application should be subject to the content of its claims, and the specific implementation manner and the like in the specification can be used to explain the content of the claims; obviously, those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application; thus, if these modifications and changes of the present application belong to the scope of the claims of the present application and its equivalent technology, the present application also intends to include these modifications and changes.

Claims

1. A method for optically correcting camera lens imaging, characterized in that, include: Acquire the raw imaging image from the optically corrected camera; extract distortion features from the raw imaging image, the distortion features including radial distortion parameters and tangential distortion parameters; When the radial distortion parameter and the tangential distortion parameter meet the preset distortion correction conditions, target recognition is performed on the original imaging image to obtain multiple recognized targets; Multiple recognition targets are filtered according to rules to obtain regular recognition targets. The pixel reconstruction matrix of the regular recognition targets is extracted based on the regular recognition target library. The pixel reconstruction matrix records the offset information of each pixel in the regular recognition target. The corresponding region of the original imaging image is reconstructed according to the pixel reconstruction matrix. Radial distortion correction parameters and tangential distortion correction parameters are extracted from the reconstructed image region. An optical correction model is constructed based on the radial distortion correction parameters and the tangential distortion correction parameters. The original imaging image is then distorted using the optical correction model to obtain the target imaging image.

2. The method as described in claim 1, characterized in that, Raw image data is acquired by the image sensor corresponding to the optically corrected camera, and the raw image data is output to the image processing module through the camera driver interface to form a raw imaging image.

3. The method as described in claim 1, characterized in that, Target recognition is performed on the original imaging image to obtain multiple recognized targets, specifically including: The original imaging image is subjected to multi-level binarization processing to obtain multiple binarized images; Boundary contours are extracted from each binarized image to obtain multiple boundary contour features. A pre-trained target detection model is then used to identify targets for each boundary contour feature, resulting in multiple identified targets.

4. The method as described in claim 1, characterized in that, Before performing target recognition on the original imaging image to obtain multiple recognized targets, the method further includes: performing grayscale processing on the original imaging image.

5. The method as described in claim 1, characterized in that, By performing rule-based filtering on multiple identification targets, the specific rule-based identification targets include: For any target to be identified, extract the geometric shape features corresponding to that target. Obtain the target category corresponding to the identified target, obtain the standard geometric shape parameters corresponding to the target category, and extract feature contrast based on the standard geometric shape parameters and the geometric shape features corresponding to the identified target; Iterate through each recognition target and select the recognition target with the lowest feature contrast as the regularized recognition target.

6. The method as described in claim 1, characterized in that, Extracting the pixel reconstruction matrix of the regularized target based on the regularized target library specifically includes: Obtain a rule-based target library, and match the boundary contour information of the rule-based recognition target with the standard geometric template of the corresponding category in the rule-based target library; Based on the matching results, the position of each pixel of the target in the original image is mapped to the coordinate position of the standard template to generate the pixel reconstruction matrix of the regularized target.

7. The method as described in claim 1, characterized in that, Based on the pixel restoration matrix, region reconstruction is performed on the corresponding region of the original imaging image, including: Obtain the corresponding region pixels of the regularized target in the original imaging image; Each pixel is mapped to a standard template coordinate position based on the offset in the pixel reconstruction matrix. For non-integer coordinate positions, the pixel value is calculated by interpolation. All mapped pixels are then used to form a reconstructed region. The reconstructed region is integrated into the original imaging image to obtain the reconstructed image region.

8. An optical correction camera lens module, comprising an imaging processing unit, said imaging processing unit being used to execute the optical correction camera lens imaging method according to any one of claims 1 to 7, characterized in that, The imaging processing unit includes: An imaging data extraction module is used to acquire the original imaging image of an optically corrected camera; and to extract distortion features from the original imaging image, wherein the distortion features include radial distortion parameters and tangential distortion parameters. An imaging data processing module is used to perform target recognition on the original imaging image when the radial distortion parameter and the tangential distortion parameter meet preset distortion correction conditions, thereby obtaining multiple recognized targets. The imaging data processing module is also used to perform regularized filtering on multiple recognition targets to obtain regularized recognition targets, extract the pixel restoration matrix of the regularized recognition targets based on the regularized target library, reconstruct the corresponding region of the original imaging image according to the pixel restoration matrix, and extract radial distortion correction parameters and tangential distortion correction parameters according to the reconstructed image region. An imaging distortion correction module is used to construct an optical correction model using the radial distortion correction parameters and the tangential distortion correction parameters, and to perform distortion correction on the original imaging image using the optical correction model to obtain the target imaging image.

9. A computer terminal device, characterized in that, The computer terminal device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute an optical correction camera lens imaging method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing at least one computer program, characterized in that, The computer program is loaded and executed by a processor to perform the operations of an optical correction camera lens imaging method as described in any one of claims 1 to 7.

Images