A sensor-based optical lens mounting detection method and system

Abstract

The application relates to the technical field of optical lens detection, and particularly discloses an optical lens installation detection method and system based on a sensor, which comprises the following steps: setting an image sensor device, a lens installation station and a calibration device in an optical lens detection scene, arranging a checkerboard calibration reference plate and a plurality of position-fixed fixed reference balls in the calibration device, setting a detection reference point to obtain internal calibration information of a camera, combining pixel coordinates of the fixed reference balls, calculating and storing three-dimensional reference coordinates of the fixed reference balls in a workpiece coordinate system, calculating verification coordinates of the fixed reference balls after a set number of optical lens calibration procedures are completed, comparing the reference coordinates with the verification coordinates, and judging whether the image sensor device or the checkerboard calibration reference plate has a position deviation, so that the problems that calibration data is invalid, detection results are inaccurate and substandard products / excellent products are misjudged due to a slight position deviation of the image sensor device or the checkerboard calibration reference plate in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of optical lens inspection technology, and more specifically, to a sensor-based optical lens installation inspection method and system. Background Technology

[0002] In the manufacturing of precision optical equipment, the accurate mounting of optical lenses is crucial. Traditional inspection methods are insufficient to meet the high-precision requirements of modern industry. Therefore, the industry widely adopts sensor-based inspection systems to evaluate the mounting quality of optical lenses in real time and with high accuracy. Such systems typically include an image sensor device, a lens mounting station, and a calibration device, where the calibration device contains a checkerboard calibration reference board. By acquiring and analyzing images of the checkerboard calibration reference board, the system determines the camera's internal calibration information for the image sensor device, including internal parameters (such as intrinsic parameter matrices and distortion coefficients) and external parameters (such as rotation matrices and translation vectors). This information forms the basis for all subsequent precise measurements.

[0003] However, in actual industrial production environments, image sensor devices or checkerboard calibration reference boards may experience minute positional shifts that are difficult to detect with the naked eye due to accidental impacts or maintenance operations. This shift causes the internal calibration information of the image sensor device to no longer accurately reflect its true spatial orientation, rendering the mathematical transformation relationships upon which the device relies for precise measurements invalid. When an image sensor device or checkerboard calibration reference board experiences positional shifts, the conversion from image points to actual spatial positions based on old, outdated calibration data becomes systematically inaccurate, making it difficult to distinguish between qualified and unqualified components.

[0004] There is currently no effective technical solution to the above problems. Summary of the Invention

[0005] This application aims to solve the technical problems mentioned in the background art by providing a sensor-based optical lens mounting detection method and system to address the technical problem in precision optical equipment manufacturing where image sensor devices or checkerboard calibration reference boards experience slight positional shifts due to accidental collisions or maintenance operations, leading to calibration data failure and subsequent misjudgment of defective / superior products.

[0006] To solve the above problems, the solution proposed in this application is as follows:

[0007] As one aspect of this application, a sensor-based optical lens mounting detection method is provided to identify whether an image sensor device or a checkerboard calibration reference plate used in an optical lens detection application has experienced a positional shift, comprising the following steps:

[0008] Step S1: Provide an optical lens detection scenario. In the optical lens detection scenario, set up an image sensor device, a lens mounting station and a calibration device. The calibration device includes a checkerboard calibration reference plate and multiple fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position.

[0009] Step S2: Establish a detection reference point on the checkerboard calibration reference plate, and obtain the camera internal calibration information in the image sensor device based on the detection reference point. The camera internal calibration information includes internal parameters of the intrinsic parameter matrix and distortion coefficients and external parameters including rotation matrix and translation vector.

[0010] Step S3: Obtain an image containing multiple fixed reference spheres, and identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres;

[0011] Step S4: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system, and use them as reference coordinates.

[0012] Step S5: After completing the set number of optical lens calibration processes, use the pixel coordinates of each fixed reference sphere and the camera's internal calibration information to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing the set number of optical lens calibration processes, and use them as verification coordinates.

[0013] Step S6: Compare the reference coordinates with the verification coordinates and obtain the comparison results. Based on the comparison results, determine whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

[0014] Furthermore, step S2 specifically includes:

[0015] Step S21: Use a checkerboard calibration reference board and set the corner points on the checkerboard calibration reference board as detection reference points;

[0016] Step S22: Use an industrial camera to acquire multiple images of the checkerboard calibration reference board in different poses. Use the checkerboard corner detection function in the OpenCV library to identify the detection reference points in each acquired image of the checkerboard calibration reference board and obtain the corresponding pixel coordinates.

[0017] Step S23: Using the pixel coordinates corresponding to the detection reference point in each image of the checkerboard calibration reference board, solve for the internal parameters including the intrinsic parameter matrix and distortion coefficients, and the external parameters including the rotation matrix and translation vector using the least squares method.

[0018] Furthermore, step S3 specifically includes:

[0019] Step S31a: Obtain an image containing multiple fixed reference spheres;

[0020] Step S32a: The Canny edge detection algorithm is used to identify the edge contours of each fixed reference sphere in the image, which contains multiple fixed reference spheres. Based on the edge contours of each fixed reference sphere, the least squares circle fitting algorithm is used to calculate the pixel coordinates of the center of each fixed reference sphere.

[0021] Furthermore, step S32a also includes:

[0022] The edge contours of each fixed reference sphere were corrected using the Zernike moment fitting method.

[0023] Furthermore, step S4 specifically includes:

[0024] Step S41a: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere into three-dimensional coordinates of each fixed reference sphere in the camera coordinate system.

[0025] Step S42a: Based on the pixel coordinates of each fixed reference sphere and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in the image and the camera's pose.

[0026] Step S43a: Using the triangulation method, based on the position of each fixed reference ball in the image and the camera posture, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system are calculated in reverse, and these coordinates are used as reference coordinates.

[0027] Furthermore, step S3 specifically includes:

[0028] Acquire multiple consecutive frames of images containing multiple fixed reference spheres, identify each frame containing multiple fixed reference spheres, and obtain the pixel coordinates of each fixed reference sphere in each frame containing multiple fixed reference spheres.

[0029] Furthermore, step S4 specifically includes:

[0030] Step S41b: Using the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres, combined with the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres into the three-dimensional coordinates of each fixed reference sphere in the camera coordinate system.

[0031] Step S42b: Based on the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres in its respective corresponding image and the camera pose.

[0032] Step S43b: Using the triangulation method, based on the position of each fixed reference ball in its corresponding image and the camera pose, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system in each frame of the image containing multiple fixed reference balls, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system in each frame of the image containing multiple fixed reference balls are calculated in reverse.

[0033] Step S44b: Extract the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system in each frame of the image containing multiple fixed reference spheres according to the one-to-one correspondence between the multiple fixed reference spheres in each frame of the image, and obtain the X coordinate value sequence, Y coordinate sequence and Z coordinate sequence for each fixed reference sphere.

[0034] Step S45b: Apply median filtering to the X-coordinate value sequence, Y-coordinate sequence, and Z-coordinate sequence of each fixed reference sphere and extract the median value to obtain the X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere. Use the obtained X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere as the reference coordinates.

[0035] Furthermore, step S6 specifically includes:

[0036] Step S61: Calculate the Euclidean distance between the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system and the reference coordinates after completing a set number of optical lens calibration processes, and store it in a structured manner to form a distance dataset.

[0037] Step S62: Analyze the distance dataset to obtain the analysis results of whether the image sensor device or the checkerboard calibration reference board has shifted position. When all values ​​in the distance dataset are greater than the set distance threshold, the result of the image sensor device or the checkerboard calibration reference board shifting position is obtained.

[0038] Furthermore, after step S6, the method further includes:

[0039] Step S7: When it is determined that the image sensor device or the checkerboard calibration reference plate has shifted position, feedback information is obtained based on the comparison results. The feedback information includes a suggested lens detection stop instruction, a suggested image sensor device recalibration instruction, or provides the time of the anomaly and the deviation data indicated by the fixed reference ball.

[0040] As a second aspect of this application, a sensor-based optical lens mounting and inspection system is provided. This system is applied in an optical lens inspection scenario, which includes an image sensor device, a lens mounting station, and a calibration device. The calibration device comprises a checkerboard calibration reference plate and multiple fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position.

[0041] The camera internal calibration information acquisition module is used to acquire camera internal calibration information in the image sensor device by establishing a detection reference point on a checkerboard calibration reference plate. The camera internal calibration information includes internal parameters of the intrinsic parameter matrix and distortion coefficients and external parameters including rotation matrix and translation vector.

[0042] The first pixel coordinate acquisition module is used to acquire an image containing multiple fixed reference spheres and identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres.

[0043] The reference coordinate acquisition module is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system by combining the pixel coordinates of each fixed reference sphere with the camera's internal calibration information, and use them as reference coordinates.

[0044] The verification coordinate acquisition module is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing a set number of optical lens calibration processes, using the pixel coordinates of each fixed reference sphere and the internal calibration information of the camera, and use them as verification coordinates.

[0045] The judgment and analysis module is used to compare the reference coordinates with the verification coordinates and obtain the comparison result. Based on the comparison result, it determines whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

[0046] As described above, this application provides a sensor-based optical lens installation inspection method and system. By setting up an image sensor device, a lens installation station, and a calibration device in the optical lens inspection scenario, and arranging a checkerboard calibration reference board and multiple fixed reference spheres in the calibration device, accurate identification of positional offsets of the image sensor device or the checkerboard calibration reference board is achieved. First, the internal calibration information of the camera is obtained by establishing a detection reference point. Then, the pixel coordinates of the fixed reference spheres are combined with the internal calibration information of the camera to calculate and store their three-dimensional reference coordinates in the workpiece coordinate system. After completing a set number of optical lens calibration processes, the three-dimensional verification coordinates of the fixed reference spheres are calculated again. By comparing the reference coordinates with the verification coordinates, it is determined whether the image sensor device or the checkerboard calibration reference board has experienced positional offset. This solves the problem in the prior art where slight positional offsets of the image sensor device or the checkerboard calibration reference board lead to calibration data failure, inaccurate detection results, and misjudgment of defective / good products. By introducing fixed reference spheres as a reference object, this solution can monitor the spatial attitude changes of key inspection equipment, avoiding the impact of subtle offsets that are difficult to detect in traditional methods on the detection accuracy, and ensuring the accuracy of optical lens installation inspection. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a sensor-based optical lens mounting detection method provided in this application embodiment;

[0048] Figure 2 A system structure block diagram of a sensor-based optical lens mounting and detection system provided in this application embodiment;

[0049] Reference numerals: 100, Optical lens mounting detection system; 101, Camera internal calibration information acquisition module; 102, First pixel coordinate acquisition module; 103, Reference coordinate acquisition module; 104, Verification coordinate acquisition module; 105, Judgment and analysis module. Detailed Implementation

[0050] To better illustrate the present invention, the invention will now be described in further detail with reference to the accompanying drawings.

[0051] It should be understood that, in order to make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0052] The following description uses at least one specific embodiment as an example. In this embodiment:

[0053] Firstly, such as Figure 1 As shown, a sensor-based optical lens mounting detection method is provided to identify whether an image sensor device or a checkerboard calibration reference plate used in optical lens detection applications has experienced positional shift. The method includes the following steps:

[0054] Step S1: Provide an optical lens detection scenario. In the optical lens detection scenario, set up an image sensor device, a lens mounting station and a calibration device. The calibration device includes a checkerboard calibration reference plate and multiple fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position.

[0055] Step S2: Establish a detection reference point on the checkerboard calibration reference board. Based on this detection reference point, obtain the camera's internal calibration information from the image sensor device. This internal calibration information includes intrinsic parameters such as the intrinsic matrix and distortion coefficients, and extrinsic parameters including the rotation matrix and translation vector. For example, a specific corner point on the checkerboard calibration reference board can be manually selected as the detection reference point. Then, the image sensor device acquires multiple images of the checkerboard calibration reference board at different angles and distances. Using these images and the selected detection reference point, a standard camera calibration algorithm (such as the Zhang Zhengyou calibration method) can be used to calculate the camera's intrinsic matrix, distortion coefficients, rotation matrix, and translation vector. These parameters together constitute the camera's internal calibration information, which is the basis for subsequent 3D coordinate calculations. Step S3: Acquire an image containing multiple fixed reference spheres. Identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres. For example, the image sensor device can capture an image containing all the fixed reference spheres. Then, an image processing algorithm can be used to identify the fixed reference spheres in the image. For example, by setting a brightness threshold for the image, fixed reference spheres in the image can be separated from the background. Then, connected component analysis is performed on the separated regions to identify the contours of each fixed reference sphere. Next, the geometric center of each contour can be calculated and used as the pixel coordinates of the fixed reference sphere.

[0056] Step S4: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate the 3D coordinates of each fixed reference sphere in the workpiece coordinate system, and use these as reference coordinates. For example, using the intrinsic parameter matrix and distortion coefficients in the camera's internal calibration information, convert the pixel coordinates of the fixed reference spheres into 3D coordinates in the camera coordinate system. Then, combining the rotation matrix and translation vector (i.e., extrinsic parameters) in the camera's internal calibration information, further transform the 3D coordinates in the camera coordinate system to the workpiece coordinate system. These 3D coordinates calculated in the initial stable state are defined as reference coordinates and used for subsequent position offset determination.

[0057] Step S5: After completing the set number of optical lens calibration processes, the pixel coordinates of each fixed reference sphere are combined with the camera's internal calibration information to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing the set number of optical lens calibration processes. These coordinates are then used as verification coordinates. For example, on the production line, the system automatically triggers a detection after a certain number of optical lenses are installed and calibrated. At this time, the image sensor device will again acquire an image including the fixed reference spheres and repeat the calculation process of steps S3 and S4 to obtain the three-dimensional coordinates of the fixed reference spheres in the workpiece coordinate system at the current moment. These newly calculated three-dimensional coordinates are the verification coordinates.

[0058] Step S6: Compare the reference coordinates with the verification coordinates and obtain the comparison results. Based on the comparison results, determine whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

[0059] By introducing multiple fixed reference spheres with fixed positions in the optical lens inspection scenario and combining them with the camera's internal calibration information from the image sensor device, a stable spatial reference system is established. In step S1, the image sensor device, lens mounting station, and calibration device are appropriately configured to lay the foundation for subsequent image acquisition and coordinate calculation. The checkerboard calibration reference board and the fixed reference spheres in the calibration device work together, with the checkerboard calibration reference board primarily used to acquire the camera's internal calibration information. In step S2, by establishing detection reference points and acquiring the camera's internal calibration information based on them, it is ensured that the image sensor device can accurately convert two-dimensional image information into three-dimensional spatial information. Subsequently, in step S3, the system acquires an image including the fixed reference spheres and identifies the pixel coordinates of each fixed reference sphere. These pixel coordinates are the two-dimensional projection positions of the fixed reference spheres on the image plane. In step S4, using these pixel coordinates and the previously acquired camera internal calibration information, the system can accurately calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system and use them as reference coordinates. These reference coordinates represent the system's reference position in a stable state.

[0060] In actual production, to monitor whether the image sensor device or the checkerboard calibration reference plate has shifted position, the system will execute step S5 again after completing a set number of optical lens calibration procedures. At this time, the system will re-acquire the image of the fixed reference sphere and calculate its three-dimensional coordinates in the workpiece coordinate system, using these as verification coordinates. Finally, in step S6, the system compares the reference coordinates with the verification coordinates. By comparing these coordinates, the system can quantitatively assess whether the image sensor device or the checkerboard calibration reference plate has shifted position. If there is a significant difference between the two, it indicates that the device may have shifted, requiring corresponding adjustments or recalibration. The entire process forms a closed-loop detection mechanism, ensuring the continuous accuracy of optical lens installation detection.

[0061] In this embodiment, multiple fixed reference spheres are arranged at fixed positions around the outer periphery of the checkerboard calibration reference plate, and the reference coordinates of these fixed reference spheres in the workpiece coordinate system are obtained under the initial stable state of the system. During subsequent production, the verification coordinates of the fixed reference spheres are obtained periodically or after a certain number of calibration steps. By comparing the reference coordinates and the verification coordinates, it is possible to intuitively and quantitatively determine whether the image sensor device or the checkerboard calibration reference plate has experienced a positional shift.

[0062] Furthermore, in step S2 above, the process of acquiring the camera's internal calibration information can be further refined into the following steps, specifically including:

[0063] Step S21: Use a checkerboard calibration reference board and set the corner points on the checkerboard calibration reference board as detection reference points;

[0064] Step S22: Use an industrial camera to acquire multiple images of the checkerboard calibration reference board in different poses. Use the checkerboard corner detection function in the OpenCV library to identify the detection reference points in each acquired image of the checkerboard calibration reference board and obtain the corresponding pixel coordinates.

[0065] Step S23: Using the pixel coordinates corresponding to the detection reference point in each image of the checkerboard calibration reference board, solve for the internal parameters including the intrinsic parameter matrix and distortion coefficients, and the external parameters including the rotation matrix and translation vector using the least squares method.

[0066] Specifically, in step S21, a checkerboard calibration reference board is selected as the calibration tool, and its corner points are set as detection reference points. In step S22, an industrial camera is used to acquire multiple images of the checkerboard calibration reference board in different poses. By changing the relative position and orientation of the checkerboard calibration reference board or the industrial camera, images of the checkerboard calibration reference board can be obtained from multiple perspectives, which is crucial for accurately calculating camera parameters. Subsequently, the checkerboard corner detection function in the OpenCV library is used to process these acquired images to identify and extract the pixel coordinates of the detection reference points (i.e., checkerboard corner points) in each image, thereby obtaining high-precision pixel coordinate data. Based on this, in step S23, the camera's internal calibration information is solved using the pixel coordinates corresponding to the detection reference points in each image of the checkerboard calibration reference board, employing the least squares method. This camera's internal calibration information includes internal parameters such as the intrinsic parameter matrix and distortion coefficients, as well as external parameters including rotation matrices and translation vectors. The least squares method is a commonly used optimization algorithm that estimates model parameters by minimizing the sum of squared errors between observed data and model predicted data. In this scenario, it is used to calculate the camera's intrinsic and extrinsic parameters from a large amount of pixel coordinate data to describe the geometric characteristics of the camera's imaging and its position and orientation in the world coordinate system.

[0067] In this embodiment, by utilizing the known geometric characteristics of a checkerboard calibration reference board, combined with images acquired by an industrial camera in different poses, and employing image processing algorithms (such as the checkerboard corner detection function in the OpenCV library) and mathematical optimization methods (such as the least squares method), the camera's internal calibration information is systematically obtained. The corners of the checkerboard calibration reference board serve as high-precision detection benchmarks, providing a stable reference for the camera. Multiple images in different poses ensure the diversity of calibration data, thereby enabling a more comprehensive capture of the camera's imaging characteristics. By fitting these data using the least squares method, the camera's intrinsic parameter matrix, distortion coefficients, rotation matrix, and translation vector are solved. These parameters form the basis for subsequent 3D reconstruction and position detection.

[0068] Furthermore, step S3 can be further refined into the following steps, specifically including:

[0069] Step S31a: Obtain an image containing multiple fixed reference spheres;

[0070] Step S32a: The Canny edge detection algorithm is used to identify the edge contours of each fixed reference sphere in the image, which contains multiple fixed reference spheres. Based on the edge contours of each fixed reference sphere, the least squares circle fitting algorithm is used to calculate the pixel coordinates of the center of each fixed reference sphere.

[0071] Step S31a involves acquiring digital images of multiple fixed reference spheres within the calibration device using an image sensor. These images form the basis for subsequent reference sphere position recognition. Specifically, in step S32a, the Canny edge detection algorithm, a multi-level edge detection algorithm, aims to identify regions with drastic brightness changes in the image, thereby extracting the edge contours of the fixed reference spheres. Further, after acquiring the edge contours of each fixed reference sphere, a least-squares circle fitting algorithm is used to process these contours. The least-squares circle fitting algorithm is a commonly used geometric fitting method. Its purpose is to determine the center and radius of the optimal fitting circle by minimizing the sum of the squared distances from data points to the fitted circle, thereby calculating the pixel coordinates of the centers of each fixed reference sphere and providing accurate two-dimensional position information for subsequent three-dimensional coordinate calculations.

[0072] The proposed solution first acquires an image containing a fixed reference sphere, providing a data source for subsequent image processing. Then, the Canny edge detection algorithm is used to process the image. This algorithm detects edges in the image by calculating the gradient intensity and direction. It effectively filters out noise and accurately locates the boundary of the fixed reference sphere, resulting in a clear edge contour. Based on this, a least-squares circle fitting algorithm is employed. This algorithm uses mathematical optimization to fit an optimal circle from the detected edge points and determines the center point of that circle. Since the fixed reference sphere is typically circular, this fitting method effectively eliminates minor errors that may exist during edge detection, thereby accurately determining the center position of the fixed reference sphere in the image, i.e., its pixel coordinates.

[0073] In some of the above embodiments, in step S32a, the Canny edge detection algorithm is used to identify the edge contours of each fixed reference sphere in the image, which includes multiple fixed reference spheres. Based on these edge contours, the pixel coordinates of the center of each fixed reference sphere are calculated using a least-squares circle fitting algorithm. However, in actual optical lens detection scenarios, the image may be affected by noise, uneven illumination, or defects in the fixed reference spheres themselves, resulting in irregularities or noise in the identified edge contours. If these problems are not addressed, these irregular or noisy edge contours may reduce the accuracy of the least-squares circle fitting, thereby affecting the accuracy of the fixed reference sphere pixel coordinates, and ultimately potentially leading to errors in subsequent 3D coordinate calculations and position offset judgments.

[0074] To address this, an optimization scheme is proposed that improves the calculation accuracy of the center pixel coordinates of the fixed reference sphere by correcting the edge contour.

[0075] Specifically, step S32a further includes:

[0076] The edge contours of each fixed reference sphere were corrected using the Zernike moment fitting method.

[0077] The Zernike moment fitting method is an image moment analysis technique based on orthogonal polynomials, characterized by rotation invariance and robustness to noise. By projecting the edge contour information of an image onto a Zernike polynomial basis, a set of Zernike moments can be obtained. These moments can effectively describe the shape features of the image and exhibit good invariance to image translation, rotation, and scaling. In this application, the Zernike moment fitting method is used to correct the edge contours of various fixed reference spheres initially identified by the Canny edge detection algorithm. The correction process aims to eliminate noise and irregularities in the edge contours, making them closer to ideal circles, thereby providing more accurate input data for subsequent least-squares circle fitting.

[0078] The above technical solution effectively overcomes the adverse effects of image noise and edge irregularities when identifying the edge contour of a fixed reference sphere and calculating its pixel coordinates. The Zernike moment fitting method's correction of the edge contour allows subsequent least-squares circle fitting to be based on smoother and more accurate edge data, significantly improving the calculation accuracy of the fixed reference sphere's center pixel coordinates. This improved accuracy is crucial for subsequent 3D coordinate calculations, ensuring the accuracy of both reference and verification coordinates. This, in turn, enhances the reliability and sensitivity of judging the positional offset of image sensor devices or checkerboard calibration reference boards, avoiding misjudgments or omissions caused by measurement errors.

[0079] In some preferred embodiments, the implementation is as follows: First, an image containing multiple fixed reference spheres is acquired using an industrial camera. Next, the image is processed using the Canny edge detection algorithm to initially identify the edge contours of each fixed reference sphere. After obtaining these preliminary edge contours, the set of edge pixels of each fixed reference sphere is used as input, and a Zernike moment fitting method is applied. This method calculates the Zernike moments of the edge contours and uses these moments to reconstruct or correct the edge contours, making them smoother and more regular, and removing local irregularities caused by noise or image acquisition defects. For example, the order of a Zernike polynomial can be set, and the corrected edge curve can be obtained through fitting. Finally, these edge contours corrected by Zernike moment fitting are used as input, and a least-squares circle fitting algorithm is used to accurately calculate the pixel coordinates of the center of each fixed reference sphere.

[0080] Furthermore, step S4 specifically includes:

[0081] Step S41a: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere into three-dimensional coordinates of each fixed reference sphere in the camera coordinate system.

[0082] Step S42a: Based on the pixel coordinates of each fixed reference sphere and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in the image and the camera's pose.

[0083] Step S43a: Using the triangulation method, based on the position of each fixed reference ball in the image and the camera posture, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system are calculated in reverse, and these coordinates are used as reference coordinates.

[0084] Specifically, in step S41a, the pixel coordinates of the fixed reference sphere identified in the image are converted into three-dimensional coordinates in the camera coordinate system using the intrinsic parameter matrix and distortion coefficients in the camera's internal calibration information. This conversion process utilizes the geometric model of camera imaging to map two-dimensional image points to the camera coordinate system in three-dimensional space. Step S42a can be understood as using the extrinsic parameters (rotation matrix and translation vector) in the camera's internal calibration information, combined with the pixel position of the fixed reference sphere in the image, to determine the camera's attitude relative to the workpiece coordinate system. This facilitates the subsequent conversion of the three-dimensional coordinates in the camera coordinate system to a unified workpiece coordinate system. In practical applications, step S43a employs triangulation, aiming to reverse-calculate the precise three-dimensional coordinates of the fixed reference sphere in the workpiece coordinate system using image data acquired from different viewpoints or at different time points, combined with the known camera attitude and the three-dimensional coordinates in the camera coordinate system.

[0085] In some of the above embodiments, step S3 involves acquiring an image containing multiple fixed reference spheres and identifying their pixel coordinates. However, the acquisition of a single image may be affected by factors such as instantaneous noise, changes in illumination, or camera shake, resulting in certain random errors in the acquired pixel coordinates, which in turn affects the accuracy and stability of subsequent 3D coordinate calculations.

[0086] In this embodiment, step S3 is further proposed above, specifically including:

[0087] Acquire multiple consecutive frames of images containing multiple fixed reference spheres, identify each frame containing multiple fixed reference spheres, and obtain the pixel coordinates of each fixed reference sphere in each frame containing multiple fixed reference spheres.

[0088] Specifically, acquiring multiple consecutive frames of images containing multiple fixed reference spheres refers to an image sensor device continuously acquiring multiple images at a certain frame rate over a period of time. These images all contain multiple fixed reference spheres from a calibration device. This continuous acquisition method aims to capture more visual information, providing a richer data foundation for subsequent recognition and computation. Recognizing each frame containing multiple fixed reference spheres means performing a fixed reference sphere recognition algorithm, such as edge detection, on each independent frame, thereby independently determining the pixel coordinates of each fixed reference sphere in each frame. This yields a time-series pixel coordinate data, rather than pixel coordinates at a single moment.

[0089] The solution proposed in this application acquires multiple consecutive frames of images and independently identifies the pixel coordinates of a fixed reference sphere for each frame. This effectively addresses the interference that may occur when acquiring a single image. Subsequently, statistical processing, such as median filtering, can be performed on these multiple frames of data to reduce the impact of random errors.

[0090] In some preferred embodiments, the image sensor device can continuously acquire 10 frames of images at a rate of 30 frames per second. In each frame, the edge contours of each fixed reference sphere are identified using the Canny edge detection algorithm, and the pixel coordinates of the center of each fixed reference sphere are calculated based on these contours using a least-squares circle fitting algorithm. For example, for a specific fixed reference sphere, 10 sets of pixel coordinates (x_i, y_i) will be obtained in the 10 frames, where i ranges from 1 to 10. These multi-frame pixel coordinate data can then be used for further filtering or averaging to obtain more accurate final pixel coordinates.

[0091] In some of the above embodiments, a method is proposed to acquire multiple consecutive frames of images containing multiple fixed reference spheres, and to identify the pixel coordinates of each fixed reference sphere in each frame. However, in practical applications, due to factors such as changes in ambient lighting, camera noise, vibration, or occasional interference, the pixel coordinates identified in a single frame may have certain errors or fluctuations. If these potentially erroneous pixel coordinates are directly used for 3D coordinate calculation, the stability and accuracy of the final reference coordinates may be affected.

[0092] In this embodiment, step S4 is further proposed above, specifically including:

[0093] Step S41b: Using the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres, combined with the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres into the three-dimensional coordinates of each fixed reference sphere in the camera coordinate system.

[0094] Step S42b: Based on the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres in its respective corresponding image and the camera pose.

[0095] Step S43b: Using the triangulation method, based on the position of each fixed reference ball in its corresponding image and the camera pose, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system in each frame of the image containing multiple fixed reference balls, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system in each frame of the image containing multiple fixed reference balls are calculated in reverse.

[0096] Step S44b: Extract the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system in each frame of the image containing multiple fixed reference spheres according to the one-to-one correspondence between the multiple fixed reference spheres in each frame of the image, and obtain the X coordinate value sequence, Y coordinate sequence and Z coordinate sequence for each fixed reference sphere.

[0097] Step S45b: Apply median filtering to the X-coordinate value sequence, Y-coordinate sequence, and Z-coordinate sequence of each fixed reference sphere and extract the median value to obtain the X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere. Use the obtained X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere as the reference coordinates.

[0098] Specifically, in step S41b, the pixel coordinates of the fixed reference sphere identified in each frame of the image are converted into three-dimensional coordinates in the camera coordinate system using the intrinsic parameter matrix and distortion coefficients in the camera's internal calibration information. This conversion process utilizes the geometric model of camera imaging to map two-dimensional image points to the camera coordinate system in three-dimensional space. Step S42b can be understood as using the extrinsic parameters (rotation matrix and translation vector) in the camera's internal calibration information, combined with the pixel position of the fixed reference sphere in the image, to determine the camera's attitude relative to the workpiece coordinate system. This helps in the subsequent conversion of the three-dimensional coordinates in the camera coordinate system to a unified workpiece coordinate system. In practical applications, step S43b employs triangulation, aiming to reverse-calculate the precise three-dimensional coordinates of the fixed reference sphere in the workpiece coordinate system using image data acquired from different viewpoints or at different time points, combined with the known camera attitude and the three-dimensional coordinates in the camera coordinate system. Triangulation is a commonly used three-dimensional reconstruction technique that can effectively improve the accuracy of three-dimensional coordinate calculation. Further, step S44b refers to the need to perform structured processing on these coordinates after obtaining the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system from multiple frames of images. Specifically, for each specific fixed reference sphere, the X, Y, and Z coordinates calculated in different frames of images are compiled into a sequence. For example, if N frames of images are obtained, each fixed reference sphere will have a sequence containing N X coordinate values, a sequence containing N Y coordinate values, and a sequence containing N Z coordinate values. As a preferred embodiment, in step S45b, the obtained X coordinate value sequence, Y coordinate sequence, and Z coordinate sequence are subjected to median filtering and the median value is extracted to obtain more stable and accurate X, Y, and Z coordinate values ​​of the fixed reference sphere, and these filtered coordinates are used as the final reference coordinates.

[0099] By acquiring multiple consecutive frames of images and processing the pixel coordinates of the fixed reference sphere identified in each frame, the three-dimensional coordinates of the fixed reference sphere in the workpiece coordinate system are calculated. The use of multiple frames of data ensures that the position information of each fixed reference sphere in three-dimensional space is measured multiple times. Based on this, coordinates are extracted from these multi-frame measurement data to form X, Y, and Z coordinate sequences. Median filtering is then applied to these sequences to suppress random noise, measurement errors, or occasional interference that may exist in single-frame images. The characteristics of median filtering allow it to ignore extreme values ​​in the sequence, thereby extracting more representative and stable coordinate median values ​​and overcoming the coordinate instability problems that may be caused by single-frame or simple averaging of multiple frames of data.

[0100] In some preferred embodiments, a specific example is illustrated below. Assume that in an optical lens inspection scenario, an image sensor device is configured to continuously acquire images at a rate of 30 frames per second. When the reference coordinates of a fixed reference sphere need to be obtained, the image sensor device continuously acquires, for example, 100 frames containing multiple fixed reference spheres. For each of these 100 frames, the pixel coordinates of each fixed reference sphere are first identified using Canny edge detection and least-squares circle fitting algorithms. Subsequently, using the camera's internal calibration information, the pixel coordinates of each fixed reference sphere in each frame are converted into three-dimensional coordinates in the camera coordinate system, and further, its three-dimensional coordinates in the workpiece coordinate system are calculated using triangulation methods. Specifically, for the first fixed reference sphere, 100 X-coordinate values, 100 Y-coordinate values, and 100 Z-coordinate values ​​are obtained in the 100 frames. These values ​​constitute the X-coordinate sequence, Y-coordinate sequence, and Z-coordinate sequence, respectively. For example, the X-coordinate value sequence might be [100.1, 100.2, 99.9, 100.0, 100.5, ..., 100.3]. Next, median filtering is applied to each of these three sequences. For instance, for the X-coordinate value sequence, the median value is taken as the final X-coordinate value for the fixed reference sphere after sorting. Similarly, median filtering is applied to the Y and Z-coordinate sequences. In this way, even if the coordinate calculation of a fixed reference sphere deviates significantly due to transient interference in a few frames of the image, median filtering can effectively eliminate the influence of these outliers, thus obtaining a more stable and accurate reference coordinate. For example, if there is an outlier of 105.0 in the sequence, median filtering will select other more concentrated values ​​as the median value, rather than the average value skewed by this outlier. Finally, the X, Y, and Z coordinates of all fixed reference spheres after median filtering are determined as the reference coordinates for subsequent position offset determination.

[0101] In some of the above embodiments, a scheme is proposed to compare the reference coordinates with the verification coordinates and obtain the comparison results, and to determine whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift based on the comparison results.

[0102] Specifically, step S6 includes:

[0103] Step S61: Calculate the Euclidean distance between the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system and the reference coordinates after completing a set number of optical lens calibration processes, and store it in a structured manner to form a distance dataset.

[0104] Step S62: Analyze the distance dataset to obtain the analysis results of whether the image sensor device or the checkerboard calibration reference board has shifted position. When all values ​​in the distance dataset are greater than the set distance threshold, the result of the image sensor device or the checkerboard calibration reference board shifting position is obtained.

[0105] Step S61 calculates the Euclidean distance between the reference coordinates and verification coordinates of each fixed reference sphere, transforming the abstract "comparison" into a concrete numerical quantification, thus enabling precise measurement of the displacement. Consequently, the displacement of each fixed reference sphere is objectively reflected in the distance dataset. Further, step S62 performs a comprehensive analysis of this distance dataset. When the displacement of all fixed reference spheres exceeds a preset distance threshold, it eliminates potential measurement errors or minor local disturbances in individual reference spheres, thereby accurately determining whether the image sensor device or the checkerboard calibration reference board has experienced an overall positional shift.

[0106] In some preferred embodiments, it is assumed that in an optical lens inspection scenario, the calibration device includes four fixed reference spheres. During the initial calibration phase, the reference coordinates of these four fixed reference spheres in the workpiece coordinate system are P1_base(x1,y1,z1), P2_base(x2,y2,z2), P3_base(x3,y3,z3), and P4_base(x4,y4,z4), respectively. After completing a set number of optical lens calibration steps, the verification coordinates of these four fixed reference spheres in the workpiece coordinate system are obtained again, namely P1_verify(x1',y1',z1'), P2_verify(x2',y2',z2'), P3_verify(x3',y3',z3'), and P4_verify(x4',y4',z4').

[0107] Based on step S61 above, calculate the Euclidean distance between the reference coordinates and the verification coordinates of each fixed reference sphere:

[0108] D1=sqrt((x1'-x1)^2+(y1'-y1)^2+(z1'-z1)^2)

[0109] D2=sqrt((x2'-x2)^2+(y2'-y2)^2+(z2'-z2)^2)

[0110] D3=sqrt((x3'-x3)^2+(y3'-y3)^2+(z3'-z3)^2)

[0111] D4=sqrt((x4'-x4)^2+(y4'-y4)^2+(z4'-z4)^2)

[0112] These distance values ​​D1, D2, D3, and D4 are stored in a structured manner, forming a distance dataset {D1, D2, D3, D4}.

[0113] Further, according to step S62 above, a distance threshold is set, for example, a distance threshold of 0.1 mm. If all values ​​in the calculated distance dataset, i.e., D1, D2, D3, and D4, are greater than 0.1 mm, it is determined that the image sensor device or the checkerboard calibration reference plate has experienced a positional shift. For example, if D1 = 0.15 mm, D2 = 0.18 mm, D3 = 0.12 mm, and D4 = 0.16 mm, since all distances are greater than 0.1 mm, the system will issue a positional shift warning. Conversely, if any one or more of the distance values ​​are less than or equal to 0.1 mm, it is considered that no significant positional shift has occurred, or the shift is within an acceptable range.

[0114] In some of the above-described embodiments, while comparing the reference coordinates with the verification coordinates can determine whether the image sensor device or the checkerboard calibration reference board has shifted position, in actual industrial production and testing scenarios, simply obtaining the shift determination result may not be sufficient to guide subsequent production operations or troubleshooting. When a shift is detected, without further feedback mechanisms, operators may not be able to promptly understand the severity of the problem, the time of occurrence, and the specific deviation, thereby affecting production efficiency and the timeliness of problem resolution.

[0115] Therefore, following step S6, the following steps are also included:

[0116] Step S7: When it is determined that the image sensor device or the checkerboard calibration reference plate has shifted position, feedback information is obtained based on the comparison results. The feedback information includes a suggested lens detection stop instruction, a suggested image sensor device recalibration instruction, or provides the time of the anomaly and the deviation data indicated by the fixed reference ball.

[0117] Specifically, the feedback information aims to convert detected positional offsets into actionable instructions or detailed anomaly data to assist users in decision-making and troubleshooting. The suggested lens inspection stoppage instruction refers to the system automatically or semi-automatically issuing a suggestion or instruction to the operator to stop the current lens inspection process when it detects a severe positional offset exceeding a preset threshold. This is intended to prevent continued inspection under abnormal equipment conditions, thereby avoiding inaccurate inspection results or damage to the lens under test. The suggested image sensor recalibration instruction refers to the system prompting the operator to recalibrate the image sensor device when a positional offset is detected in the image sensor device or checkerboard calibration reference board. This usually requires re-performing the camera's internal calibration steps to ensure the accuracy of subsequent inspections. Furthermore, the feedback information can also provide the time of anomaly occurrence, which is crucial for tracing the source of problems, analyzing anomaly patterns, and managing production batches. Meanwhile, by using the deviation data indicated by the fixed reference spheres, the offset of each fixed reference sphere in the three-dimensional coordinates of the workpiece coordinate system can be specifically quantified, such as the specific deviation values ​​in the X, Y, and Z directions. This provides operators with precise positioning information, helping them to judge the degree and direction of the offset so as to make targeted adjustments and repairs.

[0118] This application's solution immediately acquires and provides feedback information upon detecting a positional shift in the image sensor device or checkerboard calibration reference board. When the system determines a positional shift, the feedback mechanism transforms the abstract shift result into specific, executable instructions or detailed data. For example, by providing a suggested stoppage instruction for lens inspection, the inspection process that might lead to erroneous results can be interrupted in a timely manner, avoiding wasted work and potential quality risks. Simultaneously, a suggested recalibration instruction for the image sensor device directly indicates the direction of problem-solving, guiding operators to perform necessary calibration operations and ensuring the accuracy of the inspection system is restored. Furthermore, providing the time of the anomaly and the deviation data indicated by the fixed reference sphere allows operators to accurately understand the timing, degree, and direction of the shift, which helps in quickly locating the fault, analyzing the cause, and making precise adjustments.

[0119] In some preferred embodiments, assuming that in an optical lens inspection scenario, after a period of continuous inspection, the system determines in step S6 that the image sensor device has experienced a slight positional shift. At this time, according to the scheme of this application, the system will immediately obtain feedback information. Specifically, since the shift amount has not reached a severity requiring immediate shutdown, the system may not issue a shutdown instruction, but will generate an instruction suggesting recalibration of the image sensor device and display it on the operation interface. Simultaneously, the feedback information will also record the time of the shift anomaly, such as "October 27, 2023, 14:35:22," and provide deviation data indicated by a fixed reference sphere. For example, for a certain fixed reference sphere, its X-coordinate shifted by 0.05mm, Y-coordinate shifted by 0.03mm, and Z-coordinate shifted by 0.01mm. After receiving this feedback information, the operator can recalibrate the image sensor device according to the suggestion during the next production break and use the provided deviation data as a reference to finely adjust the calibration process, thereby ensuring the long-term stability and accuracy of the inspection system.

[0120] Secondly, such as Figure 2 As shown, a sensor-based optical lens mounting inspection system 100 is provided. This system is applied in an optical lens inspection scenario, which includes an image sensor device, a lens mounting station, and a calibration device. The calibration device includes a checkerboard calibration reference plate and multiple fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position.

[0121] The camera internal calibration information acquisition module 101 is used to acquire camera internal calibration information in the image sensor device by establishing a detection reference point located on a checkerboard calibration reference plate. The camera internal calibration information includes internal parameters of the intrinsic parameter matrix and distortion coefficients and external parameters including rotation matrix and translation vector.

[0122] The first pixel coordinate acquisition module 102 is used to acquire an image containing multiple fixed reference spheres and identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres.

[0123] The reference coordinate acquisition module 103 is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system by combining the pixel coordinates of each fixed reference sphere with the camera's internal calibration information, and use them as reference coordinates.

[0124] Verification coordinate acquisition module 104 is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing a set number of optical lens calibration processes by combining the pixel coordinates of each fixed reference sphere with the camera's internal calibration information, and use them as verification coordinates.

[0125] The judgment and analysis module 105 is used to compare the reference coordinates with the verification coordinates and obtain the comparison result. Based on the comparison result, it is determined whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

[0126] As described above, this embodiment provides a sensor-based optical lens installation inspection method and system. By setting up an image sensor device, a lens installation station, and a calibration device in the optical lens inspection scenario, and arranging a checkerboard calibration reference board and multiple fixed reference spheres in the calibration device, accurate identification of positional offsets of the image sensor device or the checkerboard calibration reference board is achieved. First, the internal calibration information of the camera is obtained by establishing a detection reference point. Then, the pixel coordinates of the fixed reference spheres are combined with the internal calibration information of the camera to calculate and store their three-dimensional reference coordinates in the workpiece coordinate system. After completing a set number of optical lens calibration processes, the three-dimensional verification coordinates of the fixed reference spheres are calculated again. By comparing the reference coordinates with the verification coordinates, it is determined whether the image sensor device or the checkerboard calibration reference board has experienced positional offset. This solves the problem in the prior art where slight positional offsets of the image sensor device or the checkerboard calibration reference board lead to calibration data failure, inaccurate detection results, and misjudgment of defective / good products. By introducing fixed reference spheres as a reference object, this solution can monitor the spatial attitude changes of key inspection equipment, avoiding the impact of subtle offsets that are difficult to detect in traditional methods on the detection accuracy, and ensuring the accuracy of optical lens installation inspection.

[0127] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit them. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure.

Claims

1. A sensor-based optical lens mounting detection method for identifying whether an image sensor device or a checkerboard calibration reference plate used in optical lens detection applications has experienced positional shift, characterized in that... Includes the following steps: Step S1: Provide an optical lens detection scenario. In the optical lens detection scenario, set up an image sensor device, a lens mounting station and a calibration device. The calibration device includes a checkerboard calibration reference plate and multiple fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position. Step S2: Establish a detection reference point on the checkerboard calibration reference plate, and obtain the camera internal calibration information in the image sensor device based on the detection reference point. The camera internal calibration information includes internal parameters of the intrinsic parameter matrix and distortion coefficients and external parameters including rotation matrix and translation vector. Step S3: Obtain an image containing multiple fixed reference spheres, and identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres; Step S4: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system, and use them as reference coordinates. Specifically, step S4 includes: Step S41a: Using the pixel coordinates of each fixed reference sphere and the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere into three-dimensional coordinates of each fixed reference sphere in the camera coordinate system. Step S42a: Based on the pixel coordinates of each fixed reference sphere and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in the image and the camera's pose. Step S43a: Using the triangulation method, based on the position of each fixed reference ball in the image and the camera posture, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system are calculated in reverse and used as the reference coordinates. Step S5: After completing the set number of optical lens calibration processes, use the pixel coordinates of each fixed reference sphere and the camera's internal calibration information to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing the set number of optical lens calibration processes, and use them as verification coordinates. Step S6: Compare the reference coordinates with the verification coordinates and obtain the comparison results. Based on the comparison results, determine whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

2. The optical lens mounting and detection method according to claim 1, characterized in that, Step S2 specifically includes: Step S21: Use a checkerboard calibration reference board and set the corner points on the checkerboard calibration reference board as detection reference points; Step S22: Use an industrial camera to acquire multiple images of the checkerboard calibration reference board in different poses. Use the checkerboard corner detection function in the OpenCV library to identify the detection reference points in each acquired image of the checkerboard calibration reference board and obtain the corresponding pixel coordinates. Step S23: Using the pixel coordinates corresponding to the detection reference point in each image of the checkerboard calibration reference board, solve for the internal parameters including the intrinsic parameter matrix and distortion coefficients, and the external parameters including the rotation matrix and translation vector using the least squares method.

3. The optical lens mounting and testing method according to claim 1, characterized in that, Step S3 specifically includes: Step S31a: Obtain an image containing multiple fixed reference spheres; Step S32a: The Canny edge detection algorithm is used to identify the edge contours of each fixed reference sphere in the image, which contains multiple fixed reference spheres. Based on the edge contours of each fixed reference sphere, the least squares circle fitting algorithm is used to calculate the pixel coordinates of the center of each fixed reference sphere.

4. The optical lens mounting and testing method according to claim 3, characterized in that, Step S32a further includes: The edge contours of each fixed reference sphere were corrected using the Zernike moment fitting method.

5. The optical lens mounting and testing method according to claim 1, characterized in that, Step S3 specifically includes: Acquire multiple consecutive frames of images containing multiple fixed reference spheres, identify each frame containing multiple fixed reference spheres, and obtain the pixel coordinates of each fixed reference sphere in each frame containing multiple fixed reference spheres.

6. The optical lens mounting and detection method according to claim 5, characterized in that, Step S4 specifically includes: Step S41b: Using the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres, combined with the camera's internal calibration information, calculate and convert the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres into the three-dimensional coordinates of each fixed reference sphere in the camera coordinate system. Step S42b: Based on the pixel coordinates of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres and the external parameters in the camera's internal calibration information, identify the position of each fixed reference sphere in each frame of the image containing multiple fixed reference spheres in its respective corresponding image and the camera pose. Step S43b: Using the triangulation method, based on the position of each fixed reference ball in its corresponding image and the camera pose, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system in each frame of the image containing multiple fixed reference balls, the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system in each frame of the image containing multiple fixed reference balls are calculated in reverse. Step S44b: Extract the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system in each frame of the image containing multiple fixed reference spheres according to the one-to-one correspondence between the multiple fixed reference spheres in each frame of the image, and obtain the X coordinate value sequence, Y coordinate sequence and Z coordinate sequence for each fixed reference sphere. Step S45b: Apply median filtering to the X-coordinate value sequence, Y-coordinate sequence, and Z-coordinate sequence of each fixed reference sphere and extract the median value to obtain the X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere. Use the obtained X-coordinate value, Y-coordinate, and Z-coordinate of each fixed reference sphere as the reference coordinates.

7. The optical lens mounting and testing method according to claim 1, characterized in that, Step S6 specifically includes: Step S61: Calculate the Euclidean distance between the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system and the reference coordinates after completing a set number of optical lens calibration processes, and store it in a structured manner to form a distance dataset. Step S62: Analyze the distance dataset to obtain the analysis results of whether the image sensor device or the checkerboard calibration reference board has shifted position. When all values ​​in the distance dataset are greater than the set distance threshold, the result of the image sensor device or the checkerboard calibration reference board shifting position is obtained.

8. The optical lens mounting and testing method according to claim 1, characterized in that, After step S6, the method further includes: Step S7: When it is determined that the image sensor device or the checkerboard calibration reference plate has shifted position, feedback information is obtained based on the comparison results. The feedback information includes a suggested lens detection stop instruction, a suggested image sensor device recalibration instruction, or provides the time of the anomaly and the deviation data indicated by the fixed reference ball.

9. A sensor-based optical lens mounting and inspection system, applied in an optical lens inspection scenario, comprising an image sensor device, a lens mounting station, and a calibration device, wherein... The calibration device includes a checkerboard calibration reference plate and a plurality of fixed reference spheres arranged on the outer periphery of the checkerboard calibration reference plate and fixed in position, characterized in that it includes: The camera internal calibration information acquisition module is used to acquire camera internal calibration information in the image sensor device by establishing a detection reference point on a checkerboard calibration reference plate. The camera internal calibration information includes internal parameters of the intrinsic parameter matrix and distortion coefficients and external parameters including rotation matrix and translation vector. The first pixel coordinate acquisition module is used to acquire an image containing multiple fixed reference spheres and identify the pixel coordinates of each fixed reference sphere from the image containing multiple fixed reference spheres. The reference coordinate acquisition module is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system by combining the pixel coordinates of each fixed reference sphere with the camera's internal calibration information, and use them as reference coordinates. The reference coordinate acquisition module includes: The three-dimensional coordinate calculation unit uses the pixel coordinates of each fixed reference sphere and the camera's internal calibration information to calculate and convert the pixel coordinates of each fixed reference sphere into three-dimensional coordinates of each fixed reference sphere in the camera coordinate system. The position and camera pose recognition unit identifies the position of each fixed reference sphere in the image and the camera pose based on the pixel coordinates of each fixed reference sphere and the external parameters in the camera's internal calibration information. The reference coordinate calculation unit uses triangulation to calculate the three-dimensional coordinates of each fixed reference ball in the workpiece coordinate system based on the position of each fixed reference ball in the image and the camera posture, combined with the three-dimensional coordinates of each fixed reference ball in the camera coordinate system, and uses them as the reference coordinates. The verification coordinate acquisition module is used to calculate the three-dimensional coordinates of each fixed reference sphere in the workpiece coordinate system after completing a set number of optical lens calibration processes, using the pixel coordinates of each fixed reference sphere and the internal calibration information of the camera, and use them as verification coordinates. The judgment and analysis module is used to compare the reference coordinates with the verification coordinates and obtain the comparison result. Based on the comparison result, it determines whether the image sensor device or the checkerboard calibration reference board has experienced a positional shift.

Images