A ceramic substrate measurement method and system based on multi-camera coordinate system calibration

Abstract

The application relates to a ceramic substrate measurement method and system based on multi-camera coordinate system calibration, and relates to the technical field of ceramic substrate measurement.The application comprises the following steps: acquiring reference mark data, ceramic substrate size measurement data, measurement environment data and historical camera external parameters; according to a preset bidirectional back difference calibration instruction, rotating and lifting a preset cross suction plate arm, and acquiring positive and negative mark point image positions; analyzing the reference mark data, the ceramic substrate size measurement data, the measurement environment data, the historical camera external parameters and the positive and negative mark point image positions to generate real-time camera external parameters; controlling a preset multi-camera group to shoot a preset ceramic substrate to generate a ceramic substrate image; and according to the real-time camera external parameters and preset qualified ceramic substrate size parameters, performing coordinate conversion and size calculation on the ceramic substrate image to generate a ceramic substrate measurement result.The application has the effect of improving the coordinate system calibration precision.

Description

Technical Field

[0001] This application relates to the technical field of ceramic substrate measurement, and in particular to a method and system for ceramic substrate measurement based on multi-camera coordinate system calibration. Background Technology

[0002] Ceramic substrates are insulating substrate materials used for packaging electronic components. Their surfaces are usually processed with precise grid patterns or circuit patterns. The accuracy of geometric dimensions such as grid spacing and line width directly affects the alignment accuracy and electrical performance of subsequent chip packaging.

[0003] In related technologies, existing ceramic substrate size measurement techniques typically use a single high-resolution camera or microscope as the detection device. During the measurement process, the ceramic substrate or camera is moved to take pictures in different areas, and then multiple local images are stitched together to calculate the grid size.

[0004] Regarding the aforementioned technologies, on the one hand, existing ceramic substrate measurement technologies, when dealing with large-format ceramic substrates, are limited by the contradiction between field of view and resolution of a single camera, making it difficult to simultaneously meet the high-precision and high-efficiency inspection requirements across the entire size range. This results in insufficient assurance of full inspection accuracy and production capacity requirements in mass production scenarios. On the other hand, in the calibration process of existing multi-camera measurement systems, camera extrinsic parameter calibration and motion mechanism error calibration are usually performed separately. Camera extrinsic parameters are prone to absorbing geometric errors of the motion axis, leading to a decrease in calibration accuracy. Furthermore, there is a lack of joint decoupling and intelligent evaluation methods for multi-source errors, making it difficult to maintain high accuracy during long-term operation. This, in turn, leads to a decrease in the reliability of measurement data and equipment stability, indicating room for improvement. Summary of the Invention

[0005] To improve the accuracy of coordinate system calibration, this application provides a method and system for measuring ceramic substrates based on multi-camera coordinate system calibration.

[0006] In a first aspect, this application provides a method for measuring ceramic substrates based on multi-camera coordinate system calibration, employing the following technical solution: A method for measuring ceramic substrates based on multi-camera coordinate system calibration, comprising: Acquire baseline marker data, ceramic substrate size measurement data, measurement environment data, and historical camera extrinsic parameters; The preset bidirectional backlash calibration command controls the preset cross suction cup arm to perform rotation and lifting operations, and obtains the image positions of the forward and reverse marker points. The reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images are analyzed to generate real-time camera extrinsic parameters. Control a preset multi-camera group to capture images of a preset ceramic substrate to generate ceramic substrate images; Based on the real-time camera extrinsic parameters and the preset qualified ceramic substrate size parameters, coordinate conversion and size calculation are performed on the ceramic substrate image to generate ceramic substrate measurement results.

[0007] Optionally, the steps of analyzing reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images to generate real-time camera extrinsic parameters include: Based on ceramic substrate size measurement data and measurement environment data, historical camera extrinsic parameters are corrected to generate lightweight correction of camera extrinsic parameters and extrinsic parameter recalibration decision conclusions; Determine whether the decision conclusion of the external parameter recalibration is consistent with the preset external parameter recalibration conclusion; If they are inconsistent, the lightly corrected camera extrinsic parameters will be determined as the real-time camera extrinsic parameters; If they match, find the forward and reverse positions of the same point in the image positions of the forward and reverse marker points; The baseline marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters are analyzed to generate real-time camera extrinsic parameters.

[0008] Optionally, the steps of correcting historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data to generate lightweight corrected camera extrinsic parameters and extrinsic parameter recalibration decision conclusions include: Find the real-time camera temperature and vibration RMS values ​​in the measured environmental data; The RMS values ​​of real-time camera temperature and vibration are analyzed to generate lightweight corrections to camera extrinsic parameters and feedforward correction decision instructions. The ceramic substrate size measurement data and the preset recalibration trigger threshold are analyzed to generate recalibration decision instructions; Based on the preset calibration priority, the feedforward correction decision command and the recalibration decision command are compared to generate the external parameter recalibration decision conclusion.

[0009] Optionally, the steps of analyzing real-time camera temperature and vibration RMS values ​​to generate lightweight corrections to camera extrinsic parameters and feedforward correction decision instructions include: Calculate the difference between the real-time camera temperature and the preset reference camera temperature to generate the real-time camera temperature change. Determine whether the effective value of vibration and the real-time temperature change of the camera meet the preset feedforward triggering requirements; If the conditions are not met, the preset feedforward correction pause instruction will be determined as the feedforward correction decision instruction. If the conditions are met, the feedforward correction trigger instruction will be determined as the feedforward correction decision instruction. Calculate the product of the real-time camera temperature change and the preset extrinsic parameter correction sequence to generate the camera extrinsic parameter correction amount; Control the cross-shaped suction cup arm to adsorb the ceramic substrate and obtain the load characteristic coordinates; The load feature coordinates and the positions of the forward and reverse marker points are calculated based on the preset deformation extrinsic compensation model to generate the elastic offset extrinsic compensation amount. The sum of elastic offset extrinsic compensation, camera extrinsic correction, and historical camera extrinsic is calculated to generate a lightly corrected camera extrinsic.

[0010] Optionally, the step of analyzing ceramic substrate size measurement data and preset recalibration trigger thresholds to generate recalibration decision instructions includes: Find the internal grid measurement dimensions and grid size trend coefficient in the ceramic substrate size measurement data; Calculate the arithmetic mean of the internal mesh measurement dimensions to generate the internal mesh average size; Calculate the absolute value of the difference between the average size of the internal mesh and the preset reference mesh size to generate the mesh fluctuation amplitude; The grid size trend coefficient, grid fluctuation amplitude, and recalibration trigger threshold are compared to generate recalibration decision instructions.

[0011] Optionally, the steps of analyzing the baseline marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters to generate real-time camera extrinsic parameters include: The forward and reverse positions of the same point are analyzed to generate the global initial backlash angle prediction value; The global initial backlash angle prediction, preset camera intrinsic parameters, benchmark data, and lightly corrected camera extrinsic parameters are input into a preset error-binding optimization model for analysis, in order to generate decoupled camera extrinsic parameters and parameter uncertainties. Extract the component standard deviation from the parameter uncertainty; The decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty threshold, and preset second uncertainty threshold are analyzed to generate real-time camera extrinsic parameters.

[0012] Optionally, the steps of analyzing the forward and reverse positions of the same point to generate global initial backlash angle predictions include: Calculate the difference between the forward and reverse positions of the same point to generate the forward and reverse position difference of the same point; The forward and reverse position difference at the same point is substituted into the preset return difference angle conversion formula for calculation to generate the initial return difference angle prediction value at the same point. Calculate the arithmetic mean of the initial lag angle predictions at the same point to generate the global initial lag angle predictions.

[0013] Optionally, the steps of analyzing the decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty thresholds, and preset second uncertainty thresholds to generate real-time camera extrinsic parameters include: Determine whether the standard deviation of the component is less than the first uncertainty threshold; If it is less than, then the decoupled camera extrinsic parameters will be determined as the real-time camera extrinsic parameters; If it is not less than, then determine whether the component standard deviation is less than the second uncertainty threshold; If it is less than the preset number of decoupling pose observations and bidirectional backlash calibration instructions, the cross suction cup arm is rotated and the positions of the forward and reverse marker points are obtained for cyclic judgment. If the value is not less than the preset value, a camera malfunction alarm will be output as a notification.

[0014] Secondly, this application provides a ceramic substrate measurement system based on multi-camera coordinate system calibration, employing the following technical solution: A ceramic substrate measurement system based on multi-camera coordinate system calibration includes: The acquisition module is used to acquire reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the image positions of forward and reverse marker points; A memory for storing a program for a ceramic substrate measurement method based on multi-camera coordinate system calibration as described in any of the preceding claims; The processor and the program in the memory can be loaded and executed by the processor to implement a ceramic substrate measurement method based on multi-camera coordinate system calibration as described in any of the above.

[0015] In summary, this application includes at least one of the following beneficial technical effects: 1. By controlling the cross-shaped suction cup arm to rotate and lift according to the bidirectional backlash calibration command, and analyzing the reference mark data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker points, the real-time camera extrinsic parameters are obtained. The multi-camera group is then controlled to capture images of the ceramic substrate, and the coordinates of the ceramic substrate images are converted and the dimensions are calculated based on the real-time camera extrinsic parameters and the qualified ceramic substrate size parameters to obtain the ceramic substrate measurement results. This improves the extrinsic parameter calibration accuracy and measurement efficiency of the multi-camera measurement system, thereby achieving high-precision and rapid detection of the entire size range of ceramic substrates. 2. By correcting historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data, a lightly corrected camera extrinsic parameter and extrinsic parameter recalibration decision conclusion are obtained. It is then determined whether the extrinsic parameter recalibration decision conclusion is consistent with the extrinsic parameter recalibration conclusion. If they are inconsistent, the lightly corrected camera extrinsic parameter is determined as the real-time camera extrinsic parameter; if they are consistent, the forward and reverse positions of the same point are found in the forward and reverse marker point image positions. The reference marker data, the forward and reverse positions of the same point, and the lightly corrected camera extrinsic parameter are analyzed to obtain the real-time camera extrinsic parameter. This reduces the number of offline recalibrations, reduces equipment downtime, and thus improves the detection efficiency and equipment utilization rate of the production line during continuous operation. 3. By analyzing the forward and reverse positions of the same point, the predicted global initial backlash angle is obtained. The predicted global initial backlash angle, camera intrinsic parameters, reference marker data, and lightly corrected camera extrinsic parameter input error are bundled and analyzed in an optimization model to obtain decoupled camera extrinsic parameters and parameter uncertainties. The component standard deviation is extracted from the parameter uncertainty. The decoupled camera extrinsic parameters, component standard deviations, first uncertainty threshold, and second uncertainty threshold are analyzed to obtain real-time camera extrinsic parameters. This achieves joint decoupling of camera extrinsic parameters and motion axis geometric errors, reduces the coupling interference of mechanical errors on optical measurement accuracy, and improves the long-term accuracy stability and automatic identification capability of the measurement system by quantitatively evaluating the calibration uncertainty. Attached Figure Description

[0016] Figure 1 This is a flowchart of a ceramic substrate measurement method based on multi-camera coordinate system calibration in an embodiment of this application.

[0017] Figure 2 This is a flowchart of the steps in this application embodiment to analyze reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images to generate real-time camera extrinsic parameters.

[0018] Figure 3 This is a flowchart of the steps in this application embodiment to correct historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data, in order to generate a lightweight correction of camera extrinsic parameters and an extrinsic parameter recalibration decision conclusion.

[0019] Figure 4 This is a flowchart of the steps in this application embodiment to analyze the effective values ​​of real-time camera temperature and vibration to generate lightweight correction camera extrinsic parameters and feedforward correction decision instructions.

[0020] Figure 5 This is a flowchart of the steps in this application embodiment to analyze ceramic substrate size measurement data and preset recalibration trigger thresholds to generate recalibration decision instructions.

[0021] Figure 6 This is a flowchart of the steps in this application embodiment to analyze the reference marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters to generate real-time camera extrinsic parameters.

[0022] Figure 7 This is a flowchart of the steps in this application embodiment to analyze the forward and reverse positions of the same point to generate a global initial backlash angle prediction value.

[0023] Figure 8 This is a flowchart of the steps in this application embodiment to analyze decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty thresholds, and preset second uncertainty thresholds to generate real-time camera extrinsic parameters. Detailed Implementation

[0024] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1 to 8 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0025] This application discloses a ceramic substrate measurement method based on multi-camera coordinate system calibration. This method primarily addresses the issue of improving coordinate system calibration accuracy. Specifically, it discloses a ceramic substrate, a calibration target, a multi-camera group, a cross-cup chuck arm, and a processing terminal. The processing terminal is communicatively connected to both the multi-camera group and the cross-cup chuck arm to achieve data interaction and control. By controlling the cross-cup chuck arm to perform forward and reverse rotation and lifting operations, and controlling the multi-camera group to capture images of the calibration target and upload them to the processing terminal, the processing terminal analyzes and processes the captured images to obtain the marker point positions. Then, by analyzing the marker point positions, camera intrinsic parameters, and acquired real-time environmental data, the processing terminal obtains the recalibrated camera extrinsic parameters. This allows the multi-camera group to analyze and process the ceramic substrate, obtaining the detection results of the ceramic substrate. This improves the extrinsic parameter calibration accuracy and measurement efficiency of the multi-camera measurement system, thereby achieving high-precision and rapid detection of the entire size range of the ceramic substrate.

[0026] Reference Figure 1 This application discloses a method for measuring ceramic substrates based on multi-camera coordinate system calibration, including the following steps: Step S100: Acquire reference mark data, ceramic substrate size measurement data, measurement environment data, and historical camera extrinsic parameters.

[0027] The reference marker data refers to the set of geometric feature data of each feature point in the two target markers that are pre-set on the rigid body of the cross suction cup arm. This includes the coordinates of the feature points in the target marker coordinate system and the target installation offset matrix. The reference marker data serves as the reference for subsequent coordinate system calibration, providing known spatial constraints for subsequent calibration. The coordinates of the feature points in the target marker coordinate system refer to the three-dimensional spatial coordinates of each feature point in the target in its own coordinate system. The target installation offset matrix refers to the installation offset matrix from the target marker coordinate system to the arm coordinate system. In one embodiment, before the cross suction cup arm leaves the factory, the operator uses a coordinate measuring machine to calibrate the target markers and compiles the calibration data into a technical manual. When the downstream user uses the device for the first time, the relevant data is stored in the equipment basic database of the processing terminal according to the contents of the technical manual, and the reference marker data can be obtained.

[0028] A target marker refers to a pattern composed of multiple feature points with known three-dimensional spatial distances, set on two opposite ends of a cross-shaped suction cup arm, on a surface parallel to the suction cup surface at that end. The target marker patterns at both ends are different; the target marker at one end is denoted as... The target at the other end is marked as , If the pattern is set to a checkerboard grid, and the pattern is set to a circular array, then... The markers in the middle are denoted as , The markers in the middle are denoted as .

[0029] Ceramic substrate size measurement data refers to the data sequence of internal grid measurement dimensions and grid size trend coefficients on each ceramic substrate obtained after several recent consecutive ceramic substrate size measurements. It is used to reflect the product size measurement trend on the production line in a recent period, thereby determining whether there is a process drift and whether the coordinate system needs to be recalibrated.

[0030] Internal grid measurement dimensions refer to the set of grid size data obtained after measuring the grid size on each ceramic substrate. The processing terminal can obtain the internal grid measurement dimensions by reading data from the historical measurement database, such as reading the most recent 10 measurement data and sorting them according to the measurement time order.

[0031] The grid size trend coefficient refers to data reflecting the direction and rate of change of the grid size of a ceramic substrate over time. It can provide data support for subsequent judgment of whether process drift exists. In one embodiment, a grid is randomly selected from all ceramic substrates, and the maximum and minimum size data of the grid are extracted from the internal grid measurement dimensions. The maximum size data is subtracted from the minimum size data to obtain the size range. Then, the size data corresponding to the grid in the earliest and latest detected ceramic substrates in the internal grid measurement dimensions are extracted. The latest size data is subtracted from the earliest size data to obtain the size change trend. The absolute value of the size change trend is multiplied by the size change trend, and the product is divided by the size range and then by the standard grid size to obtain the grid size trend coefficient. When the size change trend is negative, the grid size trend coefficient is also negative, indicating that the grid size has a tendency to decrease, and vice versa.

[0032] The standard grid size refers to the size of each grid in a standard ceramic substrate. In one embodiment, the standard grid size is 1 mm.

[0033] Measurement environment data refers to the set of parameters representing the physical environment conditions of the two camera groups, including real-time camera temperature and RMS camera vibration. The measurement environment data for each camera is denoted as follows: , , , , , , , , This indicates the real-time temperature of the first camera in the left-hand camera group. This indicates the real-time temperature of the first camera in the right-hand camera group. This refers to the effective value of camera vibration in the left-side camera group. This refers to the effective value of camera vibration in the right-side camera group.

[0034] Real-time camera temperature is obtained by collecting temperature values ​​in real time through temperature sensors installed on each camera and uploading them to the processing terminal. The processing terminal then fills the data into the corresponding number column according to the camera number, thereby obtaining the real-time camera temperature of each camera.

[0035] The effective value of camera vibration refers to the data used to evaluate the overall mechanical environment stability of one side of the camera group. The causes of camera vibration include conveyor belt operation, rotating shaft movement, etc. Vibration sensors deployed on the left and right camera group frames respectively collect the effective values ​​of vibration on each side in real time and upload them to the processing terminal. The processing terminal then fills the corresponding number into the corresponding data column according to the camera number to obtain the effective value of camera vibration for each side of the camera group.

[0036] Historical camera extrinsic parameters refer to the rotation matrix and translation vector of each camera coordinate system relative to the world coordinate system, which were determined during the last coordinate system calibration or when the equipment was manufactured. The historical camera extrinsic parameters serve as the reference values ​​for this calibration. The processing terminal extracts the most recently determined camera extrinsic parameters from the historical measurement database, and this data set is the historical camera extrinsic parameters.

[0037] The historical measurement database refers to a database used to store measurement data from recent measurements of ceramic substrates and the corresponding calibrated camera extrinsic parameters. After each measurement task is completed, the processing terminal collects the corresponding measurement data and the corresponding camera extrinsic parameters and arranges them in chronological order. When the set number of data groups is full, the data group furthest away is removed in chronological order and a new data group is added, thereby realizing the real-time updating of the historical measurement database.

[0038] The equipment basic database refers to a database used to store the rated operating parameters of the equipment used to measure ceramic substrates. Operators search for the corresponding data manuals according to the model of each piece of equipment and upload the data to the processing terminal. The processing terminal then categorizes and stores the data to obtain the equipment basic database.

[0039] Step S101: Control the preset cross suction cup arm to perform rotation and lifting operations according to the preset bidirectional backlash calibration command, and obtain the image positions of the forward and reverse marker points.

[0040] The forward and reverse marker image positions refer to the set of pixel coordinates of each marker on the target captured by each camera when the cross-cup arm reaches the same marker from both the forward and reverse rotation directions and at the same lifting height. The processing terminal controls the cross-cup arm to rotate in fixed rotational angle steps according to the bidirectional backlash calibration command, and simultaneously controls the vertical axis to rise and fall in fixed lifting steps. This causes the marker on the cross-cup arm to pass through n spiral calibration poses covering the entire workspace. Each calibration pose can be represented as... , In each calibration pose, according to the bidirectional backlash calibration command, the rotation axis of the cross-shaped suction cup arm is first controlled clockwise to gradually approach the specified angle. This triggers a multi-camera setup to capture images and record the positions of all visible feature points in the image. Then, the rotation axis is controlled to rotate counterclockwise, gradually approaching the target angle. The camera is triggered again to capture images, thus recording the image positions of all visible markers. The forward position coordinates of the same marker determined by each camera are sorted according to the camera number to obtain the forward position set of the same marker. The reverse position coordinates of the same marker are sorted according to the above method to obtain the reverse position set of the same marker. Then, the forward and reverse position sets of each marker are sorted according to the marker number, and the sorted sequences are summarized to obtain the forward and reverse marker image positions.

[0041] Feature points in an image can be extracted in various ways. In one embodiment, a feature point extraction method based on subpixel refinement can be used to obtain the coordinates of feature points on a target or ceramic substrate. Specifically, for a circular array target, the image can be binarized and connected component analysis can be performed first to extract the outline of the circular region. Then, the subpixel coordinates of the center of the circle can be calculated using an ellipse fitting algorithm to obtain high-precision feature point coordinates. For a checkerboard target or an image of grid intersections on a ceramic substrate, preliminary localization can be performed first using Harris corner detection. Then, subpixel refinement can be performed in the neighborhood using gradient statistics. The orthogonal constraint residual between the gradient direction and the position offset can be iteratively minimized to obtain high-precision feature point coordinates.

[0042] The bidirectional backlash calibration command refers to the control command used to control the cross suction cup arm to perform rotation and lifting operations. It includes rotation angle step size, rotation direction, lifting direction, and lifting step size. In one embodiment, the rotation angle step size is 15°, the rotation direction is clockwise or counterclockwise, the lifting direction is upward or downward, and the lifting step size is 2mm. Each execution of the bidirectional backlash calibration command generates a pose array. This pose array records the rotation direction of the cross suction cup arm, the angle reached after rotation, and the height reached after lifting, denoted as... ,when When the value is positive, the rotation direction is clockwise; otherwise, it is counterclockwise. This indicates the height difference between the bottom of the cross-shaped suction cup arm and the bottom of the cross-shaped suction cup arm's rotating shaft after one lifting and lowering cycle. , This refers to the initial height. This refers to the lifting step size. Taking 0mm as an example, when the lifting direction is upward, then... =+2mm, when the lifting direction is downward, then =-2mm, This refers to the height layer number corresponding to the i-th pose. .

[0043] A cross-shaped suction cup arm is a robotic arm with a cross-shaped suction cup support at one end, used to adsorb and transfer ceramic substrates. Its rotation axis and vertical axis provide two degrees of freedom of motion. The cross-shaped suction cup arm is equipped with two marker targets with different patterns. The marker targets are fixed to the rigid body at the ends of any two opposite suction cup arms and are located on the parallel plane with the suction cup surface. When the cross-shaped suction cup arm rotates, it can be observed by the left and right camera groups in sequence, thus providing data support for subsequent recalibration of the coordinate system.

[0044] Step S102: Analyze the reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images to generate real-time camera extrinsic parameters.

[0045] Among them, real-time camera extrinsic parameters refer to the set of extrinsic parameters of each camera obtained after recalibration based on lightly corrected camera extrinsic parameters. The processing terminal analyzes the reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the image positions of forward and reverse marker points to obtain the real-time camera extrinsic parameters. The specific method is described in [reference needed]. Figure 2 The steps.

[0046] Step S103: Control the preset multi-camera group to take pictures of the preset ceramic substrate to generate ceramic substrate images.

[0047] The ceramic substrate image refers to a set of digital images, including the grid pattern on the surface of the ceramic substrate, obtained by a multi-camera group from either side taking pictures of the same side of the ceramic substrate. After the cross-shaped suction arm picks up the ceramic substrates from both sides and transports them simultaneously to the corresponding detection platform, the cross-shaped suction arm physically separates from the ceramic substrate. The processing terminal then controls the multi-camera group to take pictures of the ceramic substrate separately, and marks the local image of the left ceramic substrate taken by the left camera group as... Mark the image on the right as This process allows you to obtain an image of the ceramic substrate. After acquiring the image, you can control the cross-shaped suction cup arm to simultaneously pick up two ceramic substrates and transport them back to the production line.

[0048] A multi-camera group refers to a vision system consisting of eight industrial cameras. These eight industrial cameras are grouped into two groups of four, namely the left camera group and the right camera group. The two groups of cameras are fixed and symmetrically installed on both sides of the conveyor belt, and can independently perform the task of acquiring images of ceramic substrates.

[0049] Ceramic substrates refer to two independent ceramic products to be tested, which are sequentially fed into the testing area by a conveyor belt. They are mainly used in fields such as microelectronics, power electronics, hybrid microelectronics, and power modules. For example, they are used in AI chip heat dissipation modules as key components for heat dissipation of high-performance computing chips, including a left ceramic substrate and a right ceramic substrate.

[0050] Step S104: Perform coordinate transformation and size calculation on the ceramic substrate image based on the real-time camera extrinsic parameters and the preset qualified ceramic substrate size parameters to generate ceramic substrate measurement results.

[0051] The ceramic substrate measurement result refers to the conclusion output after inspecting two ceramic substrates separately, used to judge whether the grid size in the two ceramic substrates is qualified. In one embodiment, the processing terminal uses the distortion coefficient in the camera intrinsic parameters and the undistort function in standard libraries such as OpenCV to perform distortion correction on the four ceramic substrate images on the left, obtaining a distortion-free image. Then, the Harris corner point algorithm is used to extract features from the distortion-free image, detect the intersection of grid lines on the left ceramic substrate, extract its sub-pixel level two-dimensional image coordinates, and record them as... , This refers to the feature point index. The coordinates of the feature points in each camera image are transformed to a unified world coordinate system using the real-time camera extrinsic parameters on the left. The transformed coordinates are then labeled as... The feature point coordinates and camera intrinsic parameters are extracted from the device's basic database by the processing terminal and then substituted into the formula. The coordinate components in the x and y directions of each feature point in the camera coordinate system are calculated. This refers to the coordinates of the camera principal point in the camera's intrinsic parameters. This refers to the camera's equivalent focal length in the x-direction. This refers to the camera's equivalent focal length in the y-direction. The equivalent focal length is stored in the camera's intrinsic parameters. Since the ceramic substrate is placed on the inspection platform, therefore... The coordinate components in the camera coordinate system are fixed and known; then, the coordinate components in the camera coordinate system are transformed to the world coordinate system using real-time camera extrinsics to obtain the world coordinates of each feature point in the world coordinate system. The specific transformation formula is as follows: By placing all world feature point coordinates into the same dataset, the feature point coordinates are naturally aligned to obtain complete set information of the ceramic substrate on the left. Then, the spatial distance formula is used... The distance between two adjacent grid nodes is calculated to obtain the grid spacing. The qualified ceramic substrate size parameter is subtracted from the obtained grid spacing one by one. The size difference is compared with the qualified difference threshold. Taking ±2.0μm as an example, the qualified difference threshold is counted. If the number of counts exceeds 1, the ceramic substrate is determined to be unqualified. Therefore, the unqualified ceramic substrate result is determined as the ceramic substrate measurement result, and vice versa.

[0052] The qualified ceramic substrate size parameter refers to the standard grid size used to determine whether the grid size in the ceramic substrate is qualified, taking 100.0μm as an example.

[0053] Camera intrinsic parameters refer to the parameter matrix used to describe the internal optical characteristics of each camera, including focal length, principal point coordinates, and radial and tangential distortion coefficients. The processing terminal finds the corresponding focal length, principal point coordinates, and radial and tangential distortion coefficients in the device basic database according to the camera number and sorts them by camera number to obtain the camera intrinsic parameters.

[0054] Reference Figure 2 The steps for generating real-time camera extrinsic parameters by analyzing reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images include: Step S200: Based on the ceramic substrate size measurement data and measurement environment data, the historical camera extrinsic parameters are corrected to generate lightweight corrected camera extrinsic parameters and extrinsic parameter recalibration decision conclusions.

[0055] Among them, the lightweight corrected camera extrinsic parameters refer to the camera extrinsic parameters after temperature drift feedforward compensation and load deformation compensation, which can provide an initial value close to the true value for subsequent accurate determination of camera extrinsic parameters; the extrinsic parameter recalibration decision conclusion refers to the judgment conclusion used to determine whether the camera extrinsic parameters need to be recalibrated. The processing terminal corrects the historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data to obtain the lightweight corrected camera extrinsic parameters and the extrinsic parameter recalibration decision conclusion. The specific method is described in [reference needed]. Figure 3 The steps.

[0056] Step S201: Determine whether the external parameter recalibration decision conclusion is consistent with the preset external parameter recalibration conclusion.

[0057] Among them, the external parameter recalibration conclusion refers to the judgment conclusion output when it is determined that the camera external parameters need to be recalibrated.

[0058] The processing terminal determines whether the external parameter recalibration decision is consistent with the external parameter recalibration conclusion, thereby determining whether the camera external parameters need to be recalibrated.

[0059] Step S2011: If there is a discrepancy, the lightly corrected camera extrinsic parameter is determined as the real-time camera extrinsic parameter.

[0060] If the processing terminal determines that the external parameter recalibration decision is inconsistent with the external parameter recalibration conclusion, it means that the camera external parameters do not need to be recalibrated. Therefore, the processing terminal will lightly correct the camera external parameters and determine them as real-time camera external parameters.

[0061] Step S2012: If they match, find the forward and reverse positions of the same point in the forward and reverse marker image positions.

[0062] If the processing terminal determines that the external parameter recalibration decision is consistent with the external parameter recalibration conclusion, it means that the camera external parameters need to be recalibrated. Therefore, the processing terminal searches in the forward and reverse marker image positions to obtain the same forward position and the same reverse position of each marker.

[0063] The forward position of a single point refers to the coordinates of a specific marker on the target when the cross-shaped suction cup arm rotates clockwise; the reverse position of a single point refers to the coordinates of the same marker on the target when the cross-shaped suction cup arm rotates counterclockwise. The processing terminal sequentially finds the forward and reverse position coordinates of the same marker according to the marker number on the target, summarizes the forward and reverse position coordinates of the same marker, and arranges the forward and reverse position coordinates of each marker according to its number. This yields the forward and reverse positions of each marker. The forward and reverse positions of any given marker are represented as follows: , For the first The same positive position of each marker point For the first The same point is located in the opposite direction to the same point.

[0064] Step S20121: Analyze the baseline marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters to generate real-time camera extrinsic parameters.

[0065] In this step, the real-time camera extrinsic parameters are consistent with those in step S102 above. The processing terminal analyzes the reference marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters to obtain the real-time camera extrinsic parameters. The specific method is described in [reference needed]. Figure 6 The steps.

[0066] Reference Figure 3 The steps for correcting historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data to generate lightweight corrected camera extrinsic parameters and extrinsic parameter recalibration decision conclusions include: Step S300: Locate the real-time camera temperature and vibration RMS values ​​in the measured environmental data.

[0067] In this step, the real-time camera temperature and vibration RMS values ​​are consistent with those in step S100 above. The processing terminal searches for the real-time camera temperature and vibration RMS values ​​in the measurement environment data.

[0068] Step S301: Analyze the real-time camera temperature and vibration RMS values ​​to generate lightweight correction camera extrinsic parameters and feedforward correction decision instructions.

[0069] In this step, the lightweight camera extrinsic parameter correction is consistent with the lightweight camera extrinsic parameter correction in step S200 above. The feedforward correction decision instruction is used to determine whether to execute feedforward extrinsic parameter correction based on temperature and vibration. The processing terminal analyzes the real-time camera temperature and vibration RMS values ​​to obtain the lightweight camera extrinsic parameter correction and the feedforward correction decision instruction. The specific method is described in [reference needed]. Figure 4 The steps.

[0070] Step S302: Analyze the ceramic substrate size measurement data and the preset recalibration trigger threshold to generate a recalibration decision instruction.

[0071] The recalibration decision instruction is used to determine whether to recalibrate the camera's extrinsic parameters. The processing terminal analyzes the ceramic substrate size measurement data and the recalibration trigger threshold to obtain the recalibration decision instruction. The specific method is described in [reference needed]. Figure 5 The steps.

[0072] The recalibration trigger threshold refers to a set of conditional parameters used to determine whether the camera extrinsic calibration process needs to be restarted, including the grid size trend coefficient threshold and the grid fluctuation amplitude threshold.

[0073] The grid size trend coefficient threshold refers to the numerical range of values ​​that reflect the direction and rate of change of the grid size of a qualified ceramic substrate over time. In one embodiment, the grid size trend coefficient threshold is ±0.02μm / substrate. The grid fluctuation amplitude threshold refers to the maximum allowable range of values ​​after repeated measurements of the same grid spacing on the same ceramic substrate. In one embodiment, the grid fluctuation amplitude threshold is ±1.0μm.

[0074] Step S303: Based on the preset calibration priority, compare the feedforward correction decision command and the recalibration decision command to generate the external parameter recalibration decision conclusion.

[0075] In this step, the external parameter recalibration decision conclusion is consistent with the external parameter recalibration decision conclusion in step S200 above. According to the calibration priority, if the recalibration decision instruction is a recalibration instruction, the processing terminal determines the external parameter recalibration conclusion as the external parameter recalibration decision conclusion; if the recalibration decision instruction is not a recalibration instruction and the feedforward correction decision instruction is a feedforward correction pause instruction, the external parameter hold conclusion is determined as the external parameter recalibration decision conclusion; if the recalibration decision instruction is not a recalibration instruction and the feedforward correction decision instruction is a feedforward correction trigger instruction, the external parameter hold conclusion is determined as the external parameter recalibration decision conclusion.

[0076] Calibration priority refers to the priority of multidimensional decision commands when deciding whether camera extrinsics need to be recalibrated. The recalibration decision command has a higher priority than the feedforward correction decision command.

[0077] The feedforward correction pause command indicates that the camera environment parameters are normal and no feedforward correction is needed; the feedforward correction trigger command indicates that the camera environment parameters are abnormal and feedforward correction is needed.

[0078] Reference Figure 4 The steps for analyzing real-time camera temperature and vibration RMS values ​​to generate lightweight corrections to camera extrinsic parameters and feedforward correction decision instructions include: Step S400: Calculate the difference between the real-time camera temperature and the preset reference camera temperature to generate the real-time camera temperature change.

[0079] The real-time camera temperature change refers to the set of temperature deviation values ​​between the current camera temperature and the reference temperature. The processing terminal subtracts the reference camera temperature from the real-time camera temperature of each camera and sorts the resulting data according to the camera number to obtain the real-time camera temperature change.

[0080] The reference camera temperature refers to the temperature reference value used as the zero point reference for thermal drift calculation. In one embodiment, the reference camera temperature is 22°C.

[0081] Step S401: Determine whether the effective value of vibration and the real-time camera temperature change meet the preset feedforward triggering requirements.

[0082] Among them, the feedforward triggering requirement refers to the judgment criteria for the camera's external parameters to be feedforward corrected when the camera's environment is abnormal. This includes the camera temperature change requirement and the vibration effective value requirement. When the vibration effective value or real-time camera temperature change of any side of the camera group does not meet the corresponding feedforward triggering requirement, it indicates that the environment of that side of the camera group is abnormal and the camera's external parameters need to be feedforward corrected. In one embodiment, the camera temperature change requirement is that when the camera is in a normal temperature environment, the real-time camera temperature change is between [2℃, 5]; the vibration effective value requirement is that when the camera is in a normal vibration environment, the vibration effective value is between [0.5mm / s, 2mm / s].

[0083] The processing terminal determines whether the effective value of vibration and the real-time temperature change of the camera meet the feedforward triggering requirements, thereby determining whether the camera's environment is normal and whether feedforward correction of the camera's external parameters is necessary.

[0084] Step S4011: If not satisfied, the preset feedforward correction pause instruction is determined as the feedforward correction decision instruction.

[0085] If the processing terminal determines that the effective value of vibration and the real-time temperature change of the camera do not meet the feedforward triggering requirements, it indicates that the environment in which the camera is located is normal and there is no need to perform feedforward correction on the camera's external parameters. Therefore, the processing terminal determines the feedforward correction pause instruction as the feedforward correction decision instruction.

[0086] The feedforward correction pause instruction in this step is the same as the feedforward correction pause instruction in step S303 above, and will not be repeated here.

[0087] Step S4012: If satisfied, the feedforward correction trigger instruction is determined as the feedforward correction decision instruction.

[0088] If the processing terminal determines that the effective value of vibration and the real-time temperature change of the camera meet the feedforward triggering requirements, it indicates that the environment in which the camera is located is abnormal and that the camera's external parameters need to be corrected by feedforward. Therefore, the processing terminal determines the feedforward correction triggering instruction as the feedforward correction decision instruction.

[0089] The feedforward correction triggering command in this step is the same as the feedforward correction triggering command in step S303 above, and will not be described again here.

[0090] Step S40121: Calculate the product of the real-time camera temperature change and the preset extrinsic parameter correction sequence to generate the camera extrinsic parameter correction amount.

[0091] Among them, the camera extrinsic parameter correction amount refers to the estimated compensation value of the extrinsic parameters obtained by the camera after considering only the temperature effect for feedforward compensation. The processing terminal multiplies the real-time camera temperature change amount with the extrinsic parameter correction parameter sequence to obtain the camera extrinsic parameter correction amount.

[0092] The extrinsic parameter correction sequence refers to the set of sensitivity coefficients of each component of the camera's extrinsic parameters to temperature changes. In one embodiment, the extrinsic parameter correction sequence is represented as follows: , The sensitivity coefficient for the rotational component. =0.0001deg / ℃ =0.0001deg / ℃ =0.0002deg / ℃ The sensitivity coefficient for the translation component. =0.002mm / ℃, =0.001mm / ℃, =0.002mm / ℃.

[0093] Step S40122: Control the cross-shaped suction cup arm to adsorb the ceramic substrate and obtain the load characteristic coordinates.

[0094] The load feature coordinates refer to the position coordinates of each feature point on the target when the cross-shaped suction cup arm adsorbs the ceramic substrate and transports it to the detection platform, and rotates and rises in the same pose as in the bidirectional backlash calibration command. The processing terminal controls the cross-shaped suction cup arm to adsorb the ceramic substrate and rotates and rises and falls according to the bidirectional backlash calibration command, so that the camera group can take pictures of the target. The captured images are then analyzed and processed to obtain the load feature coordinates. The specific image acquisition and image analysis and processing process is the same as the method for determining the position of the forward and reverse target point images in step S101 above, and will not be repeated here.

[0095] Step S40123: Calculate the load feature coordinates and the positions of the forward and reverse marker points based on the preset deformation extrinsic compensation model to generate the elastic offset extrinsic compensation amount.

[0096] The elastic offset extrinsic parameter compensation refers to the amount of extrinsic parameter compensation required to recalibrate the camera extrinsic parameters when the cross-shaped chuck arm adsorbs the ceramic substrate. This compensation is due to the elastic deformation of the arm under load, causing a small rigid displacement of each camera relative to the world coordinate system. The compensation is expressed as follows: , For rotation compensation vector, As the translation compensation vector, the processing terminal calculates the load feature coordinates and the positions of the forward and reverse marker points based on the deformation extrinsic compensation model to obtain the elastic offset extrinsic compensation amount.

[0097] The expression for the deformation extrinsic compensation model is: .

[0098] In the formula, It refers to the first The positional difference between the no-load and loaded states of a feature point For rotation compensation vector, For translation compensation vector, It refers to the first Load feature coordinates of each feature point It refers to the first The processing terminal uses the coordinates of the feature point when it is unloaded to find the forward and reverse positions of the same point in the image of the forward and reverse marker points according to the feature point number, and substitutes them into the first formula to calculate the result. The positional differences of each feature point in the forward and reverse directions, respectively. , This refers to the sensitivity coefficient of the rotational component of the deformation extrinsic parameter. For example, the unit is ° / mm. This refers to the sensitivity coefficient of the translation component of the deformation extrinsic parameter, in order to... For example, dimensionless, This refers to the rotation intercept, in For example, the unit is mm.

[0099] Step S40124: Calculate the sum of elastic offset extrinsic compensation, camera extrinsic correction, and historical camera extrinsic to generate a lightly corrected camera extrinsic.

[0100] In this step, the lightly corrected camera extrinsic parameter is consistent with the lightly corrected camera extrinsic parameter in step S200 above. The processing terminal adds the elastic offset extrinsic parameter compensation amount, the camera extrinsic parameter correction amount and the historical camera extrinsic parameter accordingly to obtain the lightly corrected camera extrinsic parameter.

[0101] Reference Figure 5 The steps for analyzing ceramic substrate size measurement data and preset recalibration trigger thresholds to generate recalibration decision instructions include: Step S500: Locate the internal grid measurement dimensions and grid size trend coefficient in the ceramic substrate size measurement data.

[0102] In this step, the internal grid measurement size and grid size trend coefficient are consistent with those in step S100 above. The processing terminal searches in the ceramic substrate size measurement data to obtain the internal grid measurement size and grid size trend coefficient.

[0103] Step S501: Calculate the arithmetic mean of the internal mesh measurement dimensions to generate the internal mesh average size.

[0104] The average size of the internal grid refers to the average value of all grid spacings in the internal grid measurement dimensions. The processing terminal calculates the sum of all data in the internal grid measurement dimensions and divides the sum by the total amount of data to obtain the average size of the internal grid.

[0105] Step S502: Calculate the absolute value of the difference between the average size of the internal mesh and the preset reference mesh size to generate the mesh fluctuation amplitude.

[0106] Among them, the grid fluctuation amplitude refers to the change of the average grid size of the ceramic substrate relative to the reference grid size. The processing terminal subtracts the reference grid size from the average internal grid size and calculates the absolute value of the difference to obtain the grid fluctuation amplitude.

[0107] The reference grid size refers to the spacing between each grid in a qualified ceramic substrate. In one embodiment, the reference grid size is 200 μm.

[0108] Step S503: Compare the grid size trend coefficient, grid fluctuation amplitude, and recalibration trigger threshold to generate a recalibration decision instruction.

[0109] In this step, the recalibration decision instruction is the same as that in step S302 above. The processing terminal compares the grid size trend coefficient, the grid fluctuation amplitude, and the recalibration trigger threshold. If at least one of the grid size trend coefficient and the grid fluctuation amplitude is not within the range corresponding to the recalibration trigger threshold, the recalibration instruction is determined as a recalibration decision instruction. If both data are within the range corresponding to the recalibration trigger threshold, the recalibration pause instruction is determined as a recalibration decision instruction.

[0110] The recalibration pause command is a judgment conclusion output when the recent test results are normal and there is no need to recalibrate the camera's external parameters.

[0111] Reference Figure 6 The steps for generating real-time camera extrinsics by analyzing baseline marker data, forward positions of the same point, reverse positions of the same point, and lightly corrected camera extrinsics include: Step S600: Analyze the forward and reverse positions of the same point to generate the global initial backlash angle prediction value.

[0112] The global initial backlash angle prediction value refers to the unified backlash angle estimate obtained by arithmetically averaging the backlash angle estimates corresponding to the positional differences of each feature point in the image at each pose. The processing terminal analyzes the forward and reverse positions of the same point to obtain the global initial backlash angle prediction value. The specific method is described in [reference needed]. Figure 7 The steps.

[0113] Step S601: Input the global initial backlash angle prediction value, the preset camera intrinsic parameters, the benchmark data, and the lightly corrected camera extrinsic parameters into the preset error binding optimization model for analysis, so as to generate decoupled camera extrinsic parameters and parameter uncertainties.

[0114] Among them, decoupled camera extrinsic parameters refer to the camera extrinsic parameters obtained by jointly solving the camera extrinsic parameters and motion axis geometric errors through the error bundling optimization model, thus removing the influence of motion axis errors; parameter uncertainty refers to the component extracted from the diagonal elements of the parameter covariance matrix after solving through the error bundling optimization model. The processing terminal analyzes the global initial backlash angle prediction value, camera intrinsic parameters, benchmark data, and lightly corrected camera extrinsic parameter input error bundling optimization model to obtain decoupled camera extrinsic parameters and parameter uncertainty.

[0115] Error bundled optimization model refers to a mathematical model that bundles all parameters to be optimized, such as camera extrinsic parameters, rotation axis backlash error, rotation axis runout error, vertical axis geometric error, and dual-axis relationship error, into a single optimization framework. It solves this problem by minimizing the reprojection errors of all cameras, all poses, and all visible feature points in a single, joint solution. The expression for the error bundled optimization model is: .

[0116] In the formula, This refers to the vector of all parameters to be optimized. , This refers to the decoupled camera extrinsic parameters of each camera. This refers to the actual value of the global initial hysteresis angle. This refers to the rotation axis error. This refers to the vertical axis error. This refers to biaxial error. This refers to the camera serial number. , It is the indicator positioning posture number. , This refers to the feature point number. This refers to the positions of the forward and reverse marker points in the image. This refers to the camera's internal parameters. It refers to the first The rotation matrix in the real-time camera extrinsic parameters of the camera is initialized with a lightly modified rotation matrix in the camera extrinsic parameters. It refers to the first The translation vector in the real-time camera extrinsic parameters of the camera is initially set to a lightly modified translation vector in the camera extrinsic parameters. It refers to the first The theoretical coordinates of each feature point in the world coordinate system This refers to the error obtained after calculation using a robust kernel function. This refers to the residual threshold; taking 1.345 pixels as an example, This refers to the residuals before robustness. .

[0117] First, the theoretical positions of each feature point in the world coordinate system are calculated using a kinematic model. Then through the formula Calculate the residual for the current iteration cycle, then substitute the residual into the robust kernel function to obtain the robust residual. Finally, iteratively solve the robust residuals for all cameras, all poses, and all feature points using the LM algorithm, so that... The calculated value is minimized, thus finding the optimal decoupled camera extrinsic parameters.

[0118] The kinematic model includes a backlash error correction layer, a rotation axis runout error correction layer, a vertical axis error correction layer, a dual-axis relationship error correction layer, and a feature point world coordinate system transformation layer.

[0119] The formula for calculating the return error correction layer is: .

[0120] In the formula, It refers to the first The actual rotation angle of each pose. It refers to the first Predicted initial same-point backlash angle values ​​for each feature point. This refers to the rotation direction indicator; when rotated clockwise, =+1, when rotating counterclockwise =-1, This refers to the global initial backlash angle prediction value.

[0121] The calculation formula for the rotation axis runout error correction layer is as follows: .

[0122] In the formula, This refers to the actual rotation matrix of the rotation axis. This refers to the actual translational offset of the rotation axis. It refers to rotation about the z-axis The standard rotation angle of the angle, , It refers to rotation around the x-axis The standard rotation angle is calculated using the same formula as above. The calculation formula, It refers to rotation about the y-axis The standard rotation angle is calculated using the same formula as above. The calculation formula, This refers to the axial runout amplitude, measured in radians, with an initial value of 0. This refers to the radial runout amplitude, measured in mm, with an initial value of 0. and All Part of it.

[0123] The formula for calculating the vertical axis error correction layer is: .

[0124] In the formula, This refers to the vertical axis at height The actual attitude rotation matrix at that location. This refers to the vertical axis at height The actual translation vector at that point, It refers to the first The nominal vertical axis height for each pose is given by the bidirectional backlash calibration command, in mm. It refers to the straightness deviation coefficient of the vertical axis in the x-direction, and is dimensionless. It refers to the straightness deviation coefficient of the vertical axis in the y-direction. This refers to the pitch angle error coefficient, measured in radians per mm. This refers to the yaw angle error coefficient. , , and All A portion of these parameters are to be optimized, and their initial values ​​are all 0. and .

[0125] The formula for calculating the biaxial relationship error correction layer is: .

[0126] In the formula, This refers to the rotation correction matrix for biaxial relationships. This refers to the biaxial translation correction vector. This refers to the perpendicularity deviation between the rotation axis and the vertical axis in the x-direction, expressed in radians. This refers to the perpendicularity deviation between the rotation axis and the vertical axis in the y-direction. This refers to the offset of the rotation axis from the vertical axis in the x-direction, measured in mm. It refers to the axial offset between the rotation axis and the vertical axis in the y-direction. , , and All The variables to be optimized in the dataset are all initially set to 0. , .

[0127] The formula for calculating the transformation layer of the feature point world coordinate system is: .

[0128] In the formula, It refers to the first The three-dimensional coordinates of each feature point in the cross-shaped suction cup arm coordinate system Through formula Convert the coordinates from the target coordinate system to the arm coordinate system, and then take the first three rows of the matrix to obtain the result. , It refers to the first The mounting offset matrix of the target marker relative to the cross-shaped suction cup arm. , It refers to the first The three-dimensional coordinates of each feature point in the target coordinate system can be obtained by the processing terminal by searching in the reference target data. and .

[0129] After determining the extrinsic parameters of the decoupled camera, according to the formula... Calculate the reprojection residual for each observation and integrate the residuals of all observations into a single vector. Then through the formula Calculate the unit weighted variance of the observations, where, This refers to the total number of observations. This refers to the number of parameters to be optimized. Taking 59 as an example, we then calculate the Jacobian matrix at the convergence point. , Jacobian matrix Substitute into the formula Then the covariance matrix diagonal elements in That is, the first The variance of a parameter is called the parameter uncertainty.

[0130] The camera intrinsic parameters in this step are the same as those in step S103 above, and will not be repeated here.

[0131] Step S602: Extract the component standard deviation from the parameter uncertainty.

[0132] Here, the component standard deviation refers to the variance of each component obtained by taking the square root of the diagonal elements of the parameter covariance matrix after solving the error-bundling optimization model, denoted as: , refers to the covariance matrix The diagonal elements in the matrix are extracted by the processing terminal from the matrix of parameter uncertainty, and the component standard deviation is obtained by taking the square root of each element.

[0133] Step S603: Analyze the decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty threshold, and preset second uncertainty threshold to generate real-time camera extrinsic parameters.

[0134] In this step, the real-time camera extrinsic parameters are consistent with those in step S102 above. The processing terminal analyzes the decoupled camera extrinsic parameters, component standard deviations, the first uncertainty threshold, and the second uncertainty threshold to obtain the real-time camera extrinsic parameters. The specific method is described in [reference needed]. Figure 8 The steps.

[0135] The first uncertainty threshold is a limit value used to determine whether the calibration accuracy of the real-time camera extrinsic parameters meets the normal measurement requirements. In one embodiment, the first uncertainty threshold is 0.005 mm. The second uncertainty threshold is a limit value used to determine whether the camera group has failed due to hardware abnormality. In one embodiment, the second uncertainty threshold is 0.05 mm.

[0136] Reference Figure 7 The steps for analyzing the forward and reverse positions of the same point to generate the global initial backlash angle prediction include: Step S700: Calculate the difference between the forward position and the reverse position of the same point to generate the forward and reverse position difference of the same point.

[0137] Among them, the difference between the forward and reverse positions of the same point refers to the difference in position coordinates of the same marker point when the rotation and lifting posture of the cross suction cup arm are the same except for the rotation direction. The processing terminal subtracts the coordinate component of the forward position of the same point from the coordinate component of the reverse position of the same point to obtain the difference between the forward and reverse positions of the same point.

[0138] Step S701: Substitute the forward and reverse position difference of the same point into the preset return difference angle conversion formula to calculate the initial return difference angle prediction value of the same point.

[0139] The initial same-point backlash angle prediction value is the backlash angle estimate corresponding to the position difference of the same feature point on the image when the cross suction cup arm arrives from the clockwise and counterclockwise directions respectively under each pose. The processing terminal substitutes the forward and reverse position difference of the same point into the backlash angle conversion formula to calculate and obtain the initial same-point backlash angle prediction value.

[0140] The formula for converting the return angle difference is: .

[0141] In the formula, It refers to the first Predicted initial same-point backlash angle values ​​for each feature point. It refers to the first The difference in forward and reverse positions of the same feature point This refers to pixel equivalent. , This refers to the object distance; for example, 300mm. This refers to the camera's equivalent focal length; for example, 5000 pixels. It refers to the first In one embodiment, the processing terminal determines the image coordinates of each feature point in the target when the cross-shaped suction cup arm is stationary at the current moment. It then controls the rotation axis of the cross-shaped suction cup arm to rotate by a known angle, such as 5°, to re-capture the target image and determine the image coordinates of each feature point. The processing terminal calculates the movement distance of the same calibration point before and after the rotation, multiplies the movement distance by the pixel equivalent, and then divides the product by the rotation angle to obtain the rotation radius corresponding to each feature point. The rotation radius corresponding to each feature point.

[0142] Step S702: Calculate the arithmetic mean of the initial lag angle prediction values ​​to generate the global initial lag angle prediction values.

[0143] In this step, the global initial backlash angle prediction value is consistent with the global initial backlash angle prediction value in step S600 above. The processing terminal calculates the arithmetic mean of all initial backlash angle prediction values ​​at the same point to obtain the global initial backlash angle prediction value.

[0144] Reference Figure 8 The steps for generating real-time camera extrinsic parameters include analyzing the decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty thresholds, and preset second uncertainty thresholds. Step S800: Determine whether the component standard deviation is less than the first uncertainty threshold.

[0145] The processing terminal determines whether the component standard deviation is less than the first uncertainty threshold, thereby determining whether the external parameters of the newly calibrated camera are qualified.

[0146] Step S8001: If it is less than, then the decoupled camera extrinsic parameter is determined as the real-time camera extrinsic parameter.

[0147] If the processing terminal determines that the component standard deviation is less than the first uncertainty threshold, it indicates that the newly calibrated camera extrinsic parameters are qualified. Therefore, the processing terminal determines the decoupled camera extrinsic parameters as real-time camera extrinsic parameters.

[0148] Step S8002: If it is not less than, then determine whether the component standard deviation is less than the second uncertainty threshold.

[0149] If the processing terminal determines that the component standard deviation is not less than the first uncertainty threshold, it indicates that the external parameters of the newly calibrated camera are unqualified. Therefore, the processing terminal judges whether the component standard deviation is less than the second uncertainty threshold to determine whether there are hardware abnormalities such as camera loosening, target damage, or rotating shaft jamming.

[0150] Step S80021: If it is less than the preset number of re-decoupling pose observations and bidirectional backlash calibration instructions, control the cross suction cup arm to rotate and obtain the forward and reverse marker point image positions for cyclic judgment.

[0151] If the processing terminal determines that the component standard deviation is less than the second uncertainty threshold, it indicates that there are no hardware abnormalities such as camera loosening, target damage, or rotation axis jamming. Therefore, the processing terminal controls the cross suction cup arm to rotate again according to the number of re-decoupling pose observations and the bidirectional backlash calibration command, and returns to step S102 for cyclic judgment after reacquiring the forward and reverse target point image positions, thereby improving the calibration accuracy.

[0152] The number of re-decoupling pose observations refers to the number of positions that need to be re-controlled to perform a rotation and lifting operation when the component standard deviation is between the first and second uncertainty thresholds. In one embodiment, the number of re-decoupling pose observations is twice the number of the original pose. If the pose range is the same as the original pose range, the rotation angle step or lifting step is halved.

[0153] Step S80022: If it is not less than, output a preset camera fault alarm to provide a prompt.

[0154] If the processing terminal determines that the standard deviation of the component is not less than the second uncertainty threshold, it indicates that there may be hardware abnormalities such as camera loosening, target damage, or rotating shaft jamming. Therefore, the processing terminal outputs a camera fault alarm to prompt the maintenance personnel to inspect the camera, target, and cross suction cup arm in a timely manner.

[0155] A camera fault alarm is a judgment conclusion output when the uncertainty of the newly calibrated camera extrinsic parameters is too high, which may indicate hardware abnormalities such as camera loosening, target damage, or rotating shaft jamming.

[0156] Based on the same inventive concept, embodiments of this application provide a ceramic substrate measurement system based on multi-camera coordinate system calibration, comprising: The acquisition module is used to acquire reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, forward and reverse marker image positions, and load feature coordinates. The memory is used to store a program for a ceramic substrate measurement method based on multi-camera coordinate system calibration; The processor can load and execute programs in memory to implement a method for measuring ceramic substrates based on multi-camera coordinate system calibration.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0158] This application provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor, which is a method for measuring ceramic substrates based on multi-camera coordinate system calibration.

[0159] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0160] Based on the same inventive concept, embodiments of this application provide a smart terminal, including a memory and a processor. The memory stores a computer program that can be loaded and executed by the processor, which is a method for measuring ceramic substrates based on multi-camera coordinate system calibration.

[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0162] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for measuring ceramic substrates based on multi-camera coordinate system calibration, characterized in that, include: Acquire baseline marker data, ceramic substrate size measurement data, measurement environment data, and historical camera extrinsic parameters; The preset bidirectional backlash calibration command controls the preset cross suction cup arm to perform rotation and lifting operations, and acquires the image positions of the forward and reverse marker points. The reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images are analyzed to generate real-time camera extrinsic parameters. Control a preset multi-camera group to capture images of a preset ceramic substrate to generate ceramic substrate images; Based on the real-time camera extrinsic parameters and the preset qualified ceramic substrate size parameters, coordinate transformation and size calculation are performed on the ceramic substrate image to generate ceramic substrate measurement results; The steps for generating real-time camera extrinsic parameters by analyzing reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the positions of forward and reverse marker images include: Based on ceramic substrate size measurement data and measurement environment data, historical camera extrinsic parameters are corrected to generate lightweight correction of camera extrinsic parameters and extrinsic parameter recalibration decision conclusions; Determine whether the decision conclusion of the external parameter recalibration is consistent with the preset external parameter recalibration conclusion; If they are inconsistent, the lightly corrected camera extrinsic parameters will be determined as the real-time camera extrinsic parameters; If they match, then find the forward and reverse positions of the same point in the image positions of the forward and reverse marker points; The baseline marker data, the forward position of the same point, the reverse position of the same point, and the lightly corrected camera extrinsic parameters are analyzed to generate real-time camera extrinsic parameters.

2. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 1, characterized in that, The steps for correcting historical camera extrinsic parameters based on ceramic substrate size measurement data and measurement environment data to generate lightweight corrected camera extrinsic parameters and extrinsic parameter recalibration decision conclusions include: Find the real-time camera temperature and vibration RMS values ​​in the measured environmental data; The RMS values ​​of real-time camera temperature and vibration are analyzed to generate lightweight corrections to camera extrinsic parameters and feedforward correction decision instructions. The ceramic substrate size measurement data and the preset recalibration trigger threshold are analyzed to generate recalibration decision instructions; Based on the preset calibration priority, the feedforward correction decision command and the recalibration decision command are compared to generate the external parameter recalibration decision conclusion.

3. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 2, characterized in that, The steps for analyzing real-time camera temperature and vibration RMS values ​​to generate lightweight corrections to camera extrinsic parameters and feedforward correction decision instructions include: Calculate the difference between the real-time camera temperature and the preset reference camera temperature to generate the real-time camera temperature change. Determine whether the effective value of vibration and the real-time temperature change of the camera meet the preset feedforward triggering requirements; If the conditions are not met, the preset feedforward correction pause instruction will be determined as the feedforward correction decision instruction. If the conditions are met, the feedforward correction trigger instruction will be determined as the feedforward correction decision instruction. Calculate the product of the real-time camera temperature change and the preset extrinsic parameter correction sequence to generate the camera extrinsic parameter correction amount; Control the cross-shaped suction cup arm to adsorb the ceramic substrate and obtain the load characteristic coordinates; The load feature coordinates and the positions of the forward and reverse marker points are calculated based on the preset deformation extrinsic compensation model to generate the elastic offset extrinsic compensation amount. The sum of elastic offset extrinsic compensation, camera extrinsic correction, and historical camera extrinsic is calculated to generate a lightly corrected camera extrinsic.

4. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 2, characterized in that, The steps for analyzing ceramic substrate size measurement data and preset recalibration trigger thresholds to generate recalibration decision instructions include: Find the internal grid measurement dimensions and grid size trend coefficient in the ceramic substrate size measurement data; Calculate the arithmetic mean of the internal mesh measurement dimensions to generate the internal mesh average size; Calculate the absolute value of the difference between the average size of the internal mesh and the preset reference mesh size to generate the mesh fluctuation amplitude; The grid size trend coefficient, grid fluctuation amplitude, and recalibration trigger threshold are compared to generate recalibration decision instructions.

5. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 1, characterized in that, The steps for generating real-time camera extrinsics by analyzing baseline marker data, forward positions of the same point, reverse positions of the same point, and lightly corrected camera extrinsics include: The forward and reverse positions of the same point are analyzed to generate the global initial backlash angle prediction value; The global initial backlash angle prediction, preset camera intrinsic parameters, benchmark data, and lightly corrected camera extrinsic parameters are input into a preset error-binding optimization model for analysis, in order to generate decoupled camera extrinsic parameters and parameter uncertainties. Extract the component standard deviation from the parameter uncertainty; The decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty threshold, and preset second uncertainty threshold are analyzed to generate real-time camera extrinsic parameters.

6. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 5, characterized in that, The steps for analyzing the forward and reverse positions of the same point to generate the global initial backlash angle prediction include: Calculate the difference between the forward and reverse positions of the same point to generate the forward and reverse position difference of the same point; The forward and reverse position difference at the same point is substituted into the preset return difference angle conversion formula for calculation to generate the initial return difference angle prediction value at the same point. Calculate the arithmetic mean of the initial lag angle predictions at the same point to generate the global initial lag angle predictions.

7. The method for measuring ceramic substrates based on multi-camera coordinate system calibration according to claim 5, characterized in that, The steps for generating real-time camera extrinsic parameters by analyzing the decoupled camera extrinsic parameters, component standard deviations, preset first uncertainty thresholds, and preset second uncertainty thresholds include: Determine whether the standard deviation of the component is less than the first uncertainty threshold; If it is less than, then the decoupled camera extrinsic parameters will be determined as the real-time camera extrinsic parameters; If it is not less than, then determine whether the component standard deviation is less than the second uncertainty threshold; If it is less than the preset number of decoupling pose observations and bidirectional backlash calibration instructions, the cross suction cup arm is rotated and the positions of the forward and reverse marker points are obtained for cyclic judgment. If the value is not less than the preset value, a camera malfunction alarm will be output as a notification.

8. A ceramic substrate measurement system based on multi-camera coordinate system calibration, characterized in that, include: The acquisition module is used to acquire reference marker data, ceramic substrate size measurement data, measurement environment data, historical camera extrinsic parameters, and the image positions of forward and reverse marker points; A memory for storing a program for measuring ceramic substrates based on multi-camera coordinate system calibration as described in any one of claims 1 to 7; The processor and the program in the memory can be loaded and executed by the processor to implement the ceramic substrate measurement method based on multi-camera coordinate system calibration as described in any one of claims 1 to 7.

Images