Patch type infrared receiving head appearance detection method and system based on machine vision

Abstract

The application discloses a kind of based on machine vision's patch type infrared receiving head appearance detection method and system, belong to computer vision detection field.Method includes: through bottom collimating backlight source irradiation to be measured receiving head, utilize the image of two telecentric cameras of symmetrical arrangement Synchronous acquisition pin;Sub-pixel level feature positioning is carried out to image and rejects reflection false feature point;Based on binocular vision principle, reconstruct the three-dimensional space coordinates of pin end;Adopt lowest multipoint method to construct reference plane, calculate the vertical distance of each pin to reference plane as coplanarity deviation;Deviation and tolerance threshold are compared, and output determination signal.System includes object table, bottom collimating backlight source, double-side telecentric camera, hardware level synchronous trigger unit, image processing and three-dimensional reconstruction unit, coplanarity evaluation and determination unit.The application can high-precision measurement pin coplanarity, eliminate perspective distortion and stray light interference, suitable for on-line full inspection on automatic production line.

Description

Technical Field

[0001] This application belongs to the field of computer vision inspection, specifically relating to a method and system for appearance inspection of a patch-type infrared receiver head based on machine vision. Background Technology

[0002] As a core electronic component of infrared remote control and wireless sensing systems, the geometry and spatial arrangement accuracy of the surface mount technology (SMT) pins of surface mount infrared receivers directly affect the soldering quality of the surface mount process. To ensure flat contact and reliable electrical connections of the device on the circuit board, the coplanarity and other appearance parameters of the pins must be rigorously tested before leaving the factory.

[0003] Currently, the industry commonly uses machine vision-based automated optical inspection systems to inspect the appearance of pins. However, most existing vision inspection solutions employ a single-camera vertical shooting mode, which can only acquire the coordinate information of the pin in a two-dimensional projection plane and cannot perceive the height change of the pin in the direction perpendicular to the stage. Since pin coplanarity is essentially a three-dimensional spatial geometric parameter, monocular vision systems, lacking depth information, struggle to quantify the degree of pin protrusion or depression.

[0004] Furthermore, under conventional imaging lenses, when the distance between the pin and the lens changes due to bending or deformation, the lens's inherent perspective distortion effect causes a non-linear shift in the imaging magnification as the object distance changes. This results in distortion of image edge feature point extraction, further reducing the accuracy of coplanarity detection. The lack of modeling capabilities for three-dimensional spatial features makes the system prone to misjudgment or missed detection when faced with slight pin deformation, making it difficult to meet the quality control requirements of high-reliability electronic products.

[0005] Therefore, how to achieve high-precision, non-contact three-dimensional measurement of the coplanarity of the pins of a patch-type infrared receiver head, and eliminate the interference of perspective distortion on the detection results, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for appearance inspection of patch-type infrared receiver heads based on machine vision, which can effectively solve the problems in the background art mentioned above.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A machine vision-based method for appearance inspection of a patch-type infrared receiver head includes the following specific steps: Step S1, Image Acquisition: Fix the patch infrared receiver under test onto the stage, and illuminate the patch infrared receiver under test with a bottom collimated backlight; use the left telecentric camera and the right telecentric camera, which are symmetrically arranged on both sides of the patch infrared receiver under test, to synchronously acquire pin images of the patch infrared receiver under test at the same physical moment; both the left telecentric camera and the right telecentric camera are equipped with object-side telecentric lenses, and their optical axes are arranged at a 45-degree angle to the plane of the stage; Step S2, Feature Extraction: Perform sub-pixel level feature precise localization on the two acquired pin images, extract key feature points representing the physical end of the pin, and remove reflective pseudo feature points; Step S3, 3D reconstruction: Based on the pre-calibrated binocular vision system parameters, perform corresponding point matching on the key feature points extracted from the left and right images, and apply the principle of triangulation to calculate the 3D spatial coordinates of all pin ends in the world coordinate system. Step S4, Coplanarity Evaluation: Extract the three-dimensional spatial coordinates of all pin ends, and construct a reference plane using the minimum multi-point method. That is, traverse all pin end coordinates and search for at least three points with the smallest Z coordinate values ​​in the negative Z-axis direction. Based on the at least three lowest points, fit a reference plane representing the physically stable attitude. Calculate the vertical geometric distance from each pin end coordinate point to the reference plane in turn. This distance value is used as a quantitative index of coplanarity deviation. Step S5, result determination: Compare the absolute value of the coplanarity deviation of each pin with the preset tolerance threshold. If the absolute value of any pin exceeds the tolerance threshold, the patch infrared receiver head under test is determined to be unqualified; otherwise, it is determined to be qualified, and the corresponding appearance inspection determination signal is output.

[0008] Furthermore, the synchronous acquisition at the same physical moment described in step S1 is achieved by a hardware-level synchronous triggering control mechanism, which is executed by a high-speed pulse generator consisting of an FPGA-based central processing unit and a phase-locked loop module integrated inside the FPGA. At the moment the detection cycle starts, the central processing unit generates two rising edge trigger pulses simultaneously through the phase-locked loop module, and outputs them to the hardware trigger input ports of the left telecentric camera and the right telecentric camera respectively. The synchronization time deviation between the two trigger pulses is controlled within 10 nanoseconds.

[0009] Furthermore, the sub-pixel-level feature precise localization described in step S2 specifically includes: Gradient calculations are performed on the pin image to determine coarse feature points; For each coarse feature point, a search window is established along its gradient direction. Within the search window, gray values ​​are extracted at each sampling position along the gradient normal direction to form a discrete one-dimensional gray distribution sequence. The Levenberg-Marquardt iterative optimization algorithm is applied to fit a Gaussian function to the one-dimensional grayscale distribution sequence. The center position of the obtained Gaussian function is used as the sub-pixel level precise position of the pin edge. The Gaussian function model expression is as follows: Where X is the sampling position coordinate along the gradient normal direction, f(x) is the gray value at the corresponding sampling position, and A is the peak amplitude. Let the sub-pixel edge position be the solution. The standard deviation is 0.1; the positioning accuracy reaches 0.1 pixels.

[0010] Furthermore, the removal of reflective pseudo-feature points in step S2 specifically includes: applying the Sobel operator again to the first-order gradient field of the image to calculate the second-order gradient magnitude, and setting the second-order gradient threshold to the mean of the second-order gradient field of the current image plus 3 times the standard deviation. If the number of pixels in a 5x5 neighborhood of a certain feature point whose second-order gradient magnitude exceeds the threshold exceeds 50% of the total number of pixels in the neighborhood, then the point is determined to be a reflective pseudo-feature point and is removed from the feature point set.

[0011] Furthermore, the matching of corresponding points in step S3 is based on the principle of epipolar constraints and feature descriptor similarity evaluation, and the pin arrangement order constraint and the spacing constraint between adjacent pins are introduced for consistency verification to eliminate mismatched point pairs.

[0012] Furthermore, the coplanarity evaluation described in step S4, after calculating the vertical geometric distance from the coordinate point at the end of each pin to the reference plane, also includes a statistical filtering step based on the 3-Sigma criterion: calculating the mean and standard deviation of all vertical geometric distance values, marking points that exceed the mean ± 3 times the standard deviation as statistical outliers and removing them, and re-performing the step of constructing the reference plane and calculating the distance using the minimum multi-point method on the remaining set of points after removing outliers, to ensure that the coplanarity evaluation is not interfered with by individual severely deformed pins.

[0013] Furthermore, the bottom collimated backlight source mentioned in step S1 uses a collimated beam with a divergence angle of less than 2 degrees, and narrowband interference filters are installed at the front end of the lenses of the left telecentric camera and the right telecentric camera. The center wavelength of the narrowband interference filter is strictly matched with the peak emission wavelength of the bottom collimated backlight source, and is selected as 630 nanometers. The full width at half maximum (FWHM) bandwidth is limited to within 20 nanometers to physically shield stray light interference.

[0014] Furthermore, the fixing of the patch-type infrared receiver head to be tested onto the stage in step S1 is achieved through a vacuum adsorption system integrated inside the stage. The vacuum adsorption system includes an air pump, an air pipe, and a micro-pore array on the surface of the stage. When the air pump is activated, negative pressure is generated, and the main body of the patch-type infrared receiver head to be tested is firmly locked at the preset coordinate origin through the micro-pores to prevent micron-level displacement during the detection period.

[0015] Furthermore, the method also includes an environmental temperature compensation task, which involves real-time acquisition of the temperature value of the detection environment by using PT100 high-precision temperature sensors installed on the metal brackets of the left and right telecentric cameras, and forming a temperature-correction coefficient lookup table based on the mapping relationship between the temperature change and the projection matrix correction coefficient established by the pre-calibration experiment. The system retrieves or interpolates the corresponding projection matrix correction coefficient from the lookup table based on the real-time collected temperature value, and uses this correction coefficient to fine-tune the values ​​of each element in the projection matrix in real time to compensate for camera pose shift caused by thermal deformation. A machine vision-based patch-type infrared receiver appearance inspection system includes: The stage is used to fix the patch-type infrared receiver head to be tested. A bottom collimated backlight is used to generate a collimated beam that illuminates the patch-type infrared receiver head under test. The left and right telecentric cameras are symmetrically arranged on both sides of the patch infrared receiver head under test. Both are equipped with object-side telecentric lenses, and their optical axes are arranged at a 45-degree angle to the plane of the stage, so as to synchronously acquire pin images at the same physical moment. The hardware-level synchronous trigger control unit includes a high-speed pulse generator, which consists of a central processing unit FPGA and a phase-locked loop module integrated inside the FPGA. The high-speed pulse generator is used to generate two trigger pulses with a synchronization time deviation controlled within 10 nanoseconds to drive the left telecentric camera and the right telecentric camera to be exposed synchronously. The image processing and 3D reconstruction unit is used to perform steps S2 and S3, wherein step S2 includes subpixel-level precise feature localization and removal of reflective pseudo-feature points, and step S3 includes calculating the 3D spatial coordinates of the pin end based on epipolar constraints and triangulation principles. The coplanarity evaluation and judgment unit is used to extract the three-dimensional spatial coordinates of all pin ends, construct a reference plane using the minimum multi-point method, calculate the vertical geometric distance of each pin to the reference plane, and perform statistical filtering according to the 3-Sigma criterion. Then, it compares the deviation value with the preset tolerance threshold and outputs the appearance inspection judgment signal.

[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. By introducing a multi-view vision system and a telecentric imaging optical path, high-precision measurement of the coplanarity of the pins of a patch-type infrared receiver head is achieved. The constant magnification characteristic of the telecentric lens eliminates perspective distortion at the physical level, ensuring that even when the pins are warped or bent, the projection position of their edge features on the photosensitive chip remains linear, preventing measurement deviations caused by changes in object distance. Combined with sub-pixel-level feature extraction algorithms and triangulation reconstruction technology, this invention can improve the measurement accuracy in the height direction to the target level, solving the problem that existing technologies cannot accurately capture extremely small pin vertical deformations, and ensuring the reliability of soldering in subsequent surface mount processes.

[0017] 2. By employing a symmetrical optical path design and a bottom-collimated backlight, this invention significantly improves the robustness of the vision system to industrial environments. The collimated optical path, combined with narrowband filtering technology, effectively shields stray light interference from the production environment, improving image contrast and consistency. Simultaneously, by introducing a synchronous triggering mechanism and an environmental temperature compensation mechanism, the influence of motion vibration and equipment thermal deformation on measurement results is eliminated, ensuring that the detection system maintains stable detection accuracy even under long-term, high-load operation.

[0018] 3. This invention deeply optimizes image processing operators and 3D geometric algorithms, utilizing parallel processing technology to achieve extremely short single-item inspection times. This enables high-precision 3D appearance inspection to be directly integrated into automated packaging production lines, achieving full inspection and improving production efficiency. Furthermore, the system's data recording and analysis functions support continuous optimization of the production process. By monitoring pin deformation trends, it can provide early warnings of anomalies in upstream processes, achieving an intelligent leap from end-point inspection to process control. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall scheme for a machine vision-based patch infrared receiver head appearance inspection method. Figure 2 This is a schematic diagram illustrating the core principles of three-dimensional spatial coordinate reconstruction and coplanarity evaluation. Figure 3 This is a flowchart of the logic for the synchronous imaging and image preprocessing stages; Figure 4 This is a schematic diagram of the data flow for sub-pixel level feature point localization and binocular corresponding point matching. Figure 5 This is a logic flowchart for pin coplanarity detection and sorting decision based on a fitted reference plane. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The present invention will be further described in detail below with reference to specific embodiments.

[0021] Firstly, the machine vision-based patch infrared receiver head appearance inspection method disclosed in this application is strictly divided into multiple interrelated and logically rigorous stages. The logical starting point of the inspection method lies in the digital acquisition of physical information, and then the accurate reconstruction and quantitative evaluation of the three-dimensional geometric shape are achieved through a complex mathematical model, specifically including the following steps.

[0022] The first step, S1, involves converting the physical geometric features of the patch-type infrared receiver head under test into a high-quality digital image, providing a reliable data foundation for subsequent feature extraction and 3D reconstruction. This is achieved collaboratively through the following sub-steps.

[0023] In step S101, the object to be tested is precisely fixed by placing the receiver head on the stage (whose flatness is controlled within 1 micrometer). The stage integrates a vacuum adsorption system. The vacuum adsorption system consists of an air pump, an air pipe, and an array of micropores on the surface of the stage. The pore diameter ranges from 0.5 mm to 1.0 mm, and the spacing between adjacent micropores is 2 mm to 3 mm.

[0024] After the air pump generates negative pressure, the receiver head is firmly locked at the preset coordinate origin through micro-holes on the stage surface. This ensures that the object under test will not undergo micrometer-level displacement throughout the entire detection cycle, even under mechanical disturbances caused by environmental micro-vibrations or high-speed camera triggering. The stage surface is precision-ground, with its flatness controlled within 1 micrometer. The coordinate origin is defined as the geometric center of the stage surface, serving as the origin of the world coordinate system for subsequent 3D reconstruction, providing an absolute physical reference for coplanarity evaluation.

[0025] Step S102, collimated backlight illumination: A bottom collimated backlight is configured below the high-precision stage. This bottom collimated backlight is an LED array with a collimated optical path design. The LED array consists of multiple monochromatic LED chips with a center wavelength of 630 nanometers, arranged in a rectangular grid. The light emitted by the LEDs is processed by a collimating lens group to form a collimated beam with a divergence angle of less than 2 degrees, illuminating the object under test in a vertically upward direction.

[0026] The collimating lens group consists of an aspherical condenser lens and a plano-convex lens. The axial distance between the two lenses is determined according to the size of the light-emitting surface of the LED array and the required beam diameter. When the collimated beam passes through the edge of the receiver head pin, due to the diffraction effect of light on the metal edge, extremely narrow and high-contrast diffraction fringes are generated, thus forming a clear and sharp contour boundary on the photosensitive chip, providing high signal-to-noise ratio basic data for subsequent sub-pixel-level feature extraction.

[0027] Step S103, multi-view telecentric imaging: Two sets of telecentric cameras are symmetrically arranged on both sides of the receiver head under test, namely the left telecentric camera and the right telecentric camera. Each set of cameras is equipped with an object-side telecentric lens, the object-side telecentricity of which is no greater than 0.1 degrees, and the supported object-side depth of field range is ±3 mm.

[0028] The core physical characteristic of a telecentric lens lies in the fact that its object-side principal ray is parallel to the optical axis. This ensures that, within the depth of field, even if the object distance between the lead and the lens changes due to bending or tilting, the magnification of the lead features in the image remains constant, thus suppressing the influence of perspective errors on the measurement. The optical axes of both sets of telecentric cameras are arranged at a 45-degree angle to the stage plane.

[0029] The basis for using a 45-degree angle is that at this angle, the height change ΔZ of the pin along the normal direction of the stage plane and the projection displacement Δd it causes in the image plane satisfy the relationship that Δd is equal to ΔZ multiplied by sin45 degrees. Since sin45 degrees is approximately equal to 0.707, this mapping relationship facilitates the subsequent calculation of the height value using the principle of triangulation, while ensuring that the pin tip and its root are clearly imaged within the depth of field.

[0030] Step S104: Hardware-level synchronous trigger acquisition. The system introduces a hardware-level synchronous trigger control mechanism, which consists of a central processing unit and a high-speed pulse generator electrically connected to it. The central processing unit is an FPGA-based central processing unit, and the high-speed pulse generator is a phase-locked loop module integrated inside the FPGA.

[0031] At the start of the detection cycle, the central processing unit simultaneously generates two rising-edge trigger pulses via the phase-locked loop module, which are output to the hardware trigger input ports of the left and right telecentric cameras, respectively. The synchronization time deviation between the two trigger pulses is controlled within 10 nanoseconds, ensuring that both cameras begin exposure and capture the lateral grayscale image of the object under test at the same physical moment, thus solidifying the spatial correspondence between the two images in the temporal dimension. The exposure time for each camera is set to 100 microseconds to 500 microseconds, with the specific value determined based on the luminous intensity of the bottom backlight.

[0032] Step S105: Narrowband optical filtering. A narrowband filter is installed at the front of the lens of each telecentric camera. The narrowband filter is an interference bandpass filter with a center wavelength that is strictly matched with the peak emission wavelength of the bottom backlight, selected as 630 nm. The full width at half maximum (FWHM) bandwidth is limited to within 20 nm, and the peak transmittance is greater than 90%.

[0033] The filter is mounted to the front of the lens via a threaded interface, with its optical surface perpendicular to the lens's optical axis. This hardware filtering effectively attenuates common factory environment interference such as fluorescent lighting, welding arc stray light, and radiation in other wavelengths, resulting in high purity of the grayscale image data received by the image processing system. The degree of attenuation is characterized by the filter's optical density; in wavelengths outside the center wavelength ±50 nanometers, the optical density is no less than OD4.

[0034] In summary, through step S1, the spatial geometric features of the pins of the patch infrared receiver head are recorded in two grayscale images under the effects of stable clamping, collimated backlight illumination, multi-view telecentric synchronous imaging, and optical filtering. This set of synchronously acquired dual-view images carries the position information of the pin ends in three-dimensional space, providing a data source for extracting sub-pixel-level feature points of the pin edges from the image data in step S2.

[0035] Step S2, after acquiring high-quality dual-view grayscale image data in step S1, aims to accurately extract key feature points representing the physical ends of the pins from millions of original pixels, providing precise two-dimensional coordinate input for subsequent 3D reconstruction. This is achieved through the following sub-steps working together.

[0036] Step S201: Adaptive median filtering preprocessing. The original grayscale image output in step S1 is processed using an adaptive median filtering window with a preset size of 5 by 5. When the filter traverses the image pixels, it performs noise judgment on the 5 by 5 neighborhood of each pixel: calculates the grayscale median, maximum, and minimum values ​​of all pixels in the neighborhood. If the grayscale value of the current pixel is equal to the minimum or maximum value, the pixel value is replaced with the median; if the current pixel is not an extreme point, the original grayscale value is retained unchanged.

[0037] When the noise density in the neighborhood is too high, causing the median to also be an extreme value, the filter window size is expanded to 7 times 7 before re-evaluation, with an upper limit of 9 times 9 for window expansion. During the replacement of noisy pixels, the filter, combined with an edge-preserving weighting algorithm, assigns weights to each pixel in the neighborhood according to its gradient magnitude. Pixels with larger gradient magnitudes are assigned lower weights. The replacement value is determined by a weighted median, thus suppressing impulse noise while protecting the steepness of the pin edges, providing a foundation for subsequent sub-pixel localization.

[0038] Step S202: Gradient Calculation and Coarse Edge Localization. After preprocessing, the system performs gradient scanning on the entire image. A 3x3 convolution kernel of the Sobel operator is used to convolve with the image to obtain the gradient component along the horizontal direction for each pixel. and the gradient component in the vertical direction Then, according to the formula:

[0039] Calculate the gradient magnitude G using the formula:

[0040] Calculate gradient direction .in, and , , respectively, represent the first-order gradient components of the pixel in the horizontal and vertical directions, and G is the composite gradient magnitude, characterizing the intensity at which the pixel belongs to an edge. The gradient direction angle indicates the direction of the normal to the edge.

[0041] Then, the system calculates the grayscale histogram of the current image, takes the grayscale value corresponding to the cumulative distribution reaching 90% as the dynamic grayscale threshold, determines the pixels with gradient magnitudes below this threshold as background and removes them, and only retains the rough region of interest where the pixels with gradient magnitudes not lower than the threshold are located.

[0042] Within this region, search for local maximum points of gradient magnitude, that is, points in the gradient direction where the magnitude is greater than that of the two adjacent pixels, and use them as coarse feature points.

[0043] Step S203: Sub-pixel level feature precise localization. For each coarse feature point determined in step S202, the system establishes a small search window along its gradient direction, with a window size of 3 pixels wide and 7 pixels long. Within the search window, gray values ​​are extracted at each sampling position along the gradient normal direction, forming a set of discrete one-dimensional gray-level distribution sequences. The system applies a Gaussian fitting algorithm to this sequence for mathematical fitting, and the Gaussian function model expression is:

[0044] Where X represents the sampling position coordinates along the gradient normal direction, f(x) is the gray value at the corresponding sampling position, and A is the peak amplitude of the Gaussian function. The center position of the Gaussian function is the sub-pixel edge position to be solved. Let be the standard deviation of the Gaussian function, and exp represent the exponential function with the natural constant e as the base.

[0045] The parameters were solved using the Levenberg-Marquardt iterative optimization method. A The optimization objective is to minimize the sum of squared residuals for all sampling points. Initial values ​​for iteration. Set as the range of grayscale values ​​within the search window. Set as the center coordinates of the search window. Set to 1.0; the convergence condition is that the parameter change in two consecutive iterations is less than 1. Or the number of iterations reaches the maximum of 50.

[0046] After the solution is completed, the parameters The estimated value is the location of the extreme center of the gray-scale gradient function, corresponding to the sub-pixel precise location of the pin edge. The positioning result is denoted as the pixel row coordinate u and pixel column coordinate v of the feature point, where u and v are both floating-point numbers with a precision of 0.1 pixels.

[0047] Step S204: Reflective False Feature Point Removal. The system identifies reflective false feature points for each feature point determined in step S203. The second-order gradient magnitude is calculated by applying the Sobel operator again to the first-order gradient field of the image, representing the degree of change in the gray-level change rate.

[0048] At reflective points on the metal surface, grayscale values ​​exhibit local oscillations, with large second-order gradient amplitudes and isolated local distributions. The system sets the second-order gradient threshold to the mean of the second-order gradient field of the current image plus three times the standard deviation. If, within a 5x5 neighborhood of a feature point, the number of pixels with second-order gradient amplitudes exceeding this threshold exceeds 50% of the total number of pixels in the neighborhood, then the point is determined to be a false reflective feature point and is removed from the feature point set.

[0049] In summary, through the coordinated execution of each sub-step in step S2, a set of pin-end feature points with sub-pixel precision are extracted from the dual-view grayscale images. Each feature point is represented by floating-point coordinates u and v, indicating its precise position in the corresponding viewpoint image. This data provides accurate two-dimensional input for the matching of corresponding points and the calculation of three-dimensional spatial coordinates based on the principle of binocular vision in step S3.

[0050] The next step, S3, is a crucial step in transitioning from a two-dimensional image to three-dimensional space, relying on pre-calibrated binocular vision system parameters. This is achieved through the following steps.

[0051] Step S301, Binocular Vision System Calibration: Before performing 3D reconstruction, the two sets of telecentric cameras need to be precisely calibrated to obtain their intrinsic and extrinsic parameters. The calibration process uses a specially made high-precision aluminosilicate glass checkerboard calibration board. The spatial positions of the intersections of the black and white squares on the calibration board have been precisely measured in advance using metrology equipment. During the calibration process, the calibration board is placed on a stage, and by adjusting the pose of the calibration board, multiple sets of checkerboard images in different poses are acquired within the depth of field of the two sets of telecentric cameras, with no fewer than 15 pose changes.

[0052] The acquired images were processed using Zhang Zhengyou's calibration method and its improved model to calculate the parameters of two sets of telecentric cameras, including: intrinsic parameter matrices containing focal length and principal point coordinates; distortion coefficients containing radial and tangential distortion coefficients; and extrinsic parameter matrices between the two cameras, containing rotation matrix R and translation vector T. The rotation matrix R is a 3x3 identity orthogonal matrix describing the rotation relationship between the right camera coordinate system and the left camera coordinate system; the translation vector T is a 3x1 vector describing the position of the origin of the right camera coordinate system in the left camera coordinate system. These calibration parameters will serve as the basic input for subsequent triangulation calculations.

[0053] Step S302, same-name feature point matching: After calibration, the system performs same-name matching on the pin-end feature points in the left and right images extracted in step S2. The matching process is based on the epipolar constraint principle in binocular vision: for each feature point in the left image, its same-name matching point in the right image must lie on a specific epipolar line.

[0054] The system uses the extrinsic parameter matrix R and translation vector T obtained in step S301 to construct the fundamental matrix F, and then calculates the epipolar equation corresponding to the left feature point in the right image. During the one-dimensional search on the epipolar line, the system also introduces feature descriptor similarity evaluation to improve matching reliability. Specifically, the grayscale distribution of the surrounding 11x11 pixel neighborhood is extracted as the feature descriptor of each feature point. Normalized cross-correlation calculations are performed between the left feature point descriptor and the descriptors of each candidate point on the right epipolar line, and the candidate point with the highest cross-correlation value is selected as the matching point.

[0055] To further eliminate mismatches, the system also performs consistency checks based on the geometric topology of the pins, including pin arrangement constraints and adjacent pin spacing constraints. If a matching pair disrupts the known pin arrangement, or if the spacing between adjacent pins after matching deviates from the nominal value by more than a preset range, the matching pair is determined to be a mismatch and is eliminated.

[0056] Step S303: Triangulation to calculate 3D coordinates. After determining pairs of corresponding feature points, the system applies the principle of triangulation to calculate the 3D coordinates. Let the coordinates of a feature point in the left image output in step S2 be... , The coordinates of the corresponding feature points in the image on the right are: , According to the projection matrix of the left telecentric camera. and the projection matrix of the right telecentric camera Establish projection relationships, where the three-dimensional spatial coordinates X, Y, and Z of an object in the world coordinate system satisfy linear transformation equations.

[0057] For the telecentric camera on the left, the projection relationship is: For the right telecentric camera, the projection relationship is: ,in, and These are the column and row coordinates of the pixels for the feature points in the left image, respectively. and , where are the column and row coordinates of the corresponding feature points in the right-hand image, respectively, and X, Y, and Z are the three-dimensional coordinates of the feature points to be solved in the world coordinate system. This is the projection matrix of the left telecentric camera, combined with the intrinsic and extrinsic parameter matrices of the left telecentric camera; This is the projection matrix of the right telecentric camera, combined with the intrinsic and extrinsic parameter matrices of the right telecentric camera.

[0058] Both projection matrices are 3x4 matrices, and their specific values ​​are derived from the camera intrinsic and extrinsic parameters obtained in step S301. Solving the two sets of projection equations simultaneously yields four independent linear equations, while the unknowns to be solved are X, Y, and Z, forming an overdetermined system of equations. Solving this overdetermined system of equations using the least squares method yields the three-dimensional coordinates X, Y, and Z that best approximate the physical reality.

[0059] The world coordinate system is defined as follows: with the center of the stage as the origin, the XY plane is parallel to the stage plane, and the Z-axis is perpendicular to the stage plane and pointing upwards. In this coordinate system, the calculated coordinate values ​​are as follows: X represents the horizontal position of the pin along the stage in the horizontal plane, Y represents the vertical position of the pin along the stage in the horizontal plane, and Z represents the vertical height coordinate value reflecting the degree of pin tilting or sinking.

[0060] Step S304: Coordinate normalization processing. To eliminate calculation deviations caused by inconsistent placement of the receiver head under test on the stage, the system normalizes the three-dimensional coordinates of all reconstructed pin ends. Specifically, the geometric center of the three-dimensional coordinates of all pin ends is calculated, that is, the mean of the X-coordinate, the mean of the Y-coordinate, and the mean of the Z-coordinate of all pin ends are calculated respectively, and this geometric center is used as the new coordinate origin.

[0061] The mean value of each coordinate corresponding to the geometric center is subtracted from the three-dimensional coordinates of each pin's end, thus translating the reconstructed coordinate data into a virtual coordinate system with the geometric center as the origin. After normalization, subsequent coplanarity evaluation depends only on the relative geometric relationships of the pins themselves, eliminating interference from differences in placement position.

[0062] In summary, through step S3, starting from the pin end feature points in the two images output in step S2, the three-dimensional coordinates of all pin ends in the world coordinate system are reconstructed through binocular epipolar matching and triangulation, and then normalized to eliminate placement bias. This set of three-dimensional coordinate data completely describes the actual position of each pin in space, where the coordinate offset in the Z-axis direction directly reflects the height change of the pin relative to the stage plane, providing accurate three-dimensional input data for the pin coplanarity calculation and quantitative evaluation based on the fitted reference plane in step S4.

[0063] Regarding step S4, after obtaining the three-dimensional spatial coordinates of all pin ends in step S3, this step aims to construct a geometric evaluation benchmark and quantify the degree of deviation of each pin in the vertical direction. This is specifically achieved through the following sub-steps.

[0064] Step S401: Construct a fitting reference plane using the least squares method. The system extracts the three-dimensional spatial coordinates of all pin ends output in step S3 and applies the least squares method to construct the fitting reference plane. The mathematical process is as follows: find the plane equation that minimizes the sum of the squares of the perpendicular geometric distances from all pin end coordinate points to the plane.

[0065] Let the general expression for the equation of a plane be:

[0066] Where X, Y, and Z are the three-dimensional coordinate variables of a point in space, A, B, and C are the three components of the plane normal vector, and D is the constant term of the plane equation. The plane normal vector is (A, B, C), and its direction determines the orientation of the plane.

[0067] For the n pin end space points output in step S3, the coordinates of the i-th point are ( , , ), where i takes This represents the total number of pins on the infrared receiver head. The perpendicular geometric distance from this point to the plane to be determined is... The expression is:

[0068] The optimization objective of the least squares method is to find a set of parameters A, B, C, and D such that the sum of the squares of the distances from the endpoints of all n pins to the plane is equal to the sum of the squares of the distances from the endpoints of the pins to the plane. To minimize the value, constraints are introduced during the solution process to avoid trivial solutions where A, B, and C are all zero simultaneously. The normal vector of the plane to be determined is constrained to a unit vector.

[0069] Under this constraint, the optimization problem is transformed into finding the eigenvector corresponding to the minimum eigenvalue of the covariance matrix of each point. The three components of this eigenvector are the optimal solutions for A, B, and C. Then, the geometric center of the point set is substituted into the plane equation to solve for D.

[0070] The resulting fitting reference plane objectively reflects the ideal average mounting state of the infrared receiver head on the printed circuit board.

[0071] Step S402: Constructing the reference plane using the minimum multi-point method. As an alternative, the coplanarity calculation can also use the minimum multi-point method to construct the reference plane. In this mode, the system traverses all pin end coordinates output in step S3, compares the Z coordinate values ​​of each point, and searches for the three or four coordinate points with the smallest Z value in the negative Z-axis direction.

[0072] These points are considered as the actual support points where the infrared receiver contacts the board surface due to gravity when it is actually placed on the printed circuit board. The system uses these three or four lowest points as reference points and employs a three-point plane determination method or a least-squares method to fit a four-point plane, constructing an alternative reference plane. Compared to the least-squares fitted plane, this plane more realistically simulates the physically stable posture of the infrared receiver before surface mounting and soldering due to gravity.

[0073] Step S403, Pin coplanarity deviation calculation: After the reference plane is constructed, the system sequentially calculates the coordinates of the end points of each pin. , , The perpendicular geometric distance from the reference plane .

[0074] The sign of the distance value is determined by which side of the plane the point is located on: A positive value indicates that the pin is tilted upwards relative to the reference plane. A negative value indicates that the pin is recessed downwards relative to the reference plane. These signed distance values ​​are the core quantitative indicators for measuring the coplanarity of each pin.

[0075] During the calculation process, the system simultaneously applies a statistical filtering algorithm to screen for outliers. Specifically, the Sigma criterion is used: first calculate all n distance values. mean S and standard deviation S, if a certain distance value Exceeding If the range is within a certain range, then the point is marked as a statistical outlier.

[0076] After outlier labeling, the baseline plane construction and distance calculation are re-executed using the remaining point set after outlier removal: if the least squares rule was used to refit the plane previously, or if the least squares rule was used to reselect the lowest point from the remaining points. This process ensures that the fitted baseline plane and the evaluation of coplanarity deviation are not affected by individual severely deformed points.

[0077] In summary, step S4, based on the three-dimensional coordinates of the pin ends output in step S3, constructs a reference plane that conforms to the actual mounting state using the least squares method or the least-multipoint method, and calculates the vertical geometric distance of each pin to this plane, while eliminating outlier interference. This set of signed distance values ​​completely quantifies the offset magnitude and direction of each pin relative to the ideal mounting plane, where... The absolute value and sign correspond to the offset and the direction of tilting or sinking, respectively, providing a direct comparison basis for the classification and determination of whether the data is qualified or not based on the tolerance threshold in step S5.

[0078] Finally, step S5 is the result determination. After calculating the coplanarity deviation value of each pin in step S4, this step is responsible for generating the final decision instruction based on the measurement data, driving the sorting mechanism to complete the separation of qualified and unqualified products. This is specifically implemented through the following sub-steps.

[0079] Step S501: Tolerance threshold setting. The system has a preset tolerance threshold, which is determined based on the process specifications and solder paste thickness tolerance of SMT surface mount technology, and is set to 0.05 mm. The tolerance threshold is stored in the system's non-volatile memory for use in the judgment stage.

[0080] Step S502: Compare the deviation value with the threshold, and the system extracts the vertical geometric distance of each pin output in step S4. Take the absolute value of each one. It compares the values ​​with a preset tolerance threshold in real time. If all pins... If all values ​​are less than or equal to the tolerance threshold, the product is deemed to have passed the coplanarity test; if any pin has a tolerance threshold, the product is deemed to have passed the coplanarity test. If the tolerance threshold is exceeded, the product is determined to have a coplanarity defect, and the result is deemed unqualified.

[0081] Step S503: Output the judgment signal. The central processing unit generates a corresponding appearance inspection judgment signal based on the comparison result. The judgment signal is a binary switch signal; a high level indicates pass, a low level indicates fail, or vice versa, depending on the system configuration. This signal is sent to the external selection mechanism via an industrial bus. The industrial bus can use one of the following industrial fieldbus protocols: EtherCAT, Profinet, or Ethernet / IP.

[0082] Step S504: The sorting mechanism executes the physical sorting action after receiving the judgment signal. The sorting mechanism includes a pneumatic slider or an electromagnetic deflector plate. When the judgment signal is unqualified, the actuator moves to guide the test object to the unqualified product collection trough; when the judgment signal is qualified, the actuator remains in place, and the test object continues to move along the conveyor belt to the next process.

[0083] The entire detection, judgment and sorting process is completed within a preset 100-millisecond cycle, meeting the online full inspection requirements of high-speed production lines.

[0084] At this point, step S5, based on the coplanarity deviation values ​​of each pin output in step S4, compares them point by point with the preset tolerance threshold to complete the automatic classification and determination of qualified and unqualified products, and drives the sorting mechanism to perform the corresponding physical sorting action within a 100-millisecond cycle, thereby realizing online full inspection and real-time sorting of the coplanarity of the patch infrared receiver pins.

[0085] During the operation of the appearance inspection method, the system also performs a series of auxiliary monitoring and optimization tasks to ensure the stability of the inspection accuracy under long-term and variable working conditions.

[0086] Ambient temperature compensation is one of the auxiliary tasks. The system mounts a PT100 high-precision temperature sensor on the metal bracket of the telecentric camera to collect and detect the ambient temperature value in real time. Due to the thermal expansion and contraction effect of the metal bracket under the influence of day-night or seasonal temperature differences in the industrial workshop, the spatial position of the two sets of telecentric cameras relative to the stage will drift at the micrometer level, thus affecting the accuracy of the projection matrix.

[0087] To address this, the system incorporates a mathematical compensation model based on temperature gradients. This model pre-establishes a mapping relationship between temperature changes and projection matrix correction coefficients through calibration experiments: In a controlled-temperature chamber, the detection system is gradually heated and cooled within a range of 5°C to 45°C, recording the temperature value every 2°C change. Simultaneously, a high-precision checkerboard calibration plate is used to recalibrate the projection matrix. The projection matrix at each temperature point is compared one by one with the projection matrix at the standard temperature of 20°C, and the correction coefficients corresponding to each temperature point are calculated, forming a temperature-correction coefficient lookup table.

[0088] In actual operation, the system retrieves the corresponding correction coefficient from the lookup table or obtains it through linear interpolation based on the temperature value collected in real time by the PT100 sensor. The system then uses this correction coefficient to fine-tune the values ​​of each element in the projection matrix in real time to compensate for the camera pose shift caused by thermal deformation, thus ensuring the long-term stability of detection accuracy in industrial workshops with large temperature differences.

[0089] In addition, the method also involves the detection of pin horizontal features to achieve a comprehensive evaluation of the overall pin deformation.

[0090] The system projects the three-dimensional coordinates of all pin ends reconstructed in step S3 onto a preset horizontal reference plane along the Z-axis. This horizontal reference plane is the Z=0 plane, which coincides with the stage plane. After projection, the two-dimensional coordinates of the i-th pin end on the horizontal reference plane are ( , ).

[0091] The system calculates the Euclidean distance between the projection points of adjacent pins according to their physical arrangement on the infrared receiver head. The calculation formula is as follows: ,in The horizontal spacing between the i-th pin and the (i+1)-th pin is given. The horizontal spacing deviation of the pins is obtained by comparing the spacing value of each adjacent pin with the nominal spacing value.

[0092] Simultaneously, the system extracts the edge sampling points of each pin obtained in step S2, and fits the edge line equations to the two long sides of each pin. The fitting adopts the least squares method, fitting the sub-pixel sampling points of the same side edge to the line equation y=kx+b, where k is the slope of the line and b is the intercept of the line on the y-axis.

[0093] Based on the slope of the straight lines on both sides, the direction of the central axis of the pin is calculated, and then the lateral tilt angle of this central axis relative to the preset reference axis is obtained. The reference axis is taken as the X-axis direction of the stage coordinate system. The lateral tilt angle and the deviation of the spacing between adjacent pins together constitute the pin horizontal feature detection result, which is used to evaluate whether the pin has lateral bending or swaying deformation.

[0094] All detection data, including the three-dimensional coordinates output in step S3, the coplanarity distance value output in step S4, the judgment result in step S5, the ambient temperature compensation data, and the pin horizontal spacing and tilt angle, are recorded in real time and stored in a local or cloud database.

[0095] The local database uses industrial-grade solid-state drives for storage, while the cloud database is uploaded to the manufacturing execution system via a workshop industrial Ethernet network. The stored data is used to create production quality control charts, tracking the batch distribution trends of coplanarity deviations and pin pitch deviations using statistical process control methods. This helps process engineers identify anomalies in upstream processes such as mold wear and lead trimming parameter deviations, enabling feedback from finished product inspection to process control.

[0096] On the other hand, the machine vision-based patch-type infrared receiver appearance inspection system disclosed in this application includes: The stage is used to fix the patch-type infrared receiver head to be tested. Bottom collimating backlight is used to generate a collimated beam that illuminates the patch-type infrared receiver head under test; The left and right telecentric cameras are symmetrically arranged on both sides of the patch infrared receiver head under test. Both are equipped with object-side telecentric lenses, and their optical axes are arranged at a 45-degree angle to the stage plane, so as to synchronously acquire pin images at the same physical moment. The hardware-level synchronous trigger control unit includes a high-speed pulse generator, which consists of a central processing unit FPGA and a phase-locked loop module integrated inside the FPGA. The high-speed pulse generator is used to generate two trigger pulses with a synchronization time deviation controlled within 10 nanoseconds to drive the left telecentric camera and the right telecentric camera to expose synchronously. The image processing and 3D reconstruction unit is used to perform steps S2 and S3, wherein step S2 includes subpixel-level precise feature localization and removal of reflective pseudo-feature points, and step S3 includes calculating the 3D spatial coordinates of the pin end based on epipolar constraints and triangulation principles. The coplanarity evaluation and judgment unit is used to extract the three-dimensional spatial coordinates of all pin ends, construct a reference plane using the minimum multi-point method, calculate the vertical geometric distance of each pin to the reference plane, and perform statistical filtering according to the 3-Sigma criterion. Then, it compares the deviation value with the preset tolerance threshold and outputs the appearance inspection judgment signal.

[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0098] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for appearance inspection of a patch-type infrared receiver head based on machine vision, characterized in that, Includes the following steps: Step S1, Image Acquisition: Fix the patch infrared receiver under test onto the stage, and illuminate the patch infrared receiver under test with a bottom collimated backlight; use the left telecentric camera and the right telecentric camera, which are symmetrically arranged on both sides of the patch infrared receiver under test, to synchronously acquire pin images of the patch infrared receiver under test at the same physical moment; both the left telecentric camera and the right telecentric camera are equipped with object-side telecentric lenses, and their optical axes are arranged at a 45-degree angle to the plane of the stage; Step S2, Feature Extraction: Perform sub-pixel level feature precise localization on the two acquired pin images, extract key feature points representing the physical end of the pin, and remove reflective pseudo feature points; Step S3, 3D reconstruction: Based on the pre-calibrated binocular vision system parameters, perform corresponding point matching on the key feature points extracted from the left and right images, and apply the principle of triangulation to calculate the 3D spatial coordinates of all pin ends in the world coordinate system. Step S4, Coplanarity Evaluation: Extract the three-dimensional spatial coordinates of all pin ends, and construct a reference plane using the minimum multi-point method. That is, traverse all pin end coordinates and search for at least three points with the smallest Z coordinate values ​​in the negative Z-axis direction. Based on the at least three lowest points, fit a reference plane representing the physically stable attitude. Calculate the vertical geometric distance from each pin end coordinate point to the reference plane in turn. This distance value is used as a quantitative index of coplanarity deviation. Step S5, result determination: Compare the absolute value of the coplanarity deviation of each pin with the preset tolerance threshold. If the absolute value of any pin exceeds the tolerance threshold, the patch infrared receiver head under test is determined to be unqualified; otherwise, it is determined to be qualified, and the corresponding appearance inspection determination signal is output.

2. The method according to claim 1, characterized in that, The synchronous acquisition at the same physical moment described in step S1 is achieved by a hardware-level synchronous trigger control mechanism, which is executed by a high-speed pulse generator consisting of an FPGA-based central processing unit and a phase-locked loop module integrated inside the FPGA. At the moment the detection cycle starts, the central processing unit generates two rising edge trigger pulses simultaneously through the phase-locked loop module, and outputs them to the hardware trigger input ports of the left telecentric camera and the right telecentric camera respectively. The synchronization time deviation between the two trigger pulses is controlled within 10 nanoseconds.

3. The method according to claim 1, characterized in that, The sub-pixel level precise feature localization mentioned in step S2 specifically includes: Gradient calculations are performed on the pin image to determine coarse feature points; For each coarse feature point, a search window is established along its gradient direction. Within the search window, gray values ​​are extracted at each sampling position along the gradient normal direction to form a discrete one-dimensional gray distribution sequence. The Levenberg-Marquardt iterative optimization algorithm is applied to fit a Gaussian function to the one-dimensional grayscale distribution sequence. The center position of the obtained Gaussian function is used as the sub-pixel level precise position of the pin edge. The Gaussian function model expression is as follows: Where X is the sampling position coordinate along the gradient normal direction, f(x) is the gray value at the corresponding sampling position, and A is the peak amplitude. Let the sub-pixel edge position be the one to be solved. The standard deviation is 0.1; the positioning accuracy reaches 0.1 pixels.

4. The method according to claim 3, characterized in that, The removal of reflective pseudo-feature points in step S2 specifically includes: applying the Sobel operator again to the first-order gradient field of the image to calculate the second-order gradient magnitude, and setting the second-order gradient threshold to the mean of the second-order gradient field of the current image plus 3 times the standard deviation. If the number of pixels in a 5x5 neighborhood of a certain feature point whose second-order gradient magnitude exceeds the threshold exceeds 50% of the total number of pixels in the neighborhood, then the point is determined to be a reflective pseudo-feature point and is removed from the feature point set.

5. The method according to claim 1, characterized in that, The matching of corresponding points in step S3 is based on the principle of epipolar constraint and feature descriptor similarity evaluation, and introduces pin arrangement order constraint and adjacent pin spacing constraint for consistency verification to eliminate mismatched point pairs.

6. The method according to claim 1, characterized in that, The coplanarity evaluation described in step S4, after calculating the vertical geometric distance from the coordinate point at the end of each pin to the reference plane, also includes a statistical filtering step based on the 3-Sigma criterion: calculating the mean and standard deviation of all vertical geometric distance values, marking points that exceed the mean ± 3 times the standard deviation as statistical outliers and removing them, and re-performing the step of constructing the reference plane and calculating the distance using the minimum multi-point method with the remaining set of points after removing outliers, to ensure that the coplanarity evaluation is not interfered with by individual severely deformed pins.

7. The method according to claim 1, characterized in that, The bottom collimated backlight source mentioned in step S1 uses a collimated beam with a divergence angle of less than 2 degrees, and narrowband interference filters are installed at the front end of the lenses of the left telecentric camera and the right telecentric camera. The center wavelength of the narrowband interference filter is strictly matched with the peak emission wavelength of the bottom collimated backlight source, which is selected as 630 nanometers, and the full width at half maximum (FWHM) bandwidth is limited to within 20 nanometers to physically shield stray light interference.

8. The method according to claim 1, characterized in that, The step S1, which involves fixing the patch-type infrared receiver head to be tested onto the stage, is achieved through a vacuum adsorption system integrated inside the stage. The vacuum adsorption system includes an air pump, an air pipe, and an array of micropores on the surface of the stage. When the air pump is activated, negative pressure is generated, and the main body of the patch-type infrared receiver head to be tested is firmly locked at the preset coordinate origin through the micropores to prevent micron-level displacement during the detection period.

9. The method according to claim 1, characterized in that, The method also includes an environmental temperature compensation task, which involves real-time acquisition of the temperature value of the detection environment by using PT100 high-precision temperature sensors installed on the metal brackets of the left and right telecentric cameras, and forming a temperature-correction coefficient lookup table based on the mapping relationship between the temperature change and the projection matrix correction coefficient established by the pre-calibration experiment. The system retrieves or interpolates the corresponding projection matrix correction coefficient from the lookup table based on the real-time collected temperature value, and uses the projection matrix correction coefficient to fine-tune the values ​​of each element in the projection matrix in real time to compensate for the camera pose shift caused by thermal deformation.

10. A machine vision-based patch-type infrared receiver appearance inspection system, characterized in that, The system for implementing the method of any one of claims 1 to 9 comprises: The stage is used to fix the patch-type infrared receiver head to be tested. A bottom collimated backlight is used to generate a collimated beam that illuminates the patch-type infrared receiver head under test. The left and right telecentric cameras are symmetrically arranged on both sides of the patch infrared receiver head under test. Both are equipped with object-side telecentric lenses, and their optical axes are arranged at a 45-degree angle to the plane of the stage, so as to synchronously acquire pin images at the same physical moment. The hardware-level synchronous trigger control unit includes a high-speed pulse generator, which consists of a central processing unit FPGA and a phase-locked loop module integrated inside the FPGA. The high-speed pulse generator is used to generate two trigger pulses with a synchronization time deviation controlled within 10 nanoseconds to drive the left telecentric camera and the right telecentric camera to be exposed synchronously. The image processing and 3D reconstruction unit is used to perform steps S2 and S3, wherein step S2 includes subpixel-level precise feature localization and removal of reflective pseudo-feature points, and step S3 includes calculating the 3D spatial coordinates of the pin end based on epipolar constraints and triangulation principles. The coplanarity evaluation and judgment unit is used to extract the three-dimensional spatial coordinates of all pin ends, construct a reference plane using the minimum multi-point method, calculate the vertical geometric distance of each pin to the reference plane, and perform statistical filtering according to the 3-Sigma criterion. Then, it compares the deviation value with the preset tolerance threshold and outputs the appearance inspection judgment signal.

Images