A hand model detection method based on image recognition

Abstract

This application discloses a hand-made model detection method based on image recognition. A grid calibration target fixed to a platform is used to synchronously acquire calibration images during the detection cycle. The coordinates of the checkerboard corner points are extracted. Based on the ratio of the image spacing to the physical spacing between adjacent corner points, a scale conversion factor is calculated, and a correction mapping table covering the entire field of view is generated. The homography matrix is ​​solved using the physical and image coordinates of the corner points, and the rotation matrix is ​​obtained by decomposition. The angular deviation between the optical axis and the normal of the calibration target is calculated. Based on the corrected optical parameters, detection images are acquired. The coordinates of defective pixels are sequentially mapped from the homography matrix to the physical coordinates of the platform, and then from the rigid transformation matrix to the coordinates of the robotic arm base. The defective pixel size is extracted using a defect recognition algorithm. The correction mapping table is used to correct the size pixel by pixel to the physical size. The result is compared with a preset threshold to determine whether the product is qualified. The detection result and the physical coordinates of the defective product are output, effectively reducing the sorting and grasping offset rate.

Description

Technical Field

[0001] This invention relates to the field of prototype model detection technology, and in particular to a prototype model detection method based on image recognition. Background Technology

[0002] During the manufacturing process, the surface and internal quality of prototype models (i.e., rapid prototyping samples) directly affect the product development cycle and final assembly accuracy. With the rapid development of industrial cameras and image processing technology, automated inspection methods based on machine vision are gradually replacing traditional manual visual inspection, demonstrating high efficiency and high consistency in identifying defects such as micron-level scratches, dents, and bubbles.

[0003] However, existing visual inspection technologies for prototype models still have several unresolved issues.

[0004] Firstly, prototype models often use transparent or semi-transparent materials (such as photosensitive resin and transparent epoxy), and frequently contain complex internal cavity structures (such as gaps between reinforcing ribs, weight-reducing holes, and assembly clearances). In optical inspection, light undergoes transmission, refraction, and scattering within the material, and total internal reflection easily occurs at the material-air interface. This results in images containing both real defects (scratches, bubbles, inclusions) and optical artifacts such as refracted light spots and internal reflections caused by material properties and geometry. Existing image recognition systems based on opaque materials struggle to distinguish these artifacts from real defects, and internal defects are completely invisible due to total internal reflection, leading to serious missed detections and false alarms.

[0005] Secondly, due to optical design and manufacturing tolerances, industrial camera lenses inevitably suffer from radial and tangential distortion. This results in inconsistencies in pixel equivalents between the image edge region and the central region, leading to systematic deviations in dimensional measurements when the same defect is located at different positions within the field of view. Existing methods typically perform a single calibration during system installation and then continue to use the system long-term, failing to account for the drift of lens distortion over time or environmental changes. Consequently, it is difficult to maintain consistent micrometer-level dimensional measurement accuracy across the entire field of view.

[0006] Third, misalignment between the optical axis and the stage normal during camera installation is a common engineering problem. This misalignment causes a deviation between the preset illumination angle and the actual incident angle, resulting in distortion of the highlight distribution and shadow morphology in low-resolution optical response images. This, in turn, interferes with the accurate extraction of specular reflection intensity and shadow distortion, affecting the selection of optimal imaging parameters.

[0007] Fourth, for defects inside multi-layered cavities, external light sources cannot directly illuminate due to total internal reflection. Traditional methods can only optimize surface imaging by adjusting the angle of external illumination, which cannot solve the fundamental problem of the internal light path being blocked by the cavity interface.

[0008] In summary, existing technologies face multiple problems in defect detection of prototype models, such as non-real-time distortion correction, uncompensated optical axis eccentricity, accumulated coordinate transformation errors, and invisible defects inside cavities. There is an urgent need for a detection method that can systematically solve the above problems. Summary of the Invention

[0009] This application provides a hand-made model detection method based on image recognition. By generating a correction mapping table, pixel-by-pixel spatial domain interpolation correction is performed in the defect size quantization stage, which effectively eliminates the influence of lens radial and tangential distortion on size measurement. It realizes the accurate conversion of defect position from pixel coordinates to the physical coordinates that can be executed by the robotic arm, avoids the cumulative error in the coordinate conversion process, and reduces the grasping coordinate offset rate of the sorting system.

[0010] This application provides a hand-made model detection method based on image recognition, including: S101, a grid calibration target is fixed on the reference surface of the stage. During the detection image acquisition cycle, the calibration image of the grid calibration target is acquired synchronously. The checkerboard pattern and corner image coordinates of the calibration target are identified based on the calibration image. The scale conversion factor is calculated based on the spacing between adjacent corner image coordinates and the spacing between corner physical coordinates, and a correction mapping table is generated. S102, based on the physical coordinates and image coordinates of the corner points, obtain the angle deviation between the optical axis direction vector and the calibration target normal direction vector, and correct the specular reflection intensity and shadow distortion degree according to the angle deviation; S103, based on the corrected specular reflection intensity and shadow distortion degree, extract the corrected corner point image coordinates, and use the corrected corner point image coordinates as pixel coordinates to perform a three-level transformation to obtain the physical coordinates of the defect in the robot arm coordinate system; wherein, the three-level transformation includes a first-level transformation, a second-level transformation and a third-level transformation. S104 identifies defects and extracts defect pixel dimensions using a defect recognition algorithm. The extracted defect pixel dimensions are then corrected using a correction mapping table. The corrected defect dimensions are compared with a set threshold to determine whether the prototype model is qualified, and the detection result is output.

[0011] Preferably, generating the calibration mapping table includes: fixing a grid calibration target within a reserved area at the edge of the stage reference surface, with the geometric center of the calibration target defined as the origin of the stage's physical coordinate system, and the row and column directions of the checkerboard grid defined as the positive directions of the X and Y axes, respectively; acquiring calibration images, performing corner detection on the calibration images, extracting sub-pixel-level image coordinates of the checkerboard grid corners, and simultaneously performing ellipse fitting on the circular markers to extract the center coordinates as supplementary control points; calculating the pixel distance between adjacent corners based on the extracted corner image coordinates, and calculating the pixel-physical scale conversion factor for each local region by combining the known physical distance between adjacent corners on the calibration target; and associating all calculated scale conversion factors with their corresponding image coordinate positions to construct a calibration mapping table.

[0012] Preferably, the acquisition of calibration images includes synchronous acquisition within the same frame and stage translation compensation acquisition; synchronous acquisition within the same frame includes: the calibration target is located in the reserved area at the edge of the camera's field of view, and is simultaneously imaged on different positions on the sensor target surface along with the hand model to be detected; after the detection image acquisition is completed, the imaging unit is triggered to acquire the next frame image, and the lighting conditions and camera exposure parameters are kept consistent with the previous frame detection image.

[0013] Preferably, the stage translation compensation acquisition includes: when the calibration target is blocked by the hand model to be tested, a movement command is sent to the displacement platform below the stage to move a preset distance along the negative X-axis. After the translation is completed, the imaging unit acquires the calibration image with the same exposure parameters as the detection image. After the acquisition is completed, the stage is reset.

[0014] Preferably, the three-level transformation of the corrected corner image coordinates as pixel coordinates to obtain the physical coordinates of the defect in the robot arm coordinate system includes: the first-level transformation maps the defect pixel coordinates to planar coordinates in the physical coordinate system of the stage through the homography matrix; the second-level transformation maps the physical coordinates of the stage to the robot arm base coordinate system through the rigid transformation matrix; the third-level transformation embeds the homography matrix and the rigid transformation matrix into the system firmware, and calls the above two-level transformations after each defect identification to output the three-dimensional physical coordinates of the defect in the robot arm base coordinate system in sequence.

[0015] Preferably, the prototype model detection method further includes: S201, set probe positioning features on grid calibration target to form error calibration target, use robotic arm controller to obtain actual coordinates based on error calibration target; use the difference between the obtained actual coordinates and the physical coordinates obtained after three-level transformation as comprehensive output error, and construct deviation vector field model based on comprehensive output error; S202 embeds the deviation vector field model into the three-level transformation, performs reverse compensation for the errors generated in the three-level transformation, and obtains the compensated physical coordinates.

[0016] Preferably, the prototype model detection method further includes: S301 extracts cavity data from the design drawings of the prototype model, determines the cavity area, scans the critical angle of total internal reflection in the cavity area, and the imaging unit acquires images irradiated at different incident angles; compares the incident angle and the critical angle of total internal reflection of each frame of image to identify the optimal incident angle; S302 calculates the penetration depth of the evanescent wave based on the optimal incident angle, and determines whether the light can penetrate the multi-layer cavity based on the penetration depth. If it can penetrate the multi-layer cavity, it illuminates different cavity areas through the optimal incident angle and acquires detection images.

[0017] Preferably, identifying the optimal incident angle includes: acquiring images by changing the incident angle in step size and extracting the wall brightness, and drawing a brightness curve; the inflection point of the curve is the critical angle for total internal reflection, and the local peak brightness above the critical angle is the optimal incident angle.

[0018] One or more technical solutions provided in this application have at least the following technical effects or advantages: by generating a correction mapping table, pixel-by-pixel spatial domain interpolation correction is performed in the defect size quantization process, which effectively eliminates the influence of lens radial distortion and tangential distortion on size measurement, realizes the accurate conversion of defect position from pixel coordinates to the physical coordinates that can be executed by the robotic arm, avoids the cumulative error in the coordinate conversion process, and reduces the grasping coordinate offset rate of the sorting system. Inverse compensation is performed on the deviation vector field to reduce the overall system error of the defect physical coordinate output; By utilizing the evanescent wave resonance tunneling effect, light energy can bypass the total internal reflection interface and enter the cavity that was originally inaccessible by optical means, making the defects inside the cavity detectable and imageable, thus fundamentally solving the problem of optical occlusion of defects inside the cavity of the transparent prototype model. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a hand-made model detection method based on image recognition according to the present invention. Figure 2 This is a schematic diagram of the joint error calibration process of the present invention; Figure 3 This is a schematic diagram of the process for evanescent wave resonance tunneling imaging in the cavity region according to the present invention. Detailed Implementation

[0020] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0021] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] Example 1: Figure 1 This is a flowchart illustrating a hand-based prototype model detection method according to an embodiment of the present invention, including: S101, fix the grid calibration target on the reference surface of the stage, synchronously acquire the calibration image of the grid calibration target in each detection image acquisition cycle, identify the checkerboard pattern and corner image coordinates of the calibration target based on the calibration image, calculate the scale conversion factor based on the spacing between adjacent corner image coordinates and the spacing between corner physical coordinates, and generate a correction mapping table. Furthermore, a grid calibration target is fixed on the reference surface of the stage. The base material of the grid calibration target is alumina ceramic, which can effectively suppress the grid spacing changes caused by ambient temperature fluctuations, and withstand the vibrations generated by frequent stage movements and wiping and cleaning operations without damaging the surface accuracy. The surface of the calibration target is treated with diffuse reflection sandblasting to ensure that the imaging unit can receive stable and uniform reflected light intensity at any field of view, avoiding local overexposure of the grid pattern due to changes in the light source angle. The composite pattern of the grid calibration target consists of a checkerboard array and a circular marker array. The checkerboard array consists of 19×19 internal corner points (i.e., the intersection points of adjacent black and white squares). The checkerboard pattern has a clear topological structure, and the four neighboring areas of each internal corner point are arranged in alternating black and white. The main function of the checkerboard is to correct distortion. The system provides a regularly distributed control point grid. The circular marker array refers to embedding a circular marker at each corner point of the checkerboard grid. The circular marker coincides with the center of the checkerboard grid corner point. The circular marker provides sub-pixel-level center coordinates through ellipse fitting, serving as an independent check and supplement for checkerboard grid corner point detection. The grid calibration target is located at the upper right corner of the stage reference plane. This area is outside the placement area of ​​the hand model to be detected and is within the edge reserved area of ​​the imaging unit's field of view. The upper surface of the calibration target is coplanar with the upper surface of the stage. The geometric center of the calibration target (i.e., the central intersection of the checkerboard grid array) is defined as the origin of the stage's physical coordinate system. The row direction (from left to right) of the checkerboard grid is defined as the positive X-axis direction, the column direction (from near to far) is defined as the positive Y-axis direction, and the direction perpendicular to the target surface upwards is defined as the positive Z-axis direction.

[0024] Calibration images of the grid calibration target are acquired under two scenarios. The first scenario involves synchronous acquisition within the same frame. The calibration target is located in a reserved area at the edge of the camera's field of view. In the detection optical path, it and the hand model to be tested are simultaneously imaged at different positions on the sensor target surface. After acquiring the high-resolution detection image, the imaging unit is immediately triggered to acquire the next frame. At this time, the lighting conditions and camera exposure parameters are completely consistent with the previous frame, ensuring that the brightness characteristics of the calibration target image synchronously reflect the actual optical response under the current lighting conditions. The calibration target image and the detection image are separated by only one frame in time. The imaging state of the fixed target can accurately reflect the lens distortion and illumination distribution at the current moment; the second scenario is stage translation compensation acquisition. When the calibration target is installed, the pixel coordinates of the target center in the current field of view are recorded. After the detection image is acquired, the pixel value at that coordinate position is checked. If the area is occluded, translation compensation is activated. The system sends a movement command to the displacement platform below the stage, moving a preset distance Δx along the negative X-axis. The movement time is 30~50ms. After the translation is completed, the imaging unit acquires a frame of calibration image with the same exposure parameters as the detection image. The time-calibrated target has entered the unobstructed area at the center of the field of view, and the grid pattern is clearly visible. Calibration images were acquired through the two scenarios described above. These images were then Gaussian filtered, and functions from the OpenCV library were used to coarsely locate the corner points, obtaining their coordinates. For each coarsely located corner point, an 11×11 pixel local window was taken centered on it, and a sub-pixel corner refinement algorithm was performed within the window to obtain the finely located corner point coordinates. Based on the coarse coordinates of the checkerboard corner points, a sub-image was extracted from a circular area with a radius of 10 pixels centered on each corner point. The Canny edge detection operator is applied within the image to extract the edge pixel set of the circular marker. For the edge pixel set, a direct least squares ellipse fitting algorithm is used to obtain the coordinates of the center of the circular marker. After extracting the coordinates of all corner points, they are aligned, sorted, and matched. Using the direction vectors of all adjacent corner point pairs, the rotation angle of the raster in the image coordinate system is estimated by histogram voting. Starting from the top left corner point of the image, all corner points are traversed sequentially along the estimated principal direction and vertical direction. Each corner point is assigned its row and column number, and a 19×19 corner point image coordinate matrix is ​​output.

[0025] The coordinates of the checkerboard corner points extracted from the calibration image are used to generate a calibration mapping table by calculating the image distance between adjacent corner points pair by pair and establishing a proportional relationship with the known physical spacing. Specifically, for each pair of horizontally adjacent corner points, the image pixel distance between them is: ,in, This represents the Euclidean pixel distance, i.e., the horizontal pixel distance, between a corner point located in the i-th row and j-th column of the calibration target array and its horizontally adjacent corner point to its right. This represents the x-coordinate of a corner point located in the i-th row and j-th column of the calibration target array in the image coordinate system. Let represent the ordinate of the corner point located in the i-th row and j-th column of the calibration target array in the image coordinate system. Indicates and The x-coordinate of the image of the adjacent corner point on the right. Indicates and The vertical coordinates of the right-side adjacent corner points are calculated using the same formula as the vertical pixel distance between them. Since the physical distance between adjacent corner points on the calibration target is a known constant, the horizontal scale conversion factor can be calculated for the local region corresponding to each pair of adjacent corner points using the following formula: ,in, This is the horizontal scale conversion factor. To calibrate the physical distance between adjacent corner points on the target, Indicates the horizontal pixel distance within the calibration target array, and the vertical scale conversion factor: ,in, This is the vertical scale conversion factor. To calibrate the physical distance between adjacent corner points on the target, The vertical pixel distance, horizontal scale conversion factor, and vertical scale conversion factor reflect the actual physical size of a single pixel within the current local region. The horizontal and vertical scale conversion factors are constructed into horizontal and vertical sample sets, respectively. Each sample set contains the midpoint image coordinates and either the horizontal or vertical scale conversion factor of the adjacent corner pair. A 20×20 uniform grid is divided. For each grid node, all sample points are searched within a radius R=150 pixels centered on that node. Inverse distance weighted interpolation is performed on the searched sample points to estimate the horizontal conversion factor at that node. The formula is as follows: ,in, Represents the coordinates of the grid nodes. At this point, the horizontal scale conversion factor is obtained after interpolation calculation, where N is the total number of sample points participating in the interpolation calculation for this node. Let be the weight coefficient of the t-th sample point. Let t be the horizontal scale conversion factor of the t-th sample point. The vertical scale conversion factor is obtained by interpolation in the same way. Since lens distortion is usually radially symmetrical, the horizontal and vertical conversion factors should ideally be equal. If the relative deviation between the two is greater than the preset deviation threshold, the weighted average of the two is taken as the final scale conversion factor. The scale conversion factors of all network nodes are stored in correspondence with their coordinates to form a two-dimensional correction mapping table.

[0026] S102, based on the physical coordinates and image coordinates of the corner points, obtain the angle deviation between the optical axis direction vector and the calibration target normal direction vector, and correct the specular reflection intensity and shadow distortion degree according to the angle deviation; Specifically, based on the acquired image coordinates of all corner points on the calibration target and their corresponding known physical coordinates in the physical coordinate system of the stage, since the upper surface of the calibration target coincides with the reference plane of the stage, the vertical coordinates of these corner points are all zero. Based on the pinhole model of camera imaging, the image coordinates and physical coordinates satisfy a homography mapping relationship. This homography mapping relationship is described by a 3x3 homography matrix, which includes an arbitrary scale factor. Using all the acquired corner point coordinate pairs, the initial estimate of the homography matrix is ​​obtained through a direct linear transformation method. Then, a nonlinear optimization algorithm is used to refine the result, thus obtaining the homography matrix. The homography matrix is ​​then decomposed into a rotation matrix and... Specifically, the homography matrix is ​​equal to the camera intrinsic matrix multiplied by a matrix composed of the first two column vectors of the rotation matrix and the translation vector. The camera intrinsic matrix is ​​a pre-defined matrix. By multiplying the inverse of the intrinsic matrix by the first and second columns of the homography matrix, two initial rotation vectors are obtained. These two rotation vectors are normalized, and the third column vector is calculated using the cross product operation in three-dimensional space. The third column vector represents the direction of the normal to the calibration target plane in the camera coordinate system. The entire matrix is ​​orthogonalized to obtain the final rotation matrix, which describes the attitude relationship between the camera coordinate system and the stage coordinate system.

[0027] The third column of the final rotation matrix is ​​the unit direction vector of the target normal in the camera coordinate system, while the camera's own optical axis direction vector remains constant in the camera coordinate system. The relative deviation between the target normal direction vector and the camera optical axis direction vector reflects the deflection between the camera's mounting attitude and its ideal vertical state. The pitch angle deviation is calculated by measuring the tilt of the camera optical axis relative to the normal in the XZ plane. The formula is: ,in, This is the pitch angle deviation value, which characterizes the amount of deflection of the optical axis in the forward and backward directions. The elements in the 1st row and 3rd column of the final rotation matrix. The formula for calculating the yaw angle deviation is: (The elements in the 3rd row and 3rd column of the final rotation matrix are used.) ,in, The yaw angle deviation value represents the amount of deflection of the optical axis in the left and right directions. For the elements in the 2nd row and 3rd column of the final rotation matrix, the specular reflection intensity and shadow distortion are corrected based on the aforementioned pitch and yaw deviation values. Specifically, for each preset lighting condition, the incident angle includes the tilt angle and rotation angle. Since the camera optical axis has tilt in the pitch direction and tilt in the yaw direction during installation, the incident angle is corrected as follows: ,in,( , ( ) is the corrected angle of incidence, i.e., the actual angle of incidence. , ( ) represents the original angle of incidence. This is the pitch angle deviation value. For the yaw angle deviation value, the specular reflection intensity is remapped in the Fresnel reflection model data using the corrected incident angle to eliminate the numerical deviation caused by the position shift of the highlight area due to the optical axis eccentricity; for the shadow distortion degree, the projection occlusion relationship of the light on the raised or recessed areas of the model surface is recalculated according to the corrected incident angle to correct the shadow length and grayscale distribution errors caused by the offset of the lighting direction.

[0028] S103, based on the corrected specular reflection intensity and shadow distortion degree, extract the corrected corner point image coordinates, and use the corrected corner point image coordinates as pixel coordinates to perform a three-level transformation to obtain the physical coordinates of the defect in the robot arm coordinate system; wherein, the three-level transformation includes a first-level transformation, a second-level transformation and a third-level transformation. Furthermore, based on the corrected specular reflection intensity and shadow distortion degree, the image quality under each preset illumination condition is re-evaluated. Specifically, using the corrected specular reflection intensity and shadow distortion degree as input, the comprehensive imaging quality score is recalculated. The illumination angle and spectral combination under the current optical axis eccentricity are selected. Based on the illumination angle and spectral combination, the imaging unit is controlled to acquire high-resolution detection images. A defect recognition algorithm is used on the acquired high-resolution detection images to extract the pixel coordinates of the defect area. These pixel coordinates are the position of the defect on the image plane after optical correction. These pixel coordinates are used as input for a three-level coordinate transformation, and the following transformation chain is executed: The first-level transformation uses the center point of the calibration target as the origin of the physical coordinate system of the stage, the row direction of the calibration target checkerboard as the X-axis, the column direction as the Y-axis, and the direction perpendicular to the target surface as the Z-axis. The homography matrix is ​​used to map the defect pixel coordinates to planar coordinates in the physical coordinate system of the stage. The mapping relationship is as follows: ,in, This represents the homogeneous coordinates of the defect point in the physical coordinate system of the stage. It is a homography matrix, a 3×3 square matrix. The above formula represents the homogeneous pixel coordinates of the defect point in the image coordinate system. It describes projecting the pixel coordinates of the defect point onto the physical plane of the stage using a homography matrix to obtain its two-dimensional position in the stage coordinate system. After the first-level transformation, a second-level transformation is performed. Using a pre-calibrated rigid transformation matrix, the physical coordinates of the stage are mapped to the robot arm base coordinate system. Specifically: ,in, This represents the homogeneous coordinates of the defect point in the robot arm base coordinate system, where , , These represent the x-coordinate, y-coordinate, and height coordinates of the defect point in the three-dimensional space of the robotic arm base, respectively, with 1 representing the extended dimension of the homogeneous coordinates. The rigid transformation matrix representing the transformation from the physical coordinate system of the platform to the coordinate system of the robot arm base is a 4×4 homogeneous matrix. This represents the three-dimensional homogeneous coordinates of the defect point in the physical coordinate system of the stage. , The output from the first-level transformation, as described above, maps the planar coordinates of the defect point in the physical coordinate system of the stage to the coordinate system of the robot arm base through a rigid transformation matrix, thus obtaining its absolute physical position in the three-dimensional space of the robot arm. The third-level transformation is then solidified, embedding the homography matrix and the rigid transformation matrix into the system firmware. After each defect identification is completed, the above two-level transformation is automatically invoked to sequentially convert the defect pixel coordinates into the physical coordinates of the stage, and then into the three-dimensional physical coordinates under the coordinate system of the robot arm base, as the final position output of the defect in physical space.

[0029] S104: The defect is identified by the defect recognition algorithm and the defect pixel size is extracted. The extracted defect pixel size is corrected by the correction mapping table. The corrected defect size is compared with the set threshold to determine whether the prototype model is qualified, and then the detection result is output. Specifically, the defect recognition algorithm performs target detection and segmentation on the detected image. An edge detection operator extracts the contours of all suspected defect areas in the image. Morphological analysis filters out non-defect noise, and each area is classified into specific defect types such as scratches, pits, bubbles, inclusions, or stress cracks according to a pre-defined defect type classifier. Simultaneously, the set of pixel coordinates and corresponding pixel-level geometric parameters covered by each defect area are recorded. For scratch defects, the pixel length is extracted along the defect's principal axis; for bubble or pit defects, the pixel diameter is extracted through circle fitting or equivalent diameter calculation; for planar defects, the pixel area is calculated. All pixel dimensions are uncorrected raw measurements. After obtaining the set of pixel coordinates and pixel dimensions of the defect area, for each pixel within the defect area, the local scale conversion factor corresponding to that location is looked up from the correction mapping table, and physical size conversion is performed pixel-by-pixel. For line-size defects, all pixels are incremented along the defect's principal axis. ,in, The pixel step size along the main axis. The physical length of the scratch. The scale conversion factor is used for the physical radius measurement. For surface size defects, the centroid of the defect area is identified, and the pixel radius is measured from the centroid in multiple directions. The physical radius in each direction is obtained by accumulating the pixel scale conversion factor along the path in that direction point by point. The average value of all directions is taken as the equivalent physical diameter. After correction, the physical size of each defect is compared with a preset acceptance threshold. The acceptance threshold is preset according to industry standards and stored in the system parameter library. If the physical size of a defect does not exceed the corresponding threshold, the defect is marked as an acceptable defect, and other areas are inspected. If the physical size of any defect exceeds the corresponding threshold, the entire prototype model is judged as a defective product, and the type, physical size, and physical coordinates of the defect are recorded.

[0030] After completing the size comparison of the defects, the detection results of the prototype model are output. The detection results include whether the prototype model is qualified. If it is not qualified, the detection results also include the physical coordinates of the defects.

[0031] The technical solutions in the above embodiments of this application have at least the following technical effects or advantages: by generating a correction mapping table, pixel-by-pixel spatial domain interpolation correction is performed in the defect size quantization stage, effectively eliminating the influence of lens radial distortion and tangential distortion on size measurement, realizing the accurate conversion of defect position from pixel coordinates to the physical coordinates that can be executed by the robotic arm, avoiding the cumulative error in the coordinate conversion process, and reducing the grasping coordinate offset rate of the sorting system.

[0032] Example 2: Following Example 1's establishment of a rigid coordinate transformation link from pixel to robotic arm and output of preliminary physical coordinates, this example introduces joint error calibration to eliminate the cumulative propagation effect of independent errors in the homography matrix and rigid transformation matrix, outputting compensated and accurate physical coordinates, such as... Figure 2 As shown.

[0033] S201, set probe positioning features on grid calibration target to form error calibration target, use robotic arm controller to obtain actual coordinates based on error calibration target; use the difference between the obtained actual coordinates and the physical coordinates obtained after three-level transformation as comprehensive output error, and construct deviation vector field model based on comprehensive output error; Specifically, on the grid pattern layer of the calibration target, micro-convex spherical positioning marks are set at the physical coordinate positions of the checkerboard corner points to form an error calibration target, i.e., a probe positioning feature is set. The positioning marks are made of hard alloy material and embedded in the surface of the calibration target. In optical imaging mode, the apex of the spherical cap serves as a spatial position marker for the grid corner points, presenting a high-contrast dot feature in the image. Sub-pixel-level positioning can be achieved through ellipse fitting. In mechanical touch mode, the trigger probe installed at the end of the robotic arm can stably contact the apex of the spherical cap, generating a reliable trigger signal to achieve accurate recording of physical coordinates. After obtaining the homography matrix and rigid transformation matrix, a set of calibration points, independent of the calibration process, is selected from all the positioning marks of the error calibration target. There are 25 calibration points, distributed in a 5×5 array, covering the entire imaging field of view. The robotic arm is controlled to carry a trigger probe at the end of the arm, which moves sequentially to the physical position of each calibration point and touches the apex of its spherical cap. The moment the probe touches the apex of the spherical cap, a trigger signal is generated. The robotic arm controller immediately locks the absolute encoder readings of all joints and calculates the three-dimensional coordinates of the probe sphere center in the coordinate system of the robotic arm base in real time based on the built-in positive kinematics model. This is the measured value.

[0034] Based on the homography matrix and rigid transformation matrix, the pixel coordinates of the 25 verification points are sequentially transformed using a three-level transformation to obtain the physical coordinates (i.e., predicted values) in the robotic arm coordinate system. The deviation vector between the predicted value and the measured value at each verification point is calculated using the following formula: ,in, , , Let be the components of the comprehensive output error of the j-th verification point in the three coordinate axes of the robot arm base coordinate system. , , The predicted coordinates of the j-th verification point are obtained through a three-level transformation using the homography matrix and the rigid transformation matrix. , , Let be the measured coordinates of the j-th verification point. The calculated deviation vector is the comprehensive output error. The deviation vectors calculated at the 25 verification points are associated with their spatial positions in the verification point array to construct a two-dimensional spatially distributed deviation vector field model. Specifically, the entire field of view is divided into 4×4=16 quadrilateral grid cells according to the row and column distribution of the verification points. Each grid cell is surrounded by four vertices. For a defect position falling inside any grid cell, the grid cell to which it belongs is located, and the deviation vector of the four vertices of the grid is obtained. Bilinear interpolation is performed based on the relative position of the defect position in the grid. Linear interpolation is performed in the X and Y directions respectively to obtain the continuous deviation estimate at that position. This deviation vector field model covers any pixel position within the entire imaging field of view.

[0035] S202, embeds the deviation vector field model into the third-level transformation, performs reverse compensation for the errors generated in the third-level transformation, and obtains the compensated physical coordinates; Furthermore, the deviation vector values ​​of the 25 verification points are written into the system's non-volatile memory as firmware parameters. Simultaneously written auxiliary parameters include the number of rows and columns of the verification points, the X and Y coordinate ranges of the verification points in the physical coordinate system of the stage, and the boundary processing rules for bilinear interpolation. The deviation vector field model is embedded into a three-level transformation. Specifically, during actual defect detection, the pixel coordinates are transformed using the first-level transformation to obtain the initial physical coordinates of the stage. Based on the initial physical coordinates, bilinear interpolation is performed in the deviation vector field model. All grid cells are traversed, and it is checked whether the initial physical coordinates of the stage simultaneously satisfy the following conditions: the initial physical coordinates of the stage are greater than or equal to the minimum physical coordinate value and less than or equal to the maximum physical coordinate value in the X-axis direction, and the initial physical coordinates of the stage are greater than or equal to the minimum physical coordinate value and less than or equal to the maximum physical coordinate value in the Y-axis direction. If both conditions are met, the stage is determined to belong to that grid cell. After determining the grid cell, the pre-stored deviation vectors of the four vertices (i.e., the four verification points) of that grid are read, and the normalized weight coefficients in the X and Y directions are calculated. ,in, The weighting coefficients are in the X direction. The weighting coefficients are in the Y direction. The physical coordinates of the check point at the bottom left corner of the current grid cell are ( , ) represents the physical coordinates of the upper right corner verification point. , Given the initial physical coordinates of the stage, based on weighting coefficients, bilinear interpolation is performed on the deviation vectors of the four vertices in the X, Y, and Z directions to obtain the continuous deviation estimates at that coordinate position.

[0036] ; in, This is the X-direction deviation compensation value at the location to be interpolated. This is the Y-direction deviation compensation value at the position to be interpolated. This is the Z-direction deviation compensation value at the position to be interpolated. For the weighting coefficients in the X direction, These are the weighting coefficients in the Y direction. , , This represents the deviation vector of the bottom left vertex of the grid in the X, Y, and Z directions. , , This is the deviation vector of the bottom right vertex of the grid in the X, Y, and Z directions; , , This is the deviation vector of the top-left vertex of the grid in the X, Y, and Z directions; , , The deviation vectors of the top right corner vertex of the grid in the X, Y, and Z directions are obtained as follows: , , Let be the deviation compensation vector. Based on the obtained deviation compensation vector, reverse compensation is applied to the physical coordinates of the initial stage. The expression is: ,in,( , () represents the compensated physical coordinates of the stage. , Let be the initial physical coordinates of the platform. Using the compensated physical coordinates of the platform as input, a rigid transformation matrix is ​​used to map them to the robot arm base coordinate system, resulting in preliminary robot arm base coordinates. These preliminary robot arm base coordinates are then compensated and corrected in the Z direction. ,in, This represents the final X-axis coordinate of the defect point in the robot arm base coordinate system after compensation. This represents the final Y-axis coordinate of the defect point in the robot arm base coordinate system after compensation. This represents the final Z-axis coordinate of the defect point in the robot arm base coordinate system after compensation. The initial X-direction coordinates of the defect point are obtained after rigid transformation matrix mapping in the coordinate system of the robot arm base. The initial Y-axis coordinates of the defect point are obtained after rigid transformation matrix mapping in the coordinate system of the robot arm base. The initial Z-direction coordinates of the defect point are obtained after rigid transformation matrix mapping in the coordinate system of the robot arm base. The Z-direction deviation compensation value at the position to be interpolated is ( , , () represents the coordinates of the robot arm base after compensation.

[0037] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: by performing reverse compensation on the deviation vector field, the overall system error of the defect physical coordinate output is reduced.

[0038] Example 3: Example 2 achieved high-precision localization of externally visible defects, but did not analyze the total internal reflection blocking area inside the cavity. This example utilizes the evanescent wave resonance tunneling effect to penetrate the cavity barrier, converting invisible defects into detectable signals, such as... Figure 3 As shown.

[0039] S301 extracts cavity data from the design drawings of the prototype model, determines the cavity area, scans the critical angle of total internal reflection in the cavity area, and the imaging unit acquires images irradiated at different incident angles; compares the incident angle and the critical angle of total internal reflection of each frame of image to identify the optimal incident angle; Specifically, the design drawings (CAD model file) of the prototype model to be tested are read, and the geometric information of all internal cavity structures is extracted. The extracted cavity data includes: the three-dimensional spatial coordinates of each cavity in the model design coordinate system, the geometric shape type of the cavity, the characteristic dimensions of the cavity, and the thickness parameters of the cavity wall. All the extracted cavity data are correlated and integrated according to spatial location to generate a cavity distribution map. The design coordinates in the cavity distribution map are spatially registered with the field of view of the imaging unit. By applying the rigid transformation relationship between the model design coordinate system and the physical coordinate system of the stage to the coordinates of each cavity in the cavity distribution map, the image coordinate range of each cavity region in the field of view of the imaging unit (i.e., the pixel region corresponding to each cavity) is calculated. For each cavity, a monitoring window is marked in its corresponding image region, and the monitoring window covers the cavity wall. After registration, the critical angle of total internal reflection can be scanned individually for each internal cavity in the key areas of the surface. The prototype model to be inspected is placed on the stage, which is located at the center of the adjustable light source array. The adjustable light source array adopts a hemispherical structure, with multiple LED array modules distributed in a hemispherical shape and installed on the inner wall of the hemispherical dome. The light emission direction of each LED array module can be independently adjusted within the range of 0° to 90°. The LED array module corresponding to the current spatial position of the cavity to be scanned is selected, and the incident angle of the illumination beam is continuously changed in a step of 0.5° within the range of 40° to 85°. The spectrum is fixed as green light. At each incident angle, the imaging unit acquires a low-resolution image. After each frame of image acquisition is completed, the current incident angle and the corresponding image data are recorded and stored in order of incident angle from small to large, forming the incident angle-image sequence of the cavity.

[0040] For each captured image frame, based on the monitoring window position, the grayscale values ​​of all pixels within that window are extracted, and their arithmetic mean is calculated as the average brightness value of the cavity wall region at that incident angle. The average brightness value reflects the light energy intensity coupled into the cavity at the current incident angle. When the incident angle is less than the critical angle for total internal reflection, light is transmitted normally into the cavity, and the brightness of the wall region is relatively high. When the incident angle exceeds the critical angle for total internal reflection, the transmitted light disappears, and only evanescent waves couple into the cavity, causing the brightness of the wall region to drop sharply. The obtained incident angle-wall brightness is plotted as a curve from small to large incident angle, and the inflection point of the brightness curve is identified. As the incident angle gradually increases, the wall brightness drops sharply from a high-value stable region. The angle corresponding to the inflection point in the low-value region is the critical angle of total internal reflection at the cavity interface. When the incident angle is greater than the critical angle of total internal reflection, it is necessary to check whether there is a local peak in the scan brightness curve. Since the evanescent wave will form a resonance enhancement inside the cavity at certain specific angles, the wall brightness will locally rebound. The incident angle that makes the brightness reach the local maximum value is taken as the optimal incident angle of the evanescent wave of the cavity. The critical angle of total internal reflection and the optimal incident angle are stored in the database in the form of key-value pairs to establish the critical angle-evanescent wave coupling efficiency mapping relationship. If no obvious local peak is found above the critical angle, the angle with the highest relative brightness in the range of 1°~2° above the critical angle is taken as the replacement value of the optimal incident angle.

[0041] S302 calculates the penetration depth of the evanescent wave based on the optimal incident angle, and determines whether the light can penetrate the multi-layer cavity based on the penetration depth. If it can penetrate the multi-layer cavity, it illuminates different cavity areas through the optimal incident angle and collects detection images. Furthermore, the penetration depth of the evanescent wave is calculated based on the obtained optimal incident angle, using the following formula: ,in, For the penetration depth of the evanescent wave, The wavelength of illumination light in a vacuum. For the optimal angle of incidence, The refractive index of air, Given the refractive index of the prototype model material, the calculated penetration depth is compared with the thickness of the thin layer near the cavity wall. When the evanescent wave energy reaches the other side wall of the cavity before it completely decays, the light energy can be coupled into the multi-layer cavity through the thin-walled interface step by step; conversely, when... If the evanescent wave has attenuated to an undetectable level before reaching the next wall, the wavelength is switched to another wavelength or the incident angle is finely adjusted to recalculate the penetration depth until the penetration condition is met. If it is determined that the evanescent wave can penetrate multiple cavities, high-resolution detection image acquisition is performed: For a model containing multiple cavities, multiple LED array modules are controlled to illuminate different cavity areas with their respective optimal incident angles to achieve synchronous evanescent wave illumination of all cavities. The exposure time of the imaging unit is extended to 5 times that of conventional acquisition and the analog gain is increased by 6dB to ensure that the brightness of the acquired image falls within the optimal response range of the sensor. Five frames of images are continuously acquired under the same illumination conditions, and pixel-by-pixel accumulation and averaging are performed at the FPGA hardware level to suppress random noise. Finally, a high-resolution detection image is output. In this image, the cavity internal defects that were originally completely invisible due to total internal reflection are now identifiable defect image features after evanescent wave resonant tunneling illumination and scattering conversion.

[0042] The technical solutions in the above embodiments of this application have at least the following technical effects or advantages: by utilizing the evanescent wave resonance tunneling effect, light energy can bypass the total internal reflection interface and enter the cavity that was originally inaccessible by optical means, making the defects inside the cavity from completely invisible to detectable imaging, thus fundamentally solving the optical occlusion problem of the cavity defects inside the transparent prototype model.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting prototype models based on image recognition, characterized in that, include: S101, a grid calibration target is fixed on the reference surface of the stage. During the detection image acquisition cycle, the calibration image of the grid calibration target is acquired synchronously. The checkerboard pattern and corner image coordinates of the calibration target are identified based on the calibration image. The scale conversion factor is calculated based on the spacing between adjacent corner image coordinates and the spacing between corner physical coordinates, and a correction mapping table is generated. S102, based on the physical coordinates and image coordinates of the corner points, obtain the angle deviation between the optical axis direction vector and the calibration target normal direction vector, and correct the specular reflection intensity and shadow distortion degree according to the angle deviation; S103, based on the corrected specular reflection intensity and shadow distortion degree, extract the corrected corner point image coordinates, and use the corrected corner point image coordinates as pixel coordinates to perform a three-level transformation to obtain the physical coordinates of the defect in the robot arm coordinate system; wherein, the three-level transformation includes a first-level transformation, a second-level transformation and a third-level transformation. S104 identifies defects and extracts defect pixel dimensions using a defect recognition algorithm. The extracted defect pixel dimensions are then corrected using a correction mapping table. The corrected defect dimensions are compared with a set threshold to determine whether the prototype model is qualified, and the detection result is output.

2. The method for detecting a prototype model based on image recognition according to claim 1, characterized in that, The process of generating the calibration mapping table includes: fixing a grid calibration target within a reserved area at the edge of the stage reference plane; defining the geometric center of the calibration target as the origin of the stage's physical coordinate system; defining the row and column directions of the checkerboard as the positive directions of the X and Y axes, respectively; acquiring calibration images; performing corner detection on the calibration images; extracting the sub-pixel-level image coordinates of the checkerboard corners; simultaneously performing ellipse fitting on the circular markers to extract the center coordinates as supplementary control points; calculating the pixel distance between adjacent corners based on the extracted corner image coordinates; combining this with the known physical distance between adjacent corners on the calibration target; and associating all the calculated scale conversion factors with their corresponding image coordinate positions to construct the calibration mapping table.

3. The method for detecting a prototype model based on image recognition according to claim 2, characterized in that, Acquiring calibration images includes simultaneous acquisition within the same frame and acquisition with stage translation compensation. Simultaneous acquisition within the same frame includes: the calibration target is located in the reserved area at the edge of the camera's field of view, and it is simultaneously imaged on different positions on the sensor target surface along with the hand model to be detected. After the detection image acquisition is completed, the imaging unit is triggered to acquire the next frame image, and the lighting conditions and camera exposure parameters are kept consistent with the previous frame detection image.

4. The hand-made model detection method based on image recognition according to claim 3, characterized in that, The stage translation compensation acquisition includes: when the calibration target is blocked by the hand model to be tested, a movement command is sent to the displacement platform below the stage to move a preset distance along the negative X-axis. After the translation is completed, the imaging unit acquires the calibration image with the same exposure parameters as the detection image. After the acquisition is completed, the stage is reset.

5. The method for detecting a prototype model based on image recognition according to claim 1, characterized in that, The included angle deviation includes pitch angle deviation and yaw angle deviation. The formula for calculating the pitch angle deviation is: ,in, This is the pitch angle deviation value. The elements in the 1st row and 3rd column of the final rotation matrix. The formula for calculating the yaw angle deviation is: (The elements in the 3rd row and 3rd column of the final rotation matrix are used.) ,in, This is the yaw angle deviation value. These are the elements in the 2nd row and 3rd column of the final rotation matrix.

6. The method for detecting a prototype model based on image recognition according to claim 1, characterized in that, The corrected corner point image coordinates are used as pixel coordinates to perform a three-level transformation to obtain the physical coordinates of the defect in the robot arm coordinate system. The three-level transformations are as follows: the first level transformation maps the defect pixel coordinates to planar coordinates in the physical coordinate system of the stage through the homography matrix; the second level transformation maps the physical coordinates of the stage to the coordinate system of the robot arm base through the rigid transformation matrix; the third level transformation embeds the homography matrix and the rigid transformation matrix into the system firmware. After each defect identification is completed, the above two levels of transformation are called to output the three-dimensional physical coordinates of the defect in the coordinate system of the robot arm base in sequence.

7. The method for detecting a prototype model based on image recognition according to claim 1, characterized in that, Prototype model detection methods also include: S201, set probe positioning features on grid calibration target to form error calibration target, use robotic arm controller to obtain actual coordinates based on error calibration target; use the difference between the obtained actual coordinates and the physical coordinates obtained after three-level transformation as comprehensive output error, and construct deviation vector field model based on comprehensive output error; S202 embeds the deviation vector field model into the three-level transformation, performs reverse compensation for the errors generated in the three-level transformation, and obtains the compensated physical coordinates.

8. The method for detecting a prototype model based on image recognition according to claim 7, characterized in that, The prototype model detection method also includes: S301 extracts cavity data from the design drawings of the prototype model, determines the cavity area, scans the critical angle of total internal reflection in the cavity area, and the imaging unit acquires images irradiated at different incident angles; compares the incident angle and the critical angle of total internal reflection of each frame of image to identify the optimal incident angle; S302 calculates the penetration depth of the evanescent wave based on the optimal incident angle, and determines whether the light can penetrate the multi-layer cavity based on the penetration depth. If it can penetrate the multi-layer cavity, it illuminates different cavity areas through the optimal incident angle and acquires detection images.

9. The hand-made model detection method based on image recognition according to claim 8, characterized in that, Identifying the optimal incident angle includes: acquiring images by changing the incident angle in step size and extracting the wall brightness, and drawing a brightness curve; the inflection point of the curve is the critical angle for total internal reflection, and the local peak brightness above the critical angle is the optimal incident angle.

10. The method for detecting a prototype model based on image recognition according to claim 9, characterized in that, The formula for calculating the penetration depth of an evanescent wave based on the optimal incident angle is: ,in, For the penetration depth of the evanescent wave, The wavelength of illumination light in a vacuum. For the optimal angle of incidence, The refractive index of air, is the refractive index of the prototype model material.

Images