A method, device and medium for identifying defects of optical lenses based on machine vision

Abstract

The application discloses a kind of optical lens defect identification method, equipment and medium based on machine vision, it is related to machine vision technical field, including, the image pose of each curvature feature map in curvature feature map group is calibrated according to the same surface coordinate curvature feature, form cross-view pixel index table;Curvature feature map group is carried out contrastive segmentation, and the luminance abnormal area obtained by segmentation is used as defect candidate area, the candidate area surface coordinates corresponding to each defect candidate area are checked, and defect development track chain is obtained;Along defect development track chain, respectively, check the development change of forward machine vision image and reverse machine vision image, compare forward development mode string and reverse development mode string by dynamic time warping mechanism, and the development difference position obtained by comparison is mapped as bidirectional development mode graph. Through dynamic time warping and difference mapping of forward development mode string and reverse development mode string, the characterization stability of complex curvature lens double-view defect development relationship is improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method, device and medium for identifying defects in optical lenses based on machine vision. Background Technology

[0002] With the development of precision optical manufacturing, digital surface measurement, and machine vision imaging technologies, optical lens defect detection has become an important research direction for the quality control of optical components. During grinding, polishing, coating, assembly, and service, lenses may develop defects such as scratches, pitting, bubbles, impurities, and uneven film layers. Different defects exhibit varying grayscale distributions, edge morphologies, texture structures, and spatial extension characteristics under transmission, reflection, and scattering imaging conditions. Current research mainly focuses on multi-angle illumination, forward and reverse imaging, curved surface geometric representation, image enhancement, region segmentation, visual feature description, and intelligent classification, and is gradually integrating surface measurement data with machine vision image information to meet the automated inspection needs of high-curvature lenses, aspherical lenses, and freeform surface lenses. With the continuous improvement of imaging sensor resolution and computational vision algorithms, optical lens defect identification is developing towards multi-view observation, refined characterization, and multi-source data correlation.

[0003] However, changes in the curvature of optical lenses, differences in viewing direction, and local projection stretching can cause the same defect to appear in different development positions, variation intervals, and order in forward and reverse images. This makes it difficult to directly correspond forward and reverse defect features according to a unified pixel position. Existing methods mostly rely on fixed-scale registration, local feature similarity matching, and single-image discrimination. When the development trajectory undergoes nonlinear stretching, compression, and local misordering, it is easy to decompose the bidirectional response of the same defect into irrelevant features. It is also easy to misjudge the viewing angle difference caused by the lens surface as a defect difference. It is difficult to form a stable representation that can simultaneously retain the forward and reverse development changes and correspondence, thus affecting the consistency of defect category discrimination and surface position localization. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a machine vision-based optical lens defect identification method to solve the problem of difficulty in forming a stable representation that can simultaneously retain the forward and reverse development changes and their corresponding relationships.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for identifying defects in optical lenses based on machine vision, comprising,

[0008] Acquire surface shape data, forward machine vision image, and reverse machine vision image. Project the surface coordinates in the surface shape data onto the pixel positions in the forward and reverse machine vision images. Adjust the pixel neighborhood scale of the pixel position corresponding to the surface coordinates through the local curvature in the surface shape data to generate a curvature feature map group. Calibrate the image pose of each curvature feature map in the curvature feature map group according to the same surface coordinates to form a cross-view pixel index table.

[0009] Contrast segmentation is performed on the curvature feature map group, and the segmented brightness abnormal areas are used as defect candidate areas. The cross-view pixel index table is used to check the surface coordinates of each defect candidate area, and the defect candidate areas with continuous surface coordinates are connected to obtain the defect development trajectory chain.

[0010] The development changes of the forward and reverse machine vision images are examined along the defect development trajectory chain. The position order of each development change is encoded to generate a forward development pattern string and a reverse development pattern string. The forward and reverse development pattern strings are compared through a dynamic time warping mechanism, and the development difference position obtained by comparison is mapped into a bidirectional development pattern diagram.

[0011] The defect classification model is used to identify the defect category of the bidirectional imaging pattern. The identified defect category is then combined with the corresponding candidate region surface coordinates to form the lens defect identification result.

[0012] As a preferred embodiment of the machine vision-based optical lens defect identification method of the present invention, wherein: the generation of the curvature feature map group specifically includes,

[0013] The surface coordinates in the surface data are projected onto the pixel positions of the forward machine vision image and the reverse machine vision image, respectively, and the two pixel positions corresponding to the same surface coordinates are combined into a dual-view pixel position pair.

[0014] The local curvature in the surface data is used to define the radius of the neighborhood of the dual-view pixel position relative to the surrounding neighborhood, thus forming the initial dual-view pixel neighborhood;

[0015] The horizontal and vertical coordinate spacing of adjacent surface coordinates within the initial dual-view pixel neighborhood is proportionally converted to the horizontal and vertical spacing of corresponding pixel positions in the forward and reverse machine vision images, respectively, to obtain the dual-view scaling ratio.

[0016] The horizontal scaling factor in the dual-view scaling ratio is applied to the left and right boundaries of the initial dual-view pixel neighborhood, and the vertical scaling factor in the dual-view scaling ratio is applied to the upper and lower boundaries of the initial dual-view pixel neighborhood, and the target dual-view pixel neighborhood is output.

[0017] The target pixel locations covered by the target dual-view pixel neighborhood are used as deformable convolution sampling locations. Deformable convolution is performed on the forward machine vision image and the reverse machine vision image, and the convolution results are arranged according to the surface coordinates to generate a curvature feature map group.

[0018] As a preferred embodiment of the machine vision-based optical lens defect identification method of the present invention, wherein: the formation of the cross-view pixel index table specifically involves,

[0019] The surface coordinates and corresponding pixel positions of each curvature feature map in the curvature feature map group are fed into the perspective multi-point pose algorithm. The rotation and translation relationships of each curvature feature map relative to the surface data are fitted by the projection relationship between the surface coordinates and the corresponding pixel positions to obtain the initial image pose of each curvature feature map.

[0020] Extract rotation and translation relationships from the initial image pose. Rotate the surface coordinates according to the rotation relationship and move the rotated surface coordinates according to the translation relationship. Project the moved surface coordinates onto each curvature feature map. Delete the back-projected pixel positions and corresponding surface coordinates that fall outside the neighborhood of the target dual-view pixel to form surface pixel items.

[0021] The rotation and translation relationships of each curvature feature map are refitted using the surface pixel term to generate the pose of the calibration image;

[0022] The rotation and translation relationships of each curvature feature map are updated by calibrating the image pose, thus completing the image pose calibration of each curvature feature map;

[0023] Using the same surface coordinates as the index key, the pixel positions in each calibrated curvature feature map are listed in the pixel position item corresponding to the index key, forming a cross-view pixel index table.

[0024] As a preferred embodiment of the optical lens defect identification method based on machine vision described in this invention, the contrast segmentation involves comparing the brightness differences of adjacent pixels in the curvature feature map, grouping consecutive pixels with consistent brightness changes into the same segmentation region, and defining the segmentation boundary at the brightness change inflection point to form a contrast segmentation map.

[0025] As a preferred embodiment of the machine vision-based optical lens defect identification method of the present invention, wherein: obtaining the defect imaging trajectory chain specifically involves,

[0026] By comparing the brightness of pixels inside the segmentation boundary with the adjacent pixels outside the segmentation boundary in the contrast segmentation image region by region, abnormal brightness regions where the brightness of the inner pixels is consistently higher than that of the outer adjacent pixels are screened out, and these abnormal brightness regions are used as candidate defect regions.

[0027] Using the pixel positions of the candidate regions covered by the defect candidate regions, the surface coordinates of the candidate regions corresponding to the same pixel positions are found in the cross-view pixel index table.

[0028] Based on the image acquisition order of each curvature feature map, the front and back order of each defect candidate region is set;

[0029] By connecting adjacent candidate defect regions with continuous surface coordinates, a defect development trajectory chain is obtained.

[0030] As a preferred embodiment of the machine vision-based optical lens defect identification method of the present invention, wherein: the generation of the forward development pattern string and the reverse development pattern string specifically refers to...

[0031] According to the order of the defect development trajectory chain, the positive and negative pixel positions corresponding to the surface coordinates of the candidate region are assigned the same trajectory number to form the positive trajectory pixel sequence and the negative trajectory pixel sequence.

[0032] Along the forward trajectory pixel sequence and the reverse trajectory pixel sequence, the brightness of the pixels marked with adjacent same position trajectory numbers is compared, and the pixel position where the brightness change direction is reversed is extracted to form the forward development change position and the reverse development change position.

[0033] Based on the ascending order of the corresponding trajectory numbers, the forward development change positions and the reverse development change positions are numbered consecutively. The forward numbers are then connected sequentially to form a forward development pattern string, and the reverse numbers are connected sequentially to form a reverse development pattern string.

[0034] In a preferred embodiment of the machine vision-based optical lens defect identification method of the present invention, the step of mapping the development difference positions obtained by comparison into a bidirectional development pattern map specifically involves:

[0035] The forward and reverse numbers are arranged horizontally and vertically according to the same trajectory number, respectively. The differences between each forward number and each reverse number are compared one by one to form a developing number difference array.

[0036] Taking the upper left end of the development number difference array as the starting point for regularization, the previous receiving position is connected to the adjacent array position along the diagonal, horizontal and vertical directions of the development number difference array, respectively. The array value of the adjacent array position is continued to the cumulative array value retained by the previous receiving position. The previous receiving direction with the smallest cumulative array value is retained at each adjacent array position, and the previous receiving directions are connected in reverse from the end of the development number difference array to form the development mode regularization path.

[0037] The horizontal and vertical receiving positions, as well as the diagonal receiving positions with different forward and reverse numbers, are extracted along the regular path of the developing pattern as the developing difference positions;

[0038] According to the row and column numbers of the development difference position in the development number difference array, the forward number identified by the row number and the reverse number identified by the column number are arranged on the forward and reverse sides respectively, and the forward and reverse numbers under the same development difference position are connected to form a bidirectional development pattern diagram.

[0039] As a preferred embodiment of the machine vision-based optical lens defect recognition method of the present invention, wherein: the step of merging the determined defect category with the corresponding surface coordinates to obtain the lens defect recognition result specifically involves,

[0040] The two-way development pattern diagram is fed into the defect discrimination model. The defect discrimination model judges the defect category of the two-way development pattern diagram based on the numbering arrangement, connection relationship and development characteristics associated with each number in the two-way development pattern diagram, and outputs the defect category.

[0041] Based on the same trajectory number of the development difference position in the bidirectional development pattern diagram, the surface coordinates of candidate regions with the same preceding and following order in the defect development trajectory chain are located, and the defect category and the surface coordinates of the candidate regions are merged into the lens defect identification result.

[0042] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the machine vision-based optical lens defect identification method described in the first aspect of the present invention.

[0043] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the machine vision-based optical lens defect identification method described in the first aspect of the present invention.

[0044] The beneficial effects of this invention are as follows: By dynamically time-regulating and differentiating the forward and reverse development pattern strings, the stability of the characterization of the development relationship of dual-view defects in complex curvature lenses is improved. The co-position trajectory sequence number establishes a unified order benchmark for the forward and reverse development change positions. The development number difference array carries the nonlinear spacing and order offset between bidirectional development changes. The development pattern regularization path retains the lateral, longitudinal, and diagonally connected positions with different numbers, transforming local stretching, compression, and misalignment into development difference positions. The bidirectional development pattern diagram presents the distribution order and associated form of development differences through numbering and connection relationships, providing a traceable basis for the defect discrimination model, and improving the consistency and accuracy of defect category discrimination and candidate region surface coordinate positioning. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a machine vision-based optical lens defect identification method.

[0047] Figure 2 A flowchart for generating contrast segmentation and defect development trajectory chains.

[0048] Figure 3 This is a flowchart for comparing forward and reverse development mode sequences and generating a bidirectional development mode diagram.

[0049] Figure 4 A flowchart for generating defect category discrimination and lens defect identification results. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0053] Reference Figures 1-4 This is one embodiment of the present invention, which provides a machine vision-based method for identifying defects in optical lenses, comprising the following steps:

[0054] S1. Acquire surface shape data, forward machine vision image, and reverse machine vision image. Project the surface coordinates in the surface shape data onto the pixel positions of the forward and reverse machine vision images. Adjust the pixel neighborhood scale of the pixel position corresponding to the surface coordinates through the local curvature in the surface shape data to generate a curvature feature map group. Calibrate the image pose of each curvature feature map in the curvature feature map group according to the same surface coordinates to form a cross-view pixel index table.

[0055] S1.1 Fix the optical lens to be identified at the same detection position, and scan the surface of the optical lens point by point using a non-contact optical profilometer. Record the lateral coordinates, longitudinal coordinates and surface height of each scan position in the surface shape measurement coordinate system. Combine the lateral coordinates, longitudinal coordinates and surface height to form a surface coordinate system, and assign surface coordinate numbers according to the scanning order to form surface shape data. Place the forward machine vision camera on the forward imaging side of the optical lens to be identified, and acquire forward machine vision images while keeping the illumination direction, imaging distance and focal length unchanged. Place the reverse machine vision camera on the reverse imaging side of the optical lens to be identified, and acquire reverse machine vision images at the same imaging scale.

[0056] Both forward and reverse machine vision cameras obtain camera intrinsic parameters, lens distortion parameters, and rotation and translation parameters relative to the surface shape measurement coordinate system through a camera calibration plate. The forward machine vision image establishes a projection relationship between pixel positions and surface coordinates based on the camera intrinsic parameters, lens distortion parameters, rotation parameters, and translation parameters of the forward machine vision camera. The reverse machine vision image, based on the rotation parameters of the reverse machine vision camera, transforms the horizontal and vertical pixel axes to be aligned with the surface shape measurement coordinate system, and then establishes a projection relationship between pixel positions and surface coordinates based on the camera intrinsic parameters, lens distortion parameters, rotation parameters, and translation parameters of the reverse machine vision camera. When using transmission imaging, the thickness, refractive index, and surface shape data of the optical lens to be identified are read. The refraction positions of the light rays passing through the front and rear surfaces of the optical lens to be identified are calculated along the imaging ray from the corresponding machine vision camera, and the calculated surface intersections are mapped to the surface shape measurement coordinate system. Using the surface coordinate numbers in the surface shape data as a common identifier, a correspondence is established between the forward and reverse pixel positions corresponding to the same surface coordinate number.

[0057] Using the fixed position and contour boundary of the optical lens to be identified as a common position reference, the coverage area of ​​the surface data is verified to match the lens imaging area in the forward machine vision image and the reverse machine vision image. When the coverage area is consistent, the corresponding acquisition result is retained. When the contour boundary of the surface data is projected onto the forward machine vision image and the reverse machine vision image, it is located within the corresponding lens imaging area. And when the maximum positional deviation between the projected contour boundary and the contour boundary of the corresponding lens imaging area does not exceed the upper limit of the reprojection error formed by the camera calibration, the coverage area is determined to be consistent and the corresponding acquisition result is retained. When the coverage area is inconsistent, the image is out of focus, the image is overexposed, the image is underexposed, or the contour is missing, the corresponding acquisition is performed again to form the surface data, forward machine vision image and reverse machine vision image corresponding to the same optical lens to be identified.

[0058] S1.2 Project the surface coordinates in the surface data onto the pixel positions of the forward machine vision image and the reverse machine vision image, respectively. Form a dual-view pixel position pair by pairing two pixel positions corresponding to the same surface coordinates. Specifically,

[0059] The surface coordinates, composed of horizontal, vertical, and surface height, in the surface shape data are combined with the intrinsic parameters of the forward and reverse machine vision cameras acquired by the camera calibration board, as well as the rotation and translation parameters of the two machine vision cameras relative to the surface shape measurement coordinate system. The surface coordinates are then transformed into the forward and reverse machine vision camera coordinate systems. Next, they are converted into horizontal and vertical pixel coordinates in the forward and reverse machine vision images according to focal length, principal point position, and pixel scale. When the pixel projection depth is positive and both the horizontal and vertical pixel coordinates are within the corresponding image boundaries, the pixel position corresponding to the surface coordinates is retained. When the pixel projection depth is not positive, or the horizontal or vertical pixel coordinates are out of bounds, a projection anomaly mark is added to the corresponding surface coordinates, and the corresponding pixel position is stopped from participating in pairing. Using the surface coordinate number as the corresponding identifier, the forward and reverse pixel positions existing under the same surface coordinate number are written into the same pairing record. When a pixel position on one side is missing, a single-side projection missing mark is retained, forming a dual-view pixel position pair.

[0060] S1.3. The local curvature in the surface data is used to define the radius of the neighborhood of the dual-view pixel position relative to its surrounding area, forming an initial dual-view pixel neighborhood. Specifically,

[0061] The local curvature is read from the corresponding surface coordinates of the dual-view pixel positions, and the minimum and maximum local curvature values ​​corresponding to all valid surface coordinates of the surface data are read. Based on the pixel scale of the forward and reverse machine vision images and the size of the minimum defect to be identified, the minimum neighborhood radius that can cover the size of the minimum defect to be identified and the maximum neighborhood radius that can maintain the local surface range are determined. The minimum local curvature is mapped to the maximum neighborhood radius, and the maximum local curvature is mapped to the minimum neighborhood radius. According to the position ratio of the current local curvature between the minimum and maximum local curvature values, it is mapped inversely to the area between the minimum and maximum neighborhood radii to obtain the horizontal and vertical neighborhood radii around the forward and reverse pixel positions. The local curvature of adjacent surface coordinates... If the curvature difference does not exceed the upper limit of curvature fluctuation formed by repeated measurements of the same scanning position by a non-contact optical profilometer, and there are adjacent surface coordinates with consecutive scan numbers in both the horizontal and vertical scanning directions, the corresponding neighborhood radius is expanded outward from the positive and negative pixel positions as the center. If there is a lack of local curvature, the local curvature difference between adjacent surface coordinates exceeds the upper limit of curvature fluctuation, or there is a lack of adjacent surface coordinates with consecutive scan numbers in either the horizontal or vertical scanning direction, the dual-view pixel position pairs stop participating in neighborhood generation. If the neighborhood boundary exceeds either the positive or negative machine vision image boundary, the neighborhood coordinates located outside the corresponding image boundary are deleted, and the corresponding image boundary is used as the adjusted neighborhood boundary to form the initial dual-view pixel neighborhood.

[0062] S1.4. The horizontal and vertical coordinate spacing of adjacent surface coordinates within the initial dual-view pixel neighborhood is proportionally converted to the horizontal and vertical spacing of corresponding pixel positions in the forward and reverse machine vision images, respectively, to obtain the dual-view scaling ratio. Specifically,

[0063] The non-contact optical profilometer acquires surface shape data along the horizontal and vertical scanning directions. It groups horizontally adjacent surface coordinate pairs based on consecutive scan column numbers within the same scan row, and vertically adjacent surface coordinate pairs based on the same scan column number within adjacent scan rows. From the initial dual-view pixel neighborhood, it selects horizontally adjacent surface coordinate pairs and vertically adjacent surface coordinate pairs, reads the horizontal coordinate spacing of the horizontally adjacent surface coordinate pairs, and the vertical coordinate spacing of the vertically adjacent surface coordinate pairs. It also reads the horizontal and vertical spacing of corresponding pixel positions in the forward machine vision image and the reverse machine vision image, and converts them according to the following formula: [Formula omitted for brevity]

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] In the formula, This represents the horizontal aspect ratio of the forward machine vision image, expressed in pixels per surface coordinate unit. This represents the vertical aspect ratio of the forward machine vision image, expressed in pixels per surface coordinate unit. This represents the horizontal proportion of the reverse machine vision image, expressed in pixels per surface coordinate unit. This represents the vertical scale of the reverse machine vision image, expressed in pixels per surface coordinate unit. The horizontal spacing between adjacent pixels in a forward machine vision image, expressed in pixels. This represents the horizontal coordinate spacing between adjacent surfaces in the surface data, with units consistent with the surface coordinate units in the surface data. The vertical spacing between adjacent pixels in a forward machine vision image, expressed in pixels. This represents the vertical coordinate spacing between adjacent surfaces in the surface data, with units consistent with the surface coordinate units in the surface data. This represents the horizontal spacing between adjacent pixels in a reverse machine vision image, expressed in pixels. The vertical spacing between adjacent pixels in a reverse machine vision image, in pixels. For positive machine vision image corresponding labels, For reverse machine vision image corresponding labels, For horizontal corresponding labels, For vertical correspondence, This represents the distance between two adjacent positions.

[0069] When multiple effective proportions exist in the same direction, the median value of the effective proportions is taken. When the horizontal coordinate spacing is zero, the corresponding horizontal proportion stops participating in the aggregation. When the vertical coordinate spacing is zero, the corresponding vertical proportion stops participating in the aggregation. When the corresponding pixel position is missing, the spacing missing mark is retained. The horizontal and vertical proportions of the forward machine vision image and the horizontal and vertical proportions of the reverse machine vision image are combined to form the dual-view scaling ratio.

[0070] S1.5. Apply the horizontal scaling factor of the dual-view scaling ratio to the left and right boundaries of the initial dual-view pixel neighborhood, and apply the vertical scaling factor of the dual-view scaling ratio to the top and bottom boundaries of the initial dual-view pixel neighborhood, and output the target dual-view pixel neighborhood. Specifically,

[0071] Using the forward and reverse pixel positions in the dual-view pixel position pair as neighborhood centers, the surface coordinate spans corresponding to the left, right, top, and bottom boundaries of the initial dual-view pixel neighborhood are read. These surface coordinate spans are then converted into boundary pixel spans in the forward and reverse machine vision images according to the forward horizontal, forward vertical, and reverse horizontal and vertical scaling ratios in the dual-view scaling ratio. Using the forward pixel position as the neighborhood center, the boundary pixel spans converted according to the forward horizontal scaling ratio determine the left and right boundaries in the forward machine vision image, and the boundary pixel spans converted according to the forward vertical scaling ratio determine the boundary of the forward machine vision image. The upper and lower boundaries of the image are defined; the left and right boundaries of the reverse machine vision image are determined by the boundary pixel span calculated according to the reverse horizontal ratio, with the reverse pixel position as the neighborhood center; the upper and lower boundaries of the reverse machine vision image are determined by the boundary pixel span calculated according to the reverse vertical ratio; when the determined left, right, upper or lower boundaries exceed the corresponding machine vision image boundaries, the boundary positions that exceed the corresponding machine vision image boundaries are adjusted to the corresponding machine vision image boundaries; when the left boundary is not to the left of the right boundary or the upper boundary is not above the lower boundary, the initial dual-view pixel neighborhood is retained and a boundary adjustment anomaly mark is added to form the target dual-view pixel neighborhood.

[0072] S1.6. Using the target pixel locations covered by the target dual-view pixel neighborhood as deformable convolution sampling locations, deformable convolution is performed on the forward and reverse machine vision images, and the convolution results are arranged according to surface coordinates to generate a curvature feature map group. Specifically,

[0073] The target pixel positions covered by the dual-view pixel neighborhood are arranged according to the surface coordinate number and the order of the target pixel position relative to the neighborhood center. The horizontal and vertical position differences between the target pixel position and the regular convolution sampling position are used as deformable convolution sampling offsets to move the regular convolution sampling position to the corresponding target pixel position. When the target pixel position is between adjacent integer pixels, the interpolated pixel value is obtained according to the distance relationship between the adjacent pixels and the target pixel position. When the target pixel position exceeds the boundary of the forward machine vision image or the boundary of the reverse machine vision image, the corresponding target pixel position stops participating in sampling. The pixel values ​​of each sampling position in the forward machine vision image and the reverse machine vision image are convolved according to the convolution kernel position order, and the forward convolution result and the reverse convolution result are written into the corresponding surface coordinate position respectively. All convolution results are arranged according to the surface coordinate to generate a curvature feature map group.

[0074] S1.7. The surface coordinates and corresponding pixel positions of each curvature feature map in the curvature feature map group are fed into the perspective multi-point pose algorithm. The rotation and translation relationships of each curvature feature map relative to the surface data are fitted using the projection relationship between the surface coordinates and the corresponding pixel positions to obtain the initial image pose of each curvature feature map. Specifically,

[0075] The surface coordinates, pixel positions, camera intrinsic parameters, and lens distortion parameters corresponding to the same curvature feature map in the curvature feature map set, along with those obtained from camera calibration, are fed into a perspective multi-point pose algorithm using the EPnP solution method. The surface coordinates, composed of horizontal, vertical, and surface height coordinates, are used as the 3D corresponding positions, and the pixel positions as the 2D corresponding positions. Lens distortion at the pixel positions is corrected and converted into normalized imaging positions. At least four sets of non-overlapping surface coordinates and pixel positions are retained from the effective correspondences. The surface coordinates are represented as a weighted combination of four virtual control points. Projection constraints are applied to the spatial positions of the four virtual control points in the machine vision camera coordinate system using the normalized imaging positions. The rotation and translation relationships are then recovered between the virtual control point positions in the surface measurement coordinate system and the virtual control point positions in the machine vision camera coordinate system. The recovered rotation and translation relationships are applied to all surface coordinates, and the transformed surface coordinates are reprojected onto the [other coordinate system]. For each corresponding curvature feature map, the retracement deviation between the retracement pixel position and the original pixel position is obtained. The median value of all retracement deviations and the sum of three times the median absolute deviation are used as the retracement deviation limit. After deleting the surface coordinates and pixel positions with retracement deviations higher than the retracement deviation limit, the perspective multi-point pose algorithm is executed again until no more correspondences higher than the retracement deviation limit appear. When there are no less than four remaining valid correspondences, the remaining surface coordinates are not all located on the same straight line, the normalized imaging positions corresponding to different surface coordinates do not overlap, and the transformed surface coordinates are located in front of the machine vision camera's imaging direction, the final recovered rotation and translation relationships are used to form the initial image pose of the corresponding curvature feature map. If any of the following occurs: fewer than four remaining valid correspondences, all remaining surface coordinates are located on the same straight line, the normalized imaging positions corresponding to different surface coordinates overlap, or the transformed surface coordinates are located behind the machine vision camera's imaging direction, a pose fitting failure flag is added.

[0076] In this process, the surface coordinate numbers corresponding to the same curvature feature map in the curvature feature map group are used as pairing identifiers. The surface coordinates and pixel positions are sequentially used to form a 3D and 2D position correspondence. Camera intrinsic parameters and lens distortion parameters generated from camera calibration are fed into the perspective multi-point pose algorithm. The perspective multi-point pose algorithm uses the EPnP solution method to correct lens distortion of pixel positions and convert them into normalized imaging positions. EPnP is a mature, publicly available, and widely used PnP solution algorithm in the fields of machine vision and camera pose estimation. At least four sets of non-repeating surface coordinates and pixel positions are retained from the effective correspondence. All surface coordinates are represented as a weighted combination of four virtual control points. Projection constraints are applied to the spatial positions of the four virtual control points in the machine vision camera coordinate system using the normalized imaging positions. Then, the rotation and translation relationships are recovered between the virtual control point positions in the surface data coordinate system and the virtual control point positions in the machine vision camera coordinate system. The recovered rotation and translation relationships are applied to all surface coordinates, and the transformed surface coordinates are reprojected onto the corresponding curvature feature map. The algorithm obtains the retracement deviation between the retracement pixel position and the original pixel position. The median of all retracement deviations and the sum of three times the median absolute deviation are used as the retracement deviation limit. After deleting the surface coordinates and pixel positions with retracement deviations higher than the retracement deviation limit, the perspective multi-point pose algorithm is executed again until no more correspondences higher than the retracement deviation limit appear. When there are no fewer than four remaining valid correspondences and the transformed surface coordinates are located in front of the machine vision camera's imaging direction, the final recovered rotation and translation relationships are used to form the initial image pose of the corresponding curvature feature map. If any of the following occurs: fewer than four remaining valid correspondences, all remaining surface coordinates are located on the same straight line, the normalized imaging positions corresponding to different surface coordinates overlap, or the transformed surface coordinates are located behind the machine vision camera's imaging direction, a pose fitting failure flag is added. The perspective multi-point pose algorithm is a mature and commonly used camera pose solving method in the fields of machine vision and photogrammetry. EPnP can obtain rotation and translation relationships from no fewer than four known 3D positions and corresponding 2D projection positions. Mature machine vision libraries have already provided corresponding implementations.

[0077] S1.8 Extract rotation and translation relationships from the initial image pose. Rotate the surface coordinates according to the rotation relationships, and move the rotated surface coordinates according to the translation relationships. Project the moved surface coordinates onto each curvature feature map. Delete the returned pixel positions and corresponding surface coordinates that fall outside the neighborhood of the target dual-view pixel, forming surface pixel items. Specifically,

[0078] The rotation and translation relationships are read from the initial image poses of each curvature feature map. The spatial orientation of the surface coordinates in the surface data coordinate system is changed according to the rotation relationship, and the spatial position of the rotated surface coordinates in the corresponding machine vision camera coordinate system is changed according to the translation relationship. Then, the moved surface coordinates are projected back to the corresponding curvature feature map in combination with the camera intrinsic parameters. When the projected pixel position is inside or at the boundary of the target dual-view pixel neighborhood, the projected pixel position, the original pixel position, and the corresponding surface coordinates are retained. When the projected pixel position falls outside the target dual-view pixel neighborhood, the projection depth is behind the imaging direction of the machine vision camera, or the projected pixel position exceeds the boundary of the corresponding curvature feature map, the projected pixel position and the corresponding surface coordinates are deleted. The retained surface coordinates, the original pixel position, and the projected pixel position are combined to form a surface pixel item.

[0079] S1.9. The rotation and translation relationships of each curvature feature map are refitted using surface pixel terms to generate the pose of the calibration image. Specifically,

[0080] The projection relationship between the 3D and 2D corresponding positions is re-established using the surface coordinates in the surface pixel term and the original pixel position. Taking the rotation and translation relationships in the initial image pose as the starting point for refitting, the surface coordinates in the surface pixel term are projected back onto the corresponding curvature feature map. The lateral and longitudinal projection deviations between the projected pixel position and the original pixel position are calculated, and the sum of the squares of all lateral and longitudinal projection deviations is used as the refit target value. The update amount of the rotation and translation relationships is calculated using a Gaussian-Newton iteration method, and this update amount is applied to the current rotation and translation relationships. After the update, the surface coordinates are re-projected. When the refit target value decreases, the updated rotation and translation relationships are retained; when the refit target value increases, the current update is canceled and the update amount in the next round is reduced. A refit convergence threshold not exceeding the upper limit of the camera calibration reprojection error is preset. Refitting stops when the change in the median value of the reprojection deviation corresponding to all reprojected pixel positions in two consecutive rounds does not exceed the refit convergence threshold. When there are fewer than four groups of surface pixel items and the surface coordinates corresponding to the surface pixel items are all located on the same straight line, the initial image pose is retained and a refitting failure mark is added. When the surface pixel items can maintain a stable projection relationship, the finally retained rotation and translation relationships are used to form the calibration image pose.

[0081] S1.10. Update the rotation and translation relationships of each curvature feature map by calibrating the image pose, and complete the image pose calibration of each curvature feature map. Specifically,

[0082] Replace the rotation and translation relationships in the initial image pose of each curvature feature map with the rotation and translation relationships in the calibration image pose, and re-project the surface coordinates in the surface pixel item using the replaced rotation and translation relationships. When the projected pixel position is still within the neighborhood and boundary position of the target dual-view pixel, retain the calibration correspondence between the surface coordinates and the pixel position. When the projected pixel position leaves the neighborhood of the target dual-view pixel, remove the corresponding surface pixel item and add a calibration out-of-bounds mark. Write the retained calibration correspondence into the image pose record of each curvature feature map to complete the image pose calibration of each curvature feature map.

[0083] S1.11. Using the same surface coordinates as the index key, the pixel positions in each calibrated curvature feature map are listed in the pixel position field corresponding to the index key, forming a cross-viewpoint pixel index table, specifically as follows:

[0084] Using the surface coordinate number of the same surface coordinate as the index key, the forward pixel position, reverse pixel position, curvature feature map identifier, and image acquisition sequence corresponding to the same surface coordinate in each calibrated curvature feature map are entered into the pixel position item corresponding to the index key. When the same surface coordinate corresponds to multiple pixel positions in the same curvature feature map, the pixel position with the highest degree of overlap between the re-projected pixel position and the original pixel position is retained. When the same surface coordinate has only a single-sided pixel position, the single-sided pixel position is retained and a single-sided pixel position missing mark is added. The index key and the corresponding pixel position item are arranged according to the surface coordinate number to form a cross-view pixel index table.

[0085] S2. Perform contrast segmentation on the curvature feature map group, take the segmented brightness abnormal area as the defect candidate area, use the cross-view pixel index table to check the surface coordinates of the candidate area corresponding to each defect candidate area, and connect the defect candidate areas with continuous surface coordinates to obtain the defect development trajectory chain.

[0086] S2.1 Contrast segmentation compares the brightness differences of adjacent pixels in the curvature feature map, groups consecutive pixels with consistent brightness changes into the same segmentation region, and delineates segmentation boundaries at the inflection points of brightness changes to form a contrast segmentation map. Specifically...

[0087] Within each curvature feature map group, the brightness of adjacent pixels is read point by point in the horizontal, vertical, and diagonal directions. The direction of brightness increase, decrease, and maintenance of adjacent pixels is recorded, and the brightness change amplitude between consecutive adjacent pixels is compared. When consecutive pixels have the same brightness change direction and the difference in brightness change amplitude does not exceed the brightness change consistency judgment limit, consecutive pixels are grouped into the same segmentation region. When the brightness change direction is reversed and the difference in brightness change amplitude exceeds the brightness change consistency judgment limit, segmentation boundaries are drawn on both sides of the change location. When the curvature feature map edge lacks outer adjacent pixels, the curvature feature map edge is used as the segmentation boundary. The brightness change consistency judgment limit is determined by the median absolute deviation of the absolute values ​​of the brightness difference of all adjacent pixels in the same curvature feature map. The example value is three times the median absolute deviation, which is used to exclude short-term brightness fluctuations caused by local imaging noise and form a contrast segmentation map.

[0088] S2.2. Compare the brightness of pixels inside the segmentation boundary and their adjacent pixels outside the segmentation boundary in the contrast segmentation image region by region. Identify brightness anomaly regions where the brightness of the inner pixel is consistently higher than that of the outer adjacent pixel, and use these brightness anomaly regions as candidate defect regions. Specifically...

[0089] Select segmented regions one by one from the contrast segmentation map, and read the brightness of the pixels inside the segmentation boundary and the brightness of the adjacent pixels outside the segmentation boundary along the segmentation boundary. Form brightness comparison position pairs with the inner pixels and the outer adjacent pixels that are in the same horizontal position, vertical position and boundary normal direction. When the brightness of the inner pixels in the continuous brightness comparison position pairs is higher than that of the outer adjacent pixels, the area enclosed by the continuous brightness comparison position pairs is retained. When the brightness of the inner pixels is the same as that of the outer adjacent pixels, or when the brightness of the inner pixels is lower than that of the outer adjacent pixels, the corresponding boundary segment stops participating in the formation of the brightness abnormal area. When the segmentation boundary touches the edge of the curvature feature map and lacks the brightness of the outer adjacent pixels, the corresponding boundary segment is not used as the basis for brightness comparison. The brightness abnormal area that still maintains the brightness of the inner pixels higher than that of the outer adjacent pixels after all valid boundary segments have been verified is regarded as the defect candidate area.

[0090] S2.3. Using the pixel positions of the candidate regions covered by the defect candidate regions, find the corresponding surface coordinates of the candidate regions with the same pixel positions in the cross-view pixel index table. Specifically,

[0091] Using the curvature feature map identifier of the defect candidate region and the pixel position of the candidate region covered by the defect candidate region as joint search conditions, the curvature feature map identifier, horizontal pixel coordinates, and vertical pixel coordinates are compared item by item in the pixel position field of the cross-view pixel index table. When the curvature feature map identifier and pixel coordinates are consistent, the surface coordinates recorded by the corresponding index key are read and included in the same defect candidate region. When multiple candidate region pixel positions point to the same surface coordinates, only one surface coordinate record is retained. When the candidate region pixel position does not appear in the cross-view pixel index table, a pixel index missing mark is added and the candidate region pixel position stops participating in the surface coordinate collection, thus forming the candidate region surface coordinates corresponding to each defect candidate region.

[0092] S2.4. Based on the image acquisition order of each curvature feature map, set the order of each defect candidate region, specifically as follows:

[0093] The curvature feature map identifiers corresponding to each defect candidate region are matched with the forward and reverse identifiers and image acquisition order recorded in the cross-view pixel index table, and the defect candidate regions are assigned to the forward defect candidate region group and the reverse defect candidate region group respectively. Within the forward defect candidate region group, the defect candidate regions are arranged from front to back according to the image acquisition order of the corresponding curvature feature map. Within the reverse defect candidate region group, the defect candidate regions are arranged from front to back according to the image acquisition order of the corresponding curvature feature map. Then, the defect candidate regions formed in the same acquisition are arranged in the ascending direction of the surface coordinate number of the candidate region within the same curvature feature map. When the forward and reverse defect candidate regions cover the same surface coordinate of the candidate region, the forward and reverse defect candidate regions are registered in the same front-to-back order position, and the connection relationship is not determined by the acquisition time order between the forward and reverse machine vision images. When the image acquisition order is complete, the front-to-back order is written in sequence. When the image acquisition order is missing, an acquisition order missing mark is added and the corresponding defect candidate region is stopped from participating in subsequent connections, forming defect candidate regions with front-to-back order.

[0094] S2.5 Connect adjacent defect candidate regions with continuous surface coordinates to obtain the defect development trajectory chain, specifically,

[0095] The surface coordinates of candidate regions covered by the same defect candidate region are arranged in ascending order of surface coordinate number. The surface coordinates of the first candidate region are taken as the starting surface coordinates of the defect candidate region, and the surface coordinates of the last candidate region are taken as the ending surface coordinates of the defect candidate region. In both the forward and reverse defect candidate region groups, adjacent defect candidate regions are compared pair by pair. If the surface coordinates of the ending surface coordinates of the previous defect candidate region and the starting surface coordinates of the next defect candidate region maintain an adjacent numbering relationship in the surface data, the two defect candidate regions are connected in order. The two candidate region surface coordinates are separated only by a surface not covered by the defect candidate region. When the coordinates of two candidate regions are such that the spatial interval between their surface coordinates does not exceed the sum of the intervals of the coordinates of two adjacent surface regions at the corresponding positions, the two candidate defect regions are still connected in the order of succession. The connection is terminated when the interval exceeds one surface coordinate. When multiple candidate defect regions simultaneously meet the connection conditions, the candidate defect region with the smallest spatial interval is connected. Unconnected candidate defect regions are retained as independent starting positions. The connection is terminated when no continuous candidate region surface coordinates are found. Candidate defect regions formed by forward continuous connection and candidate defect regions formed by reverse continuous connection are grouped into the same defect development trajectory chain according to the same candidate region surface coordinates. The continuously connected candidate defect regions, candidate region surface coordinates, and succession order together form the defect development trajectory chain.

[0096] S3. Examine the development changes of the forward and reverse machine vision images along the defect development trajectory chain, encode the position order of each development change, generate forward development pattern string and reverse development pattern string, compare the forward development pattern string and reverse development pattern string through dynamic time warping mechanism, and map the development difference position obtained by comparison into a bidirectional development pattern diagram.

[0097] S3.1. According to the sequential order of the defect development trajectory chain, assign corresponding trajectory numbers to the forward and reverse pixel positions corresponding to the surface coordinates of the candidate region, forming a forward trajectory pixel sequence and a reverse trajectory pixel sequence, specifically as follows:

[0098] The surface coordinates of candidate regions in the defect development trajectory chain are arranged sequentially. The surface coordinates of the candidate regions are used as the index keys in the cross-view pixel index table. The forward and reverse pixel positions corresponding to the same surface coordinates of the candidate regions are read, and the corresponding position trajectory numbers are assigned sequentially from the beginning of the defect development trajectory chain. When both forward and reverse pixel positions exist, the same position trajectory number is written to the forward and reverse pixel positions respectively. When a pixel position on one side is missing, the position trajectory number is retained and a missing pixel position mark is added. The forward and reverse pixel positions are arranged according to the position trajectory number to form a forward trajectory pixel sequence and a reverse trajectory pixel sequence.

[0099] S3.2. Along the forward trajectory pixel sequence and the reverse trajectory pixel sequence, compare the brightness of adjacent pixels with the same trajectory index, and extract the pixel positions where the brightness change direction is reversed to form the forward development change position and the reverse development change position. Specifically,

[0100] Pixel positions corresponding to consecutive same-position trajectory numbers are selected from the forward trajectory pixel sequence and the reverse trajectory pixel sequence, respectively. The brightness of adjacent pixels is read along the increasing direction of the same-position trajectory number. The increase of adjacent pixel brightness is recorded as the upward direction, the decrease of adjacent pixel brightness is recorded as the downward direction, and the same brightness of adjacent pixels is recorded as brightness maintenance. When the front side of the same pixel position is the upward direction and the back side is the downward direction, or the front side is the downward direction and the back side is the upward direction, the same pixel position is extracted as the development change position. When brightness maintenance occurs continuously, the corresponding pixel position of brightness maintenance is crossed and the brightness of the nearest non-identical pixels before and after is compared. The starting pixel position of the sequence, the ending pixel position of the sequence, and the pixel position with a single-sided pixel position missing mark do not participate in the direction reversal determination, and the forward development change position and the reverse development change position are formed respectively.

[0101] S3.3. Based on the ascending order of the corresponding trajectory numbers, assign consecutive numbers to the forward development change positions and the reverse development change positions, respectively. Concatenate the forward numbers sequentially to form a forward development pattern string, and concatenate the reverse numbers sequentially to form a reverse development pattern string. Specifically,

[0102] Based on the increasing order of the corresponding trajectory numbers attached to the forward and reverse development change positions, continuously increasing forward and reverse numbers are assigned from the start to the end of the defect development trajectory chain, and the corresponding corresponding trajectory number is retained under the forward and reverse number names. The forward numbers are connected sequentially according to the corresponding corresponding trajectory numbers, and the reverse numbers are connected sequentially according to the corresponding corresponding trajectory numbers. When both the forward and reverse development change positions exist, a forward development pattern string and a reverse development pattern string are formed. When no development change position is formed on either side, a no-development-change mark is retained and the corresponding defect development trajectory chain is stopped from entering the subsequent development number difference array.

[0103] S3.4. Arrange the forward and reverse numbers horizontally and vertically according to the same trajectory sequence number, respectively. Compare the differences between each forward and reverse number to form a developing number difference array. Specifically,

[0104] The forward number in the forward development pattern string is used as the row identifier of the development number difference array, and the reverse number in the reverse development pattern string is used as the column identifier of the development number difference array. The row and column identifiers are arranged in ascending order of the corresponding position trajectory numbers of the forward and reverse numbers. The corresponding position trajectory numbers, brightness change direction, and pixel brightness difference before and after the brightness change direction is reversed are read row by row and column by column. The sequence interval between the corresponding position trajectory numbers of the forward and reverse numbers is calculated. The same brightness change direction is recorded as the direction consistency value, and the different brightness change directions are recorded as the direction difference value. The pixel brightness difference is converted according to the pixel brightness range of the forward and reverse machine vision images. The sequence interval, direction consistency value or direction difference value, and the converted pixel brightness difference are unified to the same value range. The values ​​after the unified value range are added to obtain the development difference value of the corresponding array position. The development difference value, forward number, reverse number, and corresponding position trajectory numbers of the forward and reverse numbers are retained at the array position to form the development number difference array.

[0105] Among them, the forward number and the reverse number are used to identify the position of forward development change and the position of reverse development change, and the difference between the forward number and the reverse number is not directly used as the development difference value.

[0106] S3.5. Taking the upper left end of the development number difference array as the starting point for normalization, connect the previous receiving position to the adjacent array positions along the diagonal, horizontal, and vertical directions of the development number difference array. Continue the array values ​​of the adjacent array positions to the cumulative array value retained by the previous receiving position. Retain the previous receiving direction with the smallest cumulative array value at each adjacent array position, and connect each previous receiving direction in reverse from the end of the development number difference array to form the development mode normalization path. Specifically,

[0107] Using the first array position at the top left of the development number difference array as the starting point for normalization, the array value at the first array position is used as the initial cumulative array value. For the array positions within the development number difference array other than the first array position, the cumulative array values ​​already retained at the top left diagonal array position, the left array position, and the top array position are read respectively. The array value at the current array position is then connected to the cumulative array values ​​retained at the three preceding array positions to form diagonal candidate cumulative array values, horizontal candidate cumulative array values, and vertical candidate cumulative array values. The candidate cumulative array value with the smallest value and its corresponding preceding direction are retained at the current array position. When the candidate cumulative array values ​​are the same, the diagonal direction, horizontal direction, and vertical direction are retained in sequence. The normalization path is then connected in reverse from the bottom right of the development number difference array along the preceding direction retained at each array position to the starting point, forming the development mode normalization path.

[0108] S3.6. Along the development pattern's regular path, extract the horizontal and vertical junction positions, as well as the diagonal junction positions where the forward and reverse numbers differ, as development difference positions. Specifically,

[0109] The receiving direction between adjacent array positions is read sequentially along the regular path of the developing mode. When the receiving direction is horizontal, the array position corresponding to the horizontal receiving end is taken as the developing difference position. When the receiving direction is vertical, the array position corresponding to the vertical receiving end is taken as the developing difference position. When the receiving direction is diagonal and the forward number and reverse number corresponding to the array position are different, the array position corresponding to the diagonal receiving end is taken as the developing difference position. When the forward number and reverse number are the same in the diagonal direction, the corresponding array position is not truncated. The same array position that appears repeatedly is only retained once, forming a developing difference position with row number and column number.

[0110] S3.7. According to the row and column numbers of the development difference position in the development number difference array, arrange the forward number identified by the row number and the reverse number identified by the column number on the forward and reverse sides respectively, and connect the forward and reverse numbers under the same development difference position to form a bidirectional development pattern diagram, specifically,

[0111] Read the corresponding forward number according to the row number of the development difference position in the development number difference array, and read the corresponding reverse number according to the column number of the development difference position in the development number difference array. Arrange the forward numbers on the forward side in ascending order of the same position trajectory number, and arrange the reverse numbers on the reverse side in ascending order of the same position trajectory number. Read the corresponding same position trajectory number, brightness change direction, and pixel brightness difference before and after the brightness change direction is reversed from the forward and reverse development change positions. Read the start coordinate, end coordinate, and surface coordinate span of the corresponding candidate area surface coordinates from the defect candidate area. Retain the same position trajectory number, brightness change direction, pixel brightness difference, start coordinate, end coordinate, and surface coordinate span in the corresponding forward and reverse numbers, respectively. Establish a connection between the forward and reverse numbers corresponding to the same development difference position. When multiple development difference positions point to the same forward number or the same reverse number, retain the same number and establish corresponding connections. Arrange all numbers, the development attributes corresponding to the numbers, and the connection relationships according to the order of the development mode regularization path to form a bidirectional development mode diagram.

[0112] S4. Defect category is determined by the two-way development pattern map using the defect discrimination model. The determined defect category is then combined with the corresponding candidate region surface coordinates to form the lens defect recognition result.

[0113] S4.1. The two-way development pattern diagram is fed into the defect discrimination model. The defect discrimination model determines the defect category of the two-way development pattern diagram based on the numbering arrangement, connection relationship, and development characteristics associated with each number, and outputs the defect category. Specifically,

[0114] Based on the forward and reverse numbers corresponding to the development difference positions in the bidirectional development pattern diagram, the forward and reverse numbers are set as development nodes, and the corresponding trajectory numbers, forward side identifiers, reverse side identifiers, brightness change directions, and pixel brightness differences before and after the brightness change direction is reversed are read. According to the corresponding trajectory numbers, candidate defect regions with the same order are located in the defect development trajectory chain. From the located candidate defect regions, the average brightness, standard deviation of brightness, number of pixels covered, contour length, major axis length, minor axis length, aspect ratio, roundness, starting coordinates of the candidate region's surface coordinates, ending coordinates of the candidate region's surface coordinates, span of the candidate region's surface coordinates, and the distance from the candidate defect region's contour to the outer contour of the optical lens to be identified are read. The shortest pixel distance and the intersection state between the defect candidate region contour and the outer contour of the optical lens to be identified are read and written into the node attributes of the corresponding developing node. The average brightness of the region is represented by the average brightness of the pixels covered by the defect candidate region. The standard deviation of the region brightness is represented by the degree of dispersion between the brightness of the pixels covered by the defect candidate region and the average brightness of the region. The number of pixels covered by the region is represented by the total number of pixels in the region enclosed by the defect candidate region contour. The length of the region contour is represented by the arc length of the contour formed by connecting the boundary pixels of the defect candidate region in adjacent order. The length of the major axis and the length of the minor axis of the region are represented by the length of the major axis and the length of the minor axis of the circumscribed ellipse of the defect candidate region. The aspect ratio of the region is represented by the ratio of the length of the major axis to the length of the minor axis of the region. The roundness of the region is represented by the ratio of four times pi of the area of ​​the defect candidate region to the square of the length of the region contour.

[0115] Connect the forward and reverse development nodes corresponding to the same development difference position, and according to the order of the development difference position in the defect development trajectory chain, connect the forward development nodes that are sequentially adjacent on the forward side and the reverse development nodes that are sequentially adjacent on the reverse side. Write the connection type, connection order, number difference between forward and reverse numbers, indicative trajectory number interval, connection direction, surface coordinate spacing between adjacent candidate regions, and the absolute difference between forward and reverse pixel brightness differences into the edge attributes of the corresponding connections to form defect discrimination map data. The number difference, indicative trajectory number interval, and connection direction are used to characterize the changes in forward and reverse development. The order of development changes, the direction of brightness change, pixel brightness difference, average brightness of the region, and standard deviation of brightness of the region are used to characterize the development intensity and brightness distribution of the defect candidate region. The number of pixels covered by the region, the region outline length, the region major axis length, the region minor axis length, the region aspect ratio, and the region roundness are used to characterize the outline shape of the defect candidate region. The surface coordinate span of the candidate region and the surface coordinate spacing between adjacent candidate regions are used to characterize the extension state of the defect development trajectory. The shortest pixel distance from the outline of the defect candidate region to the outer outline of the optical lens to be identified and the intersection state are used to characterize the positional relationship between the defect candidate region and the edge of the optical lens to be identified. The difference in number and the connection direction are not used as the sole basis for defect category discrimination.

[0116] Normalize the forward and reverse serial numbers according to the maximum serial number in the same bidirectional imaging pattern; normalize the co-location serial number and the interval between co-location serial numbers according to the maximum co-location serial number in the same bidirectional imaging pattern; normalize the pixel brightness difference, the absolute difference between the forward and reverse pixel brightness differences, the average brightness of the region, and the standard deviation of the region brightness according to the pixel brightness value range of the forward and reverse machine vision images; normalize the region contour length, the region major axis length, and the region minor axis length according to the maximum pixel span of the imaging region of the optical lens to be identified; normalize the surface covered by the surface shape data. The coordinate range is normalized for the starting coordinates, ending coordinates, span, and spacing of the candidate region surface coordinates. The shortest pixel distance from the defect candidate region contour to the outer contour of the optical lens to be identified is normalized according to the maximum pixel span corresponding to the outer contour of the optical lens to be identified. The intersection of the defect candidate region contour and the outer contour of the optical lens to be identified is recorded as the first intersection state value, and the non-intersection is recorded as the second intersection state value. The normalized node attributes and edge attributes are then fed into the defect discrimination model implemented using a graph convolutional network.

[0117] The defect category set includes scratches, pitting, bubbles, impurities, edge chipping, and coating anomalies. The defect discrimination model is configured with an input encoding layer, a first-image convolutional layer, a second-image convolutional layer, a global average pooling layer, a global max pooling layer, a feature concatenation layer, a fully connected layer, and a Softmax layer. The input encoding layer maps node attributes to the same feature dimension and performs edge feature encoding on edge attributes. The first-image convolutional layer, based on the connectivity in the defect discrimination map data, gathers the development order features, brightness variation features, defect candidate region contour morphology features, surface position features, and edge features corresponding to directly adjacent development nodes. The second-image convolutional layer gathers the interval of the corresponding trajectory number between adjacent development difference positions, the surface coordinate spacing between adjacent candidate regions, the surface coordinate span of candidate regions, and the positional relationship between the defect candidate region and the outer contour of the optical lens to be identified, forming a segment. Point discrimination features are defined as follows: a global average pooling layer obtains the average value of all node discrimination features under the same feature dimension, a global max pooling layer obtains the maximum value of all node discrimination features under the same feature dimension, a feature concatenation layer connects the outputs of the global average pooling layer and the global max pooling layer to form the overall development features; a fully connected layer maps the overall development features to the output dimensions corresponding to the six defect categories, and a softmax layer outputs the category probabilities corresponding to scratches, pits, bubbles, impurities, edge chipping, and coating anomalies, respectively, and the defect category corresponding to the maximum category probability is determined as the discrimination result; when there is no effective connection between the forward and reverse development nodes in the bidirectional development pattern diagram, a defect category discrimination failure mark is added; when the category probabilities corresponding to two or more defect categories simultaneously reach the maximum value, a defect category pending verification mark is added, and the defect category is output.

[0118] The training process for the defect discrimination model is as follows:

[0119] Obtain bidirectional imaging pattern diagrams and manual review records generated from historical testing. The manual review records include information related to historical bidirectional imaging patterns. Figure 1 A corresponding label for scratches, pits, bubbles, impurities, edge chipping, and coating anomalies is generated. The manual review record is generated by the reviewer after verifying the historical forward machine vision images, historical reverse machine vision images, defect candidate region contours, candidate region surface coordinates, and corresponding detection records. The labels for scratches, pits, bubbles, impurities, and edge chipping are confirmed using the microscopic review images and contour review results of the defect candidate regions. The labels for coating anomalies are confirmed using the film thickness detection records and spectral transmittance detection records corresponding to the defect candidate regions. The bidirectional development pattern diagram generated from historical detections and the defect category labels in the manual review record are combined to form a bidirectional development pattern diagram training sample.

[0120] Training samples, validation samples, and test samples are divided according to the lens inspection batch. The bidirectional development pattern diagram formed by the same optical lens to be identified is only assigned to one of these three sample sets. The training samples of the bidirectional development pattern diagram are processed according to the development node composition method, node attributes, connection relationships, edge attributes, and normalization method used in the lens defect identification stage. The output dimension of the first convolutional layer is set to 64, and the output dimension of the second convolutional layer is set to 32. ReLU activation is used between the first and second convolutional layers, with a random deactivation ratio of 20%. The batch training size is set to 32, the initial learning rate is set to 0.1%, and the weight decay coefficient is set to 0.01%. The 64-dimensional first convolutional layer... The output is used to retain the development order features, brightness change features, defect candidate region contour morphology features, and surface position features corresponding to adjacent development nodes. The output of the 32D second graph convolutional layer is used to merge the node features that repeatedly appear at the development difference positions of adjacent nodes and retain the overall development features corresponding to the defect category discrimination. The 20% random deactivation ratio is used to limit the defect discrimination model from continuously relying on fixed development node combinations. The batch training number is 32 to keep the parameter update amplitude of adjacent training batches at the same order of magnitude. The initial learning rate of 0.1% is used to limit the single parameter update amplitude. The weight decay coefficient of 0.01% is used to limit the absolute value of the parameters of the input encoding layer, edge feature encoding, first graph convolutional layer, second graph convolutional layer, and fully connected layer from continuously increasing.

[0121] The training samples of the bidirectional imaging pattern are fed into the defect discrimination model for forward propagation. The difference between the class probability and the defect class label is calculated using a cross-entropy loss function with class weights. The class weight of each defect class is determined according to the ratio of the total number of training samples to the number of training samples for the corresponding defect class. The gradient of the defect discrimination model parameters with respect to the cross-entropy loss function with class weights is calculated through backpropagation, and the parameters of the input encoding layer, edge feature encoding layer, first graph convolutional layer, second graph convolutional layer, and fully connected layer are updated using the Adam optimization method. During training, the cross-entropy loss and macro-average F1 value of the validation samples are read in each round. When the macro-average F1 value of the validation samples in the current training round is higher than the macro-average F1 value of the validation samples in the previously retained training rounds, [the following occurs]. The current defect discrimination model parameters are retained. When the macro-average F1 value of the validation samples in the current training round is the same as that in the previously retained training rounds, the defect discrimination model parameters with the lower cross-entropy loss value of the validation samples are retained. When the macro-average F1 value of the validation samples does not increase after ten consecutive training rounds, the parameter update is stopped, and the defect discrimination model parameters corresponding to the highest macro-average F1 value of the validation samples are determined as the final defect discrimination model parameters. The test samples are fed into the defect discrimination model using the final defect discrimination model parameters. The ratio between the number of correct discriminations for scratches, pits, bubbles, impurities, edge chipping, and coating anomalies and the number of test samples for the corresponding defect categories are calculated, and the macro-average F1 value of the test samples is calculated. The test samples do not participate in the defect discrimination model parameter update.

[0122] S4.2. Based on the same trajectory number of the development difference position in the bidirectional development pattern diagram, locate the surface coordinates of the candidate region under the same sequential order in the defect development trajectory chain, and merge the defect category and the surface coordinates of the candidate region into the lens defect identification result.

[0123] Using the corresponding trajectory numbers preserved at each development difference position in the bidirectional development pattern diagram as positioning identifiers, the same corresponding trajectory number is sequentially searched in the defect development trajectory chain, and the surface coordinates of the candidate region corresponding to the same corresponding trajectory number are read. When the corresponding trajectory number corresponds to only one candidate region surface coordinate, a correspondence between the defect category and the candidate region surface coordinate is established. When the corresponding trajectory number corresponds to multiple candidate region surface coordinates consecutively, all candidate region surface coordinates are preserved in the order of the defect development trajectory chain. When the corresponding trajectory number does not appear in the defect development trajectory chain, a surface coordinate positioning failure mark is added and the corresponding development difference position is stopped from participating in the merging. The defect category, candidate region surface coordinates, and corresponding corresponding trajectory number are written into the same recognition record to form the lens defect recognition result.

[0124] This embodiment also provides a computer device applicable to the machine vision-based optical lens defect identification method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the machine vision-based optical lens defect identification method proposed in the above embodiment.

[0125] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0126] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the machine vision-based optical lens defect identification method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0127] In summary, this invention improves the characterization stability of the development relationship of dual-view defects in complex curvature lenses by dynamically warping and mapping the forward and reverse development pattern strings. The co-position trajectory number establishes a unified order benchmark for the forward and reverse development change positions. The development number difference array carries the nonlinear spacing and order offset between bidirectional development changes. The development pattern warping path retains the lateral, longitudinal, and diagonally connected positions with different numbers, transforming local stretching, compression, and misalignment into development difference positions. The bidirectional development pattern diagram presents the distribution order and associated form of development differences through numbering and connection relationships, providing a traceable basis for the defect discrimination model and improving the consistency and accuracy of defect category discrimination and candidate region surface coordinate positioning.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying defects in optical lenses based on machine vision, characterized in that: include, Acquire surface shape data, forward machine vision image, and reverse machine vision image. Project the surface coordinates in the surface shape data onto the pixel positions in the forward and reverse machine vision images. Adjust the pixel neighborhood scale of the pixel position corresponding to the surface coordinates through the local curvature in the surface shape data to generate a curvature feature map group. Calibrate the image pose of each curvature feature map in the curvature feature map group according to the same surface coordinates to form a cross-view pixel index table. Contrast segmentation is performed on the curvature feature map group, and the segmented brightness abnormal areas are used as defect candidate areas. The cross-view pixel index table is used to check the surface coordinates of each defect candidate area, and the defect candidate areas with continuous surface coordinates are connected to obtain the defect development trajectory chain. The development changes of the forward and reverse machine vision images are examined along the defect development trajectory chain. The position order of each development change is encoded to generate a forward development pattern string and a reverse development pattern string. The forward and reverse development pattern strings are compared through a dynamic time warping mechanism, and the development difference position obtained by comparison is mapped into a bidirectional development pattern diagram. The defect classification model is used to identify the defect category of the bidirectional imaging pattern. The identified defect category is then combined with the corresponding candidate region surface coordinates to form the lens defect identification result.

2. The machine vision-based optical lens defect identification method as described in claim 1, characterized in that: The generation of the curvature feature map group specifically includes: The surface coordinates in the surface data are projected onto the pixel positions of the forward machine vision image and the reverse machine vision image, respectively, and the two pixel positions corresponding to the same surface coordinates are combined into a dual-view pixel position pair. The local curvature in the surface data is used to define the radius of the neighborhood of the dual-view pixel position relative to the surrounding neighborhood, thus forming the initial dual-view pixel neighborhood; The horizontal and vertical coordinate spacing of adjacent surface coordinates within the initial dual-view pixel neighborhood is proportionally converted to the horizontal and vertical spacing of corresponding pixel positions in the forward and reverse machine vision images, respectively, to obtain the dual-view scaling ratio. The horizontal scaling factor in the dual-view scaling ratio is applied to the left and right boundaries of the initial dual-view pixel neighborhood, and the vertical scaling factor in the dual-view scaling ratio is applied to the upper and lower boundaries of the initial dual-view pixel neighborhood, and the target dual-view pixel neighborhood is output. The target pixel locations covered by the target dual-view pixel neighborhood are used as deformable convolution sampling locations. Deformable convolution is performed on the forward machine vision image and the reverse machine vision image, and the convolution results are arranged according to the surface coordinates to generate a curvature feature map group.

3. The machine vision-based optical lens defect identification method as described in claim 2, characterized in that: The formation of the cross-view pixel index table specifically involves... The surface coordinates and corresponding pixel positions of each curvature feature map in the curvature feature map group are fed into the perspective multi-point pose algorithm. The rotation and translation relationships of each curvature feature map relative to the surface data are fitted by the projection relationship between the surface coordinates and the corresponding pixel positions to obtain the initial image pose of each curvature feature map. Extract rotation and translation relationships from the initial image pose. Rotate the surface coordinates according to the rotation relationship and move the rotated surface coordinates according to the translation relationship. Project the moved surface coordinates onto each curvature feature map. Delete the back-projected pixel positions and corresponding surface coordinates that fall outside the neighborhood of the target dual-view pixel to form surface pixel items. The rotation and translation relationships of each curvature feature map are refitted using the surface pixel term to generate the pose of the calibration image; The rotation and translation relationships of each curvature feature map are updated by calibrating the image pose, thus completing the image pose calibration of each curvature feature map; Using the same surface coordinates as the index key, the pixel positions in each calibrated curvature feature map are listed in the pixel position item corresponding to the index key, forming a cross-view pixel index table.

4. The machine vision-based optical lens defect identification method as described in claim 3, characterized in that: The contrast segmentation involves comparing the brightness differences between adjacent pixels in the curvature feature map, grouping consecutive pixels with consistent brightness changes into the same segmentation region, and defining segmentation boundaries at the inflection points of brightness changes to form a contrast segmentation map.

5. The machine vision-based optical lens defect identification method as described in claim 4, characterized in that: The obtained defect development trajectory chain is specifically as follows: By comparing the brightness of pixels inside the segmentation boundary with the adjacent pixels outside the segmentation boundary in the contrast segmentation image region by region, abnormal brightness regions where the brightness of the inner pixels is consistently higher than that of the outer adjacent pixels are screened out, and these abnormal brightness regions are used as candidate defect regions. Using the pixel positions of the candidate regions covered by the defect candidate regions, the surface coordinates of the candidate regions corresponding to the same pixel positions are found in the cross-view pixel index table. Based on the image acquisition order of each curvature feature map, the front and back order of each defect candidate region is set; By connecting adjacent candidate defect regions with continuous surface coordinates, a defect development trajectory chain is obtained.

6. The machine vision-based optical lens defect identification method as described in claim 5, characterized in that: The generation of the forward development mode string and the reverse development mode string specifically involves... According to the order of the defect development trajectory chain, the positive and negative pixel positions corresponding to the surface coordinates of the candidate region are assigned the same trajectory number to form the positive trajectory pixel sequence and the negative trajectory pixel sequence. Along the forward trajectory pixel sequence and the reverse trajectory pixel sequence, the brightness of the pixels marked with adjacent same position trajectory numbers is compared, and the pixel position where the brightness change direction is reversed is extracted to form the forward development change position and the reverse development change position. Based on the ascending order of the corresponding trajectory numbers, the forward development change positions and the reverse development change positions are numbered consecutively. The forward numbers are then connected sequentially to form a forward development pattern string, and the reverse numbers are connected sequentially to form a reverse development pattern string.

7. The machine vision-based optical lens defect identification method as described in claim 6, characterized in that: The process of mapping the locations of development differences obtained from the comparison into a bidirectional development pattern map specifically involves... The forward and reverse numbers are arranged horizontally and vertically according to the same trajectory number, respectively. The differences between each forward number and each reverse number are compared one by one to form a developing number difference array. Taking the upper left end of the development number difference array as the starting point for regularization, the previous receiving position is connected to the adjacent array position along the diagonal, horizontal and vertical directions of the development number difference array, respectively. The array value of the adjacent array position is continued to the cumulative array value retained by the previous receiving position. The previous receiving direction with the smallest cumulative array value is retained at each adjacent array position, and the previous receiving directions are connected in reverse from the end of the development number difference array to form the development mode regularization path. The horizontal and vertical receiving positions, as well as the diagonal receiving positions with different forward and reverse numbers, are extracted along the regular path of the developing pattern as the developing difference positions; According to the row and column numbers of the development difference position in the development number difference array, the forward number identified by the row number and the reverse number identified by the column number are arranged on the forward and reverse sides respectively, and the forward and reverse numbers under the same development difference position are connected to form a bidirectional development pattern diagram.

8. The machine vision-based optical lens defect identification method as described in claim 7, characterized in that: The process of merging the determined defect category with the corresponding surface coordinates to form the lens defect identification result is as follows: The two-way development pattern diagram is fed into the defect discrimination model. The defect discrimination model judges the defect category of the two-way development pattern diagram based on the numbering arrangement, connection relationship and development characteristics associated with each number in the two-way development pattern diagram, and outputs the defect category. Based on the same trajectory number of the development difference position in the bidirectional development pattern diagram, the surface coordinates of candidate regions with the same preceding and following order in the defect development trajectory chain are located, and the defect category and the surface coordinates of the candidate regions are merged into the lens defect identification result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the machine vision-based optical lens defect identification method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the machine vision-based optical lens defect identification method according to any one of claims 1 to 8.

Images