Anti-loose nut anti-rotation mark automatic reading method and system based on image recognition

Abstract

This invention discloses an automatic reading method and system for anti-rotation markings on anti-loosening nuts based on image recognition, relating to the field of image data processing technology. The method includes: acquiring images under multiple supplementary lighting directions and summarizing them to obtain a multi-light image group; locating the intersection line between the nut's outer contour and the component's mating surface based on the multi-light image group; establishing geometric reference coordinates from the intersection line between the hexagonal vertices and the component's mating surface; cropping a torque seal mark candidate area image group under the geometric reference coordinates; extracting, fusing, and separating a new torque seal mark layer image and an old torque seal mark layer image based on the torque seal mark candidate area image group; bridging cross-edge segments of the new torque seal mark layer image based on the geometric reference coordinates to obtain the effective torque seal mark centerline; and finally generating a loosening state determination result. This invention can accurately identify the effective marking layer of the nut, ensuring the accuracy of the deflection angle and loosening state determination.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and more specifically, to a method and system for automatically reading anti-rotation marks on anti-loosening nuts based on image recognition. Background Technology

[0002] In the operation and maintenance of fastening connections in rail transit, power transmission lines, and industrial steel structures, torque seal marks are usually manually applied to the mating positions of the nuts and adjacent components to enable visual monitoring of nut loosening. These marks are prone to aging, damage, and contamination during long-term outdoor operation. Maintenance personnel often perform a second coating operation without completely removing the old marks. The newly coated marks will form a multi-layered distribution pattern with partial overlap and misalignment with the old marks. This multi-layered marking structure has become a common form in long-term outdoor operation and maintenance scenarios.

[0003] Existing automatic detection technologies for anti-rotation markers based on image recognition mostly extract features of the marked region through image segmentation, edge extraction, and skeletonization. The extracted marker contour or center line is used as the benchmark for calculating the deflection angle and determining the loosening state. These technologies usually rely on single global features such as color threshold and edge gradient to complete the localization and segmentation of the marked region, and directly use the extracted feature results as the basis for effective readings. They do not build a dedicated recognition and processing flow for the image features of multi-layered markers.

[0004] Because the old and new marker layers have inherent differences in coating material, thickness, edge morphology and optical reflection characteristics, the images of multi-layer overlapping areas will exhibit double-layer boundary structures, mixed color distributions, reflection differences caused by thickness steps and different forms of specular reflection features. Existing detection technologies cannot effectively distinguish and separate the old and new marker layers, and can only uniformly process the mixed image features. As a result, it is impossible to determine the marker layer that should be used as the effective benchmark. The selection of the reading benchmark lacks corresponding discrimination criteria and constraint mechanisms.

[0005] Due to the aforementioned technical defects, the detection system may mistakenly use the features of old markers as valid reading benchmarks, resulting in a mismatch between the reading results such as marker deflection angle and integrity status and the actual valid marker status. The output results of image segmentation and skeletonization will fluctuate due to the mutual interference of multiple feature layers. At the same time, the system will perform reading operations on markers with multiple overlapping layers that do not meet the valid reading conditions, and will be unable to output a judgment conclusion on the readability of the markers. This leads to a lack of consistency in the detection data within different inspection cycles, and the judgment results on the loose state of the fastened connection pair cannot provide stable data support for operation and maintenance decisions.

[0006] In view of this, the present invention proposes an automatic reading method and system for anti-loosening nut anti-rotation markings based on image recognition to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art and achieve the above objectives, the present invention provides the following technical solution: an automatic reading method for anti-loosening nut anti-rotation marks based on image recognition, comprising:

[0008] Images from multiple lighting directions are acquired and combined to obtain a multi-light image group;

[0009] Based on the multi-light image group, the intersection line between the outer contour of the nut and the mating surface of the component is located. Geometric reference coordinates are established by the intersection line between the hexagonal vertex and the mating surface of the component. The torque seal mark candidate area image group is then obtained by cropping under the geometric reference coordinates.

[0010] Color similarity, parallel boundaries and specular drift are extracted from the candidate region image group of torque seal markers and fused to obtain a hierarchical classification map. The new torque seal marker layer map and the old torque seal marker layer map are separated from the hierarchical classification map.

[0011] Based on the geometric reference coordinates, the effective torque seal mark centerline is obtained by bridging the cross-edge segment of the new torque seal mark layer diagram, and the zero reading line is determined on the component mating surface intersection line;

[0012] Based on the new torque seal mark layer diagram, the old torque seal mark layer diagram, and the hierarchical classification diagram, the readability conclusion of the torque seal mark is obtained. When the readability conclusion of the torque seal mark is readable, the deflection angle reading of the torque seal mark is obtained from the center line of the effective torque seal mark and the zero reading line, and a loosening state judgment result is generated. When the readability conclusion of the torque seal mark is unreadable, the loosening state judgment result is not judged.

[0013] Furthermore, methods for obtaining multi-illumination image sets include:

[0014] At the inspection location, use a camera to aim at the fastening connection pair and record the straight-line distance between the camera and the fastening connection pair to obtain the shooting distance;

[0015] The lighting sequence of the ring light is set based on the shooting distance to form multiple lighting directions, which are defined by the lighting sequence of the zones.

[0016] Images are sequentially acquired from multiple lighting directions and in the order of partition illumination. During the acquisition process, the acquisition time of each frame is recorded to obtain the shooting time.

[0017] Multi-illumination image groups are obtained by summarizing images from multiple illumination directions according to the lighting order of the partitions.

[0018] Furthermore, an acquisition information table is formed based on the shooting distance and shooting time, and the acquisition information table is aligned with each frame of the multi-illumination image group according to the lighting order of the partitions.

[0019] Furthermore, the method for obtaining the candidate image group of torque seal markers includes:

[0020] The multi-light image group is sorted frame by frame according to the shooting order of the information collection table, and the area occupied by the fastening connector in the multi-light image group is determined according to the shooting distance.

[0021] Candidate boundaries are determined based on the common boundaries of each frame in the multi-light image group. Closed boundaries are formed by circumferential tracking of the candidate boundaries. The outer contour of the nut is determined based on the positional consistency of the closed boundaries.

[0022] Search for intersection lines along the inner side of the nut's outer contour in the region adjacent to the nut's outer contour. Based on the intersection line segments with consistent positions in each frame of the multi-light image group, the intersection line trajectory is obtained by summarizing the intersection line trajectory. The intersection line of the component mating surface is obtained by fitting the intersection line trajectory.

[0023] The boundary direction changes circumferentially along the outer contour of the nut. Based on the position where the boundary direction changes to reach the preset turning condition, the hexagonal vertex is determined. Geometric reference coordinates are established based on the intersection line between the hexagonal vertex and the mating surface of the component.

[0024] By cropping each frame of the multi-illumination image group under geometric reference coordinates, a torque seal marker candidate region image group is obtained.

[0025] Furthermore, the torque seal marker candidate area image group maintains the shooting order of the multi-illumination image group.

[0026] Furthermore, the methods for obtaining the new torque seal mark layer map and the old torque seal mark layer map include:

[0027] The candidate image group of torque seal mark is aligned and organized based on geometric reference coordinates. The acquisition time span is calculated based on the acquisition information table. When the acquisition time span exceeds the acquisition time span threshold, multiple consecutive supplementary lighting direction images in the shooting sequence are taken to obtain the aligned image group.

[0028] A color segmentation map is obtained by dividing continuous regions with similar colors into aligned image groups using a similar color threshold.

[0029] Parallel boundary segments are identified based on the color segmentation map and the parallel boundary spacing threshold. Based on the range of occurrence of parallel boundary segments, a double-layer boundary intensity map is accumulated.

[0030] The high-light reflection region is marked based on the double-layer boundary intensity map and the aligned image group. The high-light drift distance is obtained based on the farthest distance of the high-light reflection region under multiple supplementary lighting directions. The high-light drift map is extracted from the torque seal marker candidate area image group based on the high-light drift threshold.

[0031] Using the color segments of the color segmentation map as the fusion unit, a hierarchical classification map is formed based on preset intensity conditions and specular drift distance. A new torque seal mark layer map is obtained from the upper classification area of ​​the hierarchical classification map, and an old torque seal mark layer map is obtained from the lower classification area of ​​the hierarchical classification map.

[0032] Furthermore, the method for generating the effective torque seal mark center line and determining the zero reading line includes:

[0033] The new torque seal mark layer diagram is mapped onto the geometric reference coordinates, and the edge neighborhood range is obtained by delineating the circumferential interval corresponding to the hexagonal vertex based on the geometric reference coordinates.

[0034] Based on the edge neighborhood range, connected regions are extracted from the new torque seal mark layer graph to obtain the torque seal mark fragment set. The torque seal mark fragment set is retained based on the extension length and the lower limit of the fragment length.

[0035] Based on the torque seal mark fragment set, the fragment orientation and fragment line width are determined in geometric reference coordinates. Based on the direction difference threshold and the line width difference threshold, cross-edge fragment pairs are determined to obtain the bridging candidate pair set.

[0036] Based on the bridging candidate pair set and the bridging distance threshold, the gap region is filled in the edge neighborhood to obtain a continuous torque seal mark zone map;

[0037] The effective torque seal mark centerline is obtained by extracting the center direction from the continuous torque seal mark map. The effective torque seal mark centerline is retained based on the lower limit of the continuous length of the centerline. The zero reading direction is determined based on the intersection line of the component mating surface and the effective torque seal mark centerline. The zero reading line is determined on the intersection line of the component mating surface according to the zero reading direction.

[0038] Furthermore, the position of the component mating surface intersection line in the geometric reference coordinate system is used as the reading reference line, and the direction of the corresponding intersection point of the component mating surface intersection line and the center line of the effective torque seal mark in the candidate area is taken as the reading zero direction.

[0039] Furthermore, methods for obtaining the torque seal mark deflection angle reading include:

[0040] Based on the hierarchical classification map, double-layer boundary zones are identified along the direction of the component mating surface intersection line. The coverage length of the double-layer boundary zone in the direction of the component mating surface intersection line is compared with the total coverage length of the torque seal mark candidate area image group in the direction of the component mating surface intersection line to obtain the double-layer boundary ratio. Based on the segmented boundaries in the hierarchical classification map, the number of color segments is counted. The double-layer boundary ratio and the number of color segments are summarized into an overlap interference index.

[0041] A readable threshold is set based on the collected information table. The readable threshold consists of the upper limit of the proportion of double-layer boundaries and the upper limit of the number of color segments.

[0042] The readability of the torque seal mark was determined by comparing the overlap interference index with the readability threshold.

[0043] When the torque seal mark readability conclusion is readable, the centerline direction of the effective torque seal mark centerline is obtained based on the geometric reference coordinates. The direction of the zero reading line under the geometric reference coordinates is taken as the zero position direction. The angle between the centerline direction and the zero position direction is calculated to obtain the torque seal mark deflection angle reading.

[0044] An image recognition-based automatic reading system for anti-loosening nut anti-rotation markings includes:

[0045] The image acquisition module is used to acquire images from multiple lighting directions and summarize them to obtain a multi-light image group;

[0046] The image cropping module locates the intersection line between the outer contour of the nut and the mating surface of the component based on the multi-light image group. It establishes the geometric reference coordinates by the intersection line between the hexagonal vertex and the mating surface of the component, and cropes the torque seal mark candidate area image group under the geometric reference coordinates.

[0047] The image differentiation module is used to extract color similarity, parallel boundary and specular drift from the image group of torque seal mark candidate area and fuse them to obtain a hierarchical classification map. The hierarchical classification map is used to separate the new torque seal mark layer map and the old torque seal mark layer map.

[0048] The image marking module, based on geometric reference coordinates, bridges the new torque seal mark layer map across edge segments to obtain the effective torque seal mark center line, and determines the zero reading line on the component mating surface intersection line;

[0049] The image determination module obtains the readability conclusion of the torque seal mark based on the new torque seal mark layer map, the old torque seal mark layer map, and the hierarchy map. When the torque seal mark readability conclusion is readable, the torque seal mark deflection angle reading is obtained from the center line of the effective torque seal mark and the zero reading line, and a loosening state determination result is generated. When the torque seal mark readability conclusion is unreadable, the loosening state determination result is not determined.

[0050] Compared with the prior art, the technical effects and advantages of the automatic reading method and system for anti-loosening nut anti-rotation marks based on image recognition of the present invention are as follows:

[0051] This invention forms a multi-light image group by acquiring images from multiple supplementary lighting directions. Based on this image group, the intersection line between the outer contour of the nut and the mating surface of the component is located. Geometric reference coordinates are established using the hexagonal vertices and the intersection line, and the image group of the torque seal mark candidate area is obtained by cropping. The color similarity, parallel boundaries, and specular drift of the candidate area image group are extracted and fused to form a hierarchical classification map, and new and old torque seal mark layer maps are separated. The new mark layer map is bridged with cross-edge segments to obtain the effective mark center line, and the zero reading line is determined at the intersection line of the component mating surface. The readability of the mark is determined by combining the new and old mark layer maps and the hierarchical classification map. If it is readable, the deflection angle is calculated and a loosening state result is generated. If it is unreadable, no determination is made.

[0052] This invention solves the problems in the prior art, such as the inability to effectively distinguish between new and old multi-layered markers, the reliance on a single global feature leading to inaccurate reading benchmarks, the lack of consistency in detection data, and the inability to output marker readability conclusions. It can accurately identify effective marker layers, ensure the accuracy of deflection angle and looseness status determination, keep the detection data consistent across different inspection cycles, provide stable data support for operation and maintenance decisions, and avoid performing invalid reading operations on markers that do not meet the reading conditions. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of an automatic reading system for anti-loosening nut anti-rotation markings based on image recognition, as described in an embodiment of the present invention.

[0054] Figure 2 This is a flowchart of the automatic reading method for anti-rotation marking of anti-loosening nuts based on image recognition in an embodiment of the present invention;

[0055] Figure 3 This is a flowchart illustrating the method for obtaining a new torque seal mark layer map and an old torque seal mark layer map in an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be described in detail, clearly, and completely below with reference to the accompanying drawings. It should be particularly noted that the specific embodiments described below are only for better illustrating and explaining the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and substance of the present invention, those skilled in the art can modify, adjust, or make equivalent substitutions based on the content disclosed in the present invention, and these should all be considered within the scope of protection of the present invention.

[0057] Example 1:

[0058] Please see Figure 1As shown, this embodiment discloses an automatic reading system for anti-loosening nut anti-rotation markings based on image recognition, including an image acquisition module, an image cropping module, an image differentiation module, an image marking module, and an image judgment module. Each module is connected via wired or wireless means to achieve data transmission.

[0059] The image acquisition module is used to acquire images from multiple lighting directions and summarize them to obtain a multi-light image group.

[0060] To acquire images from multiple lighting directions and synthesize them into a multi-light image group, the following process is carried out around the fastened connection pair at the inspection location:

[0061] At the inspection location, use a camera to align with the fastening connection pair. When aligning the camera, maintain the relative position of the camera and the fastening connection pair, ensuring the intersection line of the nut's outer contour and the mating surface of the component is in the same frame. After alignment, acquire and record the shooting distance. The shooting distance is the straight-line distance along the camera's optical axis from the camera's optical center to a reference point on the fastening connection pair's surface. The surface reference point is the surface position corresponding to the midpoint of the nut's outer contour adjacent to the intersection line of the component's mating surface.

[0062] The shooting distance can be obtained through two methods: ranging and reverse estimation. In the ranging method, the camera or ring light integrates a ranging module, which uses either a laser or ultrasonic ranging module. The ranging module aligns with a surface reference point and outputs a ranging reading, which is then used as the shooting distance. In the reverse estimation method, a shooting distance calibration table is pre-established, recording the correspondence between the pixel values ​​of the nut's opposite side dimension and the shooting distance under different shooting distance conditions. At the inspection location, the pixel values ​​of the nut's opposite side dimension are extracted from the distortion-corrected image, and the shooting distance matching these values ​​is retrieved from the shooting distance calibration table. This retrieved shooting distance is then used as the shooting distance.

[0063] The shooting distance is used to constrain the selection of subsequent supplementary lighting directions. The preferred range for the shooting distance is 200 mm to 1200 mm. A shooting distance of 200 mm to 500 mm corresponds to close-range acquisition conditions, while a shooting distance of 500 mm to 1200 mm corresponds to long-range acquisition conditions. The selection rule for the shooting distance is to ensure that the intersection line of the nut's outer contour and the mating surface of the component is fully presented in the image, while preserving the edge details of the area corresponding to the torque seal mark.

[0064] Before acquiring multi-illumination image sets, a camera calibration and scale conversion link is established. This link includes camera intrinsic data, distortion parameters, distortion correction results, and scale factors. The camera intrinsic data and distortion parameters are obtained through calibration using a known calibration board. The calibrated intrinsic data and distortion parameters are stored as a camera calibration table. During inspection location acquisition, the camera calibration table is read, and distortion correction is performed on each frame. Distortion correction takes the camera intrinsic data and distortion parameters as input and outputs a distortion-corrected image while maintaining the same shooting order as the original images.

[0065] The scale factor describes the correspondence between pixel scale and millimeter scale, and is used to convert millimeter quantities such as parallel boundary spacing threshold, bridging distance threshold, specular drift distance, and candidate region width to the pixel domain, or to convert distances measured in the pixel domain to the millimeter domain. The scale factor can be obtained through two methods: a conversion based on the nut's opposite side dimensions and a conversion based on a calibration plate. In the conversion based on the nut's opposite side dimensions, the millimeter value of the nut's opposite side dimensions is obtained beforehand, taking the standard size or calibration-measured size of the corresponding model of the fastening connection pair. The hexagonal vertices are located in the distortion-corrected image, and the positions of the two opposite sides are determined. The pixel value of this nut's opposite side dimension is calculated, and the scale factor is obtained by converting the millimeter value of the nut's opposite side dimension to the pixel value. In the calibration board-based conversion method, the calibration board and the fastening connection pair are positioned at the same shooting distance and within the same or adjacent frames during the inspection. The pixel values ​​corresponding to the known dimensions of the calibration board are read, and the scale factor is calculated by converting the millimeter value of the known dimensions of the calibration board into the pixel values. The scale factor varies with the shooting distance, and is indexed or corrected based on the shooting distance to ensure that the scale factor can be reused under the same shooting distance conditions. The scale factor is then output.

[0066] The shooting distance, shooting time, camera calibration table identifier, and scale factor are encapsulated into an acquisition information table. This acquisition information table is aligned with the frames in the multi-illuminance image group according to the shooting order. The acquisition information table is used to characterize the consistency between the acquisition conditions and conversion conditions corresponding to the multi-illuminance image group, and is used in subsequent steps to convert millimeter thresholds to pixel thresholds or pixel readings to millimeter readings.

[0067] The ring light's illumination sequence is set based on the shooting distance, forming multiple illumination directions. These multiple illumination directions are defined by the ring light's illumination sequence and cover different incident directions around the circumference of the component's mating surface intersection line. The number of illumination directions is selected from 2 to 8, with an optimal range of 3 to 6. The selection rule for illumination directions is 3 to 4 when the shooting distance is 200 mm to 500 mm, and 4 to 6 when the shooting distance is 500 mm to 1200 mm, to compensate for the reduced highlight difference caused by the increased shooting distance. The angle between adjacent illumination directions is 30 degrees to 120 degrees during partitioned illumination. The selection rule for the angle is to prioritize covering the circumferential direction corresponding to the hexagonal vertices to enhance the reflection difference of the multi-layered torque seal mark at the thickness step. The multiple illumination directions and their partitioned illumination sequence are then output.

[0068] Images from multiple lighting directions and in a specific lighting sequence are acquired sequentially, maintaining a constant shooting distance. The camera is kept aligned with the secure connection pair during acquisition. The corresponding zones of the ring light are illuminated sequentially according to the lighting sequence, and one frame is acquired after each zone is illuminated. The images from multiple lighting directions form an ordered sequence based on the lighting order. The capture time is recorded during acquisition, and the capture time is the acquisition time of each frame. The recording granularity is 0.1 to 2 seconds. The selection rule for the capture time is as follows: when the secure connection pair is under outdoor wind-induced disturbance or equipment vibration conditions, the recording granularity is 0.1 to 0.5 seconds; when the secure connection pair is relatively stationary, the recording granularity is 0.5 to 2 seconds. Images from multiple lighting directions and their capture times are output.

[0069] Images from multiple lighting directions are compiled in the order of capture to form a multi-illumination image group. Distortion correction is then performed frame-by-frame on this multi-illumination image group to obtain a distortion-corrected multi-illumination image group. The order of the images from the multiple lighting directions remains unchanged in the distortion-corrected multi-illumination image group. The distortion-corrected multi-illumination image group is then output along with the acquisition information table.

[0070] The image cropping module locates the intersection line between the outer contour of the nut and the mating surface of the component based on the multi-light image group. It establishes the geometric reference coordinates by the intersection line between the hexagonal vertex and the mating surface of the component, and cropes the torque seal mark candidate area image group under the geometric reference coordinates.

[0071] To locate the intersection line between the outer contour of the nut and the mating surface of the component based on a multi-light image set, a geometric reference coordinate system is established by the intersection line between the hexagonal vertex and the mating surface of the component. A candidate image set for the torque seal mark is then cropped under this geometric reference coordinate system. This process can be performed at the inspection location using the following sub-steps:

[0072] Read the distortion-corrected multi-light image set and the acquisition information table. The acquisition information table includes shooting distance, shooting time, camera calibration table identifier, and scale factor. Sort the distortion-corrected multi-light image set frame by frame according to the shooting order in the acquisition information table, so that the frame order of the distortion-corrected multi-light image set corresponds one-to-one with the shooting time. Determine the area occupied by the fastening joint within the distortion-corrected multi-light image set according to the shooting distance, which is between 200 mm and 1200 mm. The area occupied is characterized by the coverage ratio in the width and height directions of the image. The coverage ratio refers to the ratio of the pixel span of the fastening joint's outer range in the corresponding direction to the pixel span of the entire image in the corresponding direction. When the shooting distance is between 200 mm and 500 mm, the area occupied takes a coverage ratio of 0.3 to 0.7; when the shooting distance is between 500 mm and 1200 mm, the area occupied takes a coverage ratio of 0.15 to 0.4. The rule for selecting the coverage ratio is that in the multi-light image group after distortion correction, the outer contour of the nut is completely in the picture, the intersection line of the component mating surface is visible in the area near the outer contour of the nut, and the turning position of the edge of the outer contour of the nut can be distinguished from the background boundary, thus satisfying the constraint condition that the edge details can be distinguished.

[0073] The outer contour of the nut is located based on a set of multi-light images after distortion correction. The common boundary in each frame of the multi-light image set after distortion correction is used as a candidate boundary. The candidate boundary is the continuous boundary between the fastening joint and the background. A closed boundary is formed by circumferentially tracing the candidate boundary. The closed boundary covers the hexagonal vertex positions. The positional consistency of the closed boundary is compared frame by frame in the multi-light image set after distortion correction. Positional consistency is defined as the positional change of the closed boundary relative to the frame in each frame not exceeding a preset offset range. The preferred range of the preset offset range is 0.5% to 3% of the outer range of the closed boundary. The selection rule for the preset offset range is 0.5% to 1.5% when the shooting distance is 200 mm to 500 mm, and 1.5% to 3% when the shooting distance is 500 mm to 1200 mm, to cover the boundary wobble caused by the high-light reflection from the fastening joint surface. The closed boundary that satisfies the positional consistency is determined as the outer contour of the nut.

[0074] Based on the positioning of the nut's outer contour, the component mating surface intersection line is located in the region adjacent to the nut's outer contour. The region adjacent to the nut's outer contour refers to the inner neighborhood extending from the nut's outer contour towards the nut's center. This inner neighborhood is geometrically concentric with the nut's outer contour, and its extent is defined by the radial width of the inner ring. The method for determining the region adjacent to the nut's outer contour is to establish a normal direction pointing towards the nut's center at each boundary point of the nut's outer contour. Extending inward along this normal direction as radially, an inner ring is obtained; the area covered by this inner ring is the region adjacent to the nut's outer contour. In the distortion-corrected multi-light image set, the component mating surface intersection line appears as a continuous intersection line surrounding the nut, and this continuous intersection line has a consistent circumferential relationship with the nut's outer contour.

[0075] Using the outer contour of the nut as the boundary, a ring-shaped search band is constructed inside the outer contour of the nut. The ring-shaped search band extends along the normal direction towards the center of the nut, and its radial width is taken as the width of a preset intersection line search band. The preset intersection line search band width is calculated from the scale factor in the data acquisition table. The preferred range for the preset intersection line search band width is 2 mm to 10 mm. When the shooting distance is 200 mm to 500 mm, the preset intersection line search band width is 2 mm to 5 mm; when the shooting distance is 500 mm to 1200 mm, the preset intersection line search band width is 5 mm to 10 mm. The selection rule for the preset intersection line search band width is to ensure that the ring-shaped search band covers the possible location range of the component mating surface intersection line and avoids the ring-shaped search band extending too deeply inward to include the high-gloss reflection area of ​​the nut surface, thereby reducing the probability of spatial confusion between the component mating surface intersection line and the torque seal mark boundary.

[0076] Intersection segment extraction is performed synchronously within each frame of the multi-illumination image group after distortion correction. Gradient magnitude and gradient direction maps are calculated for the annular search band; gradient magnitude characterizes edge strength, and gradient direction characterizes the edge normal direction. Pixels with gradient magnitudes exceeding an edge strength threshold are marked as edge pixels, the edge strength threshold being adaptively determined by the gradient magnitude distribution within the annular search band. A gradient direction consistency constraint is further applied to the edge pixels, ensuring that the angle between the gradient direction of the candidate intersection segment and the radial direction of the nut's outer contour does not exceed a preset direction deviation threshold, preferably ranging from 20 to 60 degrees. Edge pixels satisfying both the edge strength and direction deviation thresholds are connected in a circumferential adjacent relationship to obtain several candidate intersection segments.

[0077] Within the same frame, the strongest path is tracked for candidate intersection segments. The path tracking is constrained by circumferential continuity, and the cumulative gradient magnitude is used as the path cost. The circumferentially continuous path with the minimum cost is selected as the set of intersection segments for that frame. The set of intersection segments is represented as a sequence of intersection point coordinates sampled along the circumferential direction, and the sequence of intersection point coordinates is mapped to a geometric reference coordinate system.

[0078] Positional consistency filtering is performed across frames in a multi-illumination image group. For sampling positions at the same circumferential angle, the radial position distribution of the intersection point coordinate sequence of each frame is statistically analyzed. Positional consistency is determined when the dispersion of the radial position distribution does not exceed a preset radial offset threshold. The preset radial offset threshold is calculated in the pixel domain from the scale factor in the acquisition information table, and its preferred range is 0.2 mm to 2 mm. The intersection points that satisfy positional consistency are summarized in the circumferential direction to obtain the intersection trajectory. The intersection trajectory is required to have a circumferential coverage of at least 180 degrees. The preferred range for the circumferential coverage is 180 degrees to 330 degrees. The selection rule for the circumferential coverage is 180 degrees to 240 degrees when the intersection line of the component mating surface is obscured by dirt, and 240 degrees to 330 degrees when the visibility of the component mating surface intersection line is high.

[0079] Smoothing fitting is performed on the intersection trajectory to obtain the intersection line of the component mating surface. The smoothing fitting is constrained by the circumferential continuity of the intersection trajectory, at the cost of the sum of squares of the radial distances from the intersection point to the fitted curve, and a penalty constraint is imposed on the second-order change of the curve. The output is the intersection coordinate set of the component mating surface intersection line in the geometric reference coordinate.

[0080] Hexagonal vertices are extracted based on the outer contour of the nut, and geometric reference coordinates are established using the intersection lines of these vertices and the mating surfaces of the components. The boundary direction changes are traversed circumferentially along the outer contour of the nut. The positions where the boundary direction changes to meet preset turning conditions are identified as hexagonal vertices. These preset turning conditions are defined by the amount of directional change in the boundary direction between adjacent segments. The preferred range for this directional change is 30 to 90 degrees. The selection rule for the directional change is 30 to 50 degrees when the chamfer on the outer contour of the nut is small and the boundary is clear, and 50 to 90 degrees when there are reflective gaps or large chamfers on the outer contour of the nut. The hexagonal vertices are numbered sequentially circumferentially. The intersection line of the component mating surfaces is used as the baseline for the geometric reference coordinates. The direction of the vertex with the smallest angle to the intersection line of the component mating surfaces among the hexagonal vertex numbers is used as the circumferential zero-position direction of the geometric reference coordinates. The geometric reference coordinates are determined jointly by the baseline and the circumferential zero-position direction.

[0081] A candidate image set for torque seal marks is obtained by cropping under geometric reference coordinates. The candidate torque seal marks are distributed around the intersection line of the component mating surface. The candidate torque seal marks cover the area adjacent to the outer contour of the nut and the area on both sides of the intersection line of the component mating surface. The width of the candidate torque seal marks is taken as the preset candidate area width. The preferred range of the preset candidate area width is 4 mm to 12 mm when the shooting distance is 200 mm to 500 mm, and 10 mm to 25 mm when the shooting distance is 500 mm to 1200 mm. The selection rule for the preset candidate area width is to ensure that the torque seal marks fall completely within the candidate torque seal marks under multi-layer superposition and misalignment, while avoiding the inclusion of too many unmarked areas. The scale factor in the acquisition information table is read, and the preset candidate area width is converted from the millimeter domain to the pixel domain to obtain the candidate area width pixel value. The candidate torque seal marks are cropped at the same position in each frame of the distortion-corrected multi-light image set according to the geometric reference coordinates to form the candidate torque seal mark image set. The torque seal marker candidate region image group maintains the shooting order of the distortion-corrected multi-illumination image group. Output the torque seal marker candidate region image group.

[0082] The image differentiation module is used to extract color similarity, parallel boundary and specular drift from the candidate image group of torque seal markers and fuse them to obtain a hierarchical classification map. The hierarchical classification map is used to separate the new torque seal marker layer map and the old torque seal marker layer map.

[0083] Please see Figure 3 As shown, to extract color similarity, parallel boundaries, and specular shift from the candidate image group of torque seal markers and fuse them to obtain a hierarchical classification map, the new torque seal marker layer map and the old torque seal marker layer map are separated from the hierarchical classification map. This can be performed at the inspection location according to the following sub-steps:

[0084] Alignment and organization of the torque seal mark candidate area image group are performed based on geometric reference coordinates. The torque seal mark candidate area image group maintains the shooting order of the multi-light image group after distortion correction. The acquisition information table includes shooting distance and shooting time. The shooting time of the first frame and the shooting time of the last frame of the torque seal mark candidate area image group are read according to the acquisition information table to obtain the acquisition time span. The acquisition time span is used to constrain the influence of the position change of the tight connection pair relative to the image in the multi-light image group on the subsequent specular drift extraction. The preferred range of the acquisition time span threshold is 0.3 seconds to 3 seconds. The selection rule for the acquisition time span threshold is 0.3 seconds to 1 second when the shooting distance is 200 mm to 500 mm, and 1 second to 3 seconds when the shooting distance is 500 mm to 1200 mm. When the acquisition time span exceeds the acquisition time span threshold, images under multiple consecutive supplementary lighting directions in the shooting order are taken to form an aligned image group, which is obtained by filtering the torque seal mark candidate area image group. During alignment, the same positional relationship within the candidate area image group of the torque seal mark is defined by the geometric reference coordinates, so that the same hexagonal vertex direction and the intersection line direction of the component mating surface correspond to the same candidate area position in each frame image.

[0085] Color similarity is extracted from candidate image groups based on torque seal markers, generating color segmentation maps and obtaining the main color band distribution. Color similarity is defined as the degree of color consistency within the same frame image for the same candidate region location. To ensure the repeatability of the color similarity threshold, color preprocessing is performed on each frame image of the aligned image group before calculating color similarity. Color preprocessing includes white balance correction and color space conversion. White balance correction uses the white balance gain recorded by the camera or a preset white balance gain as input, performing channel gain correction on the candidate region pixels to maintain a consistent color baseline for the metallic background region under the same lighting conditions. After white balance correction, the candidate region pixels are converted from the camera output color space to the Lab color space.

[0086] A color similarity metric is defined within the Lab color space, taking the magnitude of the color difference between two color vectors. Color consistency is determined based on the condition that the color difference does not exceed a color similarity threshold, with the preferred range for this threshold being 10 to 60. The selection rule for the color similarity threshold is as follows: 10 to 30 for shooting distances of 200 mm to 500 mm, and 30 to 60 for shooting distances of 500 mm to 1200 mm.

[0087] For each frame of the aligned image group, within the candidate region, consecutive regions with similar colors are divided into color segments using a color similarity threshold. During segmentation, the mean color of the seed pixel in the Lab color space is used as the segment reference color. The color difference between adjacent pixels in the candidate region and the segment reference color is calculated; pixels with a color difference not exceeding the color similarity threshold are included in that color segment. Simultaneously, the segment reference color is updated until the color segments no longer expand. All color segments are summarized in the order of capture to form an aligned color segmentation map. The distribution of color segments along the intersection line of the component mating surfaces within the candidate region is statistically analyzed to obtain the main color band distribution.

[0088] Parallel boundaries are extracted from the color segmentation map to generate a double-layer boundary intensity map. Parallel boundaries are defined as two adjacent boundaries with consistent orientation within the candidate region. These parallel boundaries reflect the double-layer boundary structure formed by the local overlap of multiple superimposed torque seal marks. A parallel boundary spacing threshold is set to limit proximity determination. The preferred range for the parallel boundary spacing threshold is 0.2 mm to 2 mm. The selection rule for the parallel boundary spacing threshold is 0.2 mm to 0.8 mm when the preset candidate region width is 4 mm to 12 mm, and 0.8 mm to 2 mm when the preset candidate region width is 10 mm to 25 mm. The scale factor in the acquisition information table is read, and the parallel boundary spacing threshold is converted from the millimeter domain to the pixel domain to obtain the parallel boundary spacing pixel threshold. The boundary orientation of each color segment in the color segmentation map is extracted. The boundaries of adjacent color segments are compared, and boundary segments with a boundary spacing not exceeding the parallel boundary spacing pixel threshold and consistent orientation are identified and recorded as parallel boundary segments. The location range of parallel boundary segments is accumulated in the torque seal mark candidate region image group according to the shooting order to obtain the double-layer boundary intensity map.

[0089] Spectral drift is extracted from candidate region images based on a double-layer boundary intensity map and torque seal markers, generating a spectral drift map. Spectral drift represents the degree to which the position of specular reflection features shifts under multiple illumination directions for the same candidate region. Spectral drift is used to characterize the reflection differences caused by thickness steps.

[0090] For each frame of the candidate image group for torque seal marking, a specular reflection region is marked within the candidate region. The specular reflection region is the set of pixels whose brightness meets the specular determination criteria. These criteria are adaptively determined by the brightness distribution within the candidate region, ensuring that the specular reflection region can be extracted under different lighting directions. The positional description of the specular reflection region is defined using geometric reference coordinates, enabling alignment and comparison of identical candidate region positions across different frames.

[0091] For each frame of the image, the location of the specular reflection feature is determined in geometric reference coordinates. The specular reflection feature location is taken as the brightness-weighted centroid of the specular reflection region in geometric reference coordinates. The brightness-weighted centroid is obtained by weighting the coordinates of each pixel within the specular reflection region according to its brightness value. If there are multiple unconnected specular reflection regions within the same frame, the specular reflection region with the largest brightness-weighted area is taken as the main specular reflection region, and its brightness-weighted centroid is used as the specular reflection feature location. If there are saturated pixels in the specular reflection region, the saturated pixels are removed from their vicinity, and the brightness-weighted centroid is calculated on the remaining specular reflection region. When the remaining specular reflection region is empty after removing saturated pixels, the boundary geometric centroid is calculated using the boundary pixel set of the saturated region, and this boundary geometric centroid is used as the specular reflection feature location.

[0092] After obtaining the specular reflection feature positions corresponding to each supplementary lighting direction, for the same candidate region, calculate the pairwise distances between the specular reflection feature positions under different supplementary lighting directions, and take the maximum value as the specular drift distance pixel value. Read the scale factor in the acquisition information table, convert the specular drift distance pixel value to the millimeter domain, and obtain the specular drift distance.

[0093] A specular drift threshold is set to limit the determination of specular drift. The preferred range for the specular drift threshold is 0.5% to 6% of the width of the torque seal mark candidate area. The selection rule for the specular drift threshold is 0.5% to 3% when the number of supplementary lighting directions is 2 to 4, and 3% to 6% when the number of supplementary lighting directions is 4 to 8. The positions where the specular drift distance exceeds the specular drift threshold are recorded as specular drift areas, and the results are summarized to obtain a specular drift map.

[0094] A hierarchical classification map is obtained by fusing color segmentation maps, double-layer boundary intensity maps, and specular drift maps. The hierarchical classification map is then used to separate new and old torque seal marker layer maps. During fusion, color segments of the color segmentation map are used as fusion units, ensuring each color segment has a defined spatial range in geometric reference coordinates. For each color segment, its corresponding double-layer boundary intensity and specular drift distance are read. The double-layer boundary intensity characterizes whether there are adjacent parallel boundary segments with consistent orientations in the vicinity of the color segment, while the specular drift distance characterizes whether the position of the specular reflection area in the corresponding region of the color segment shifts under multiple supplementary lighting directions. Physically, the double-layer boundary intensity and specular drift distance are complementary; the double-layer boundary intensity reflects the geometric boundary splitting caused by the superposition of old and new layers, while the specular drift distance reflects the reflection differences caused by thickness steps.

[0095] Specular drift distance exhibits several failure scenarios. These include surface contamination leading to the expansion or weakening of the specular reflection area, surface roughness causing the specular reflection area to appear as scattered bright spots, the proximity of the supplementary light incident direction to the camera's line of sight limiting the range of positional changes in the specular reflection area, and discontinuities in the positional description of the specular reflection area during local saturation. When specular drift distance is in a failure scenario, it no longer characterizes the thickness step; instead, the double-layer boundary intensity is used to characterize the extent to which parallel boundary segments appear under multiple supplementary light directions. Therefore, the double-layer boundary intensity retains independent discriminative significance in the fusion rule, while the specular drift distance serves as supplementary evidence for the reflection differences of the thickness step.

[0096] To standardize the discrimination criteria, preset intensity conditions and specular drift thresholds are set. The preset intensity conditions limit the intensity of the double-layer boundary to a level suitable for layer discrimination, while the specular drift threshold limits the specular drift distance to a level suitable for layer discrimination. The preferred range for the preset intensity conditions is that the number of occurrences of parallel boundary segments in the double-layer boundary intensity map under multiple lighting directions accounts for 20% to 70% of the total number of lighting directions. The selection rule for the preset intensity conditions is 20% to 40% when the number of lighting directions is 2 to 4, and 40% to 70% when the number of lighting directions is 4 to 8. The preferred range for the specular drift threshold is 0.5% to 6% of the width of the torque seal mark candidate area. The selection rule for the specular drift threshold is 0.5% to 3% when the number of lighting directions is 2 to 4, and 3% to 6% when the number of lighting directions is 4 to 8.

[0097] Under the above constraints, hierarchical labeling is performed on color segments. When the double-layer boundary intensity reaches a preset intensity condition and the specular drift distance reaches a specular drift threshold, the color segment is labeled as belonging to the upper layer. When the double-layer boundary intensity reaches the preset intensity condition but the specular drift distance does not reach the specular drift threshold, the color segment is labeled as belonging to the lower layer. The same process is repeated for each color segment. The hierarchical labeling results are then summarized to form a hierarchical labeling map. The regions belonging to the upper layer in the hierarchical labeling map are summarized to obtain a new torque seal labeling map, and the regions belonging to the lower layer in the hierarchical labeling map are summarized to obtain an old torque seal labeling map. The hierarchical labeling map, the new torque seal labeling map, and the old torque seal labeling map are then output.

[0098] The image marking module, based on geometric reference coordinates, bridges the new torque seal mark layer image across edge segments to obtain the effective torque seal mark centerline, and determines the zero reading line on the component mating surface intersection line.

[0099] To obtain the effective torque seal mark centerline by bridging the cross-edge segment of the new torque seal mark layer map based on geometric reference coordinates, and to determine the zero reading line on the component mating surface intersection line, the following sub-steps can be performed at the inspection location:

[0100] Read the geometric reference coordinates, component mating surface intersection lines, new torque seal mark layer diagram, and acquisition information table. The geometric reference coordinates are established by the intersection lines of the hexagonal vertices and component mating surfaces. The new torque seal mark layer diagram is obtained by separating it from the hierarchical classification diagram. The acquisition information table includes a scale factor. Map the new torque seal mark layer diagram onto the geometric reference coordinates, keeping the relative position of the new torque seal mark layer diagram and the component mating surface intersection lines unchanged. Delineate the circumferential interval corresponding to the hexagonal vertices according to the geometric reference coordinates, forming the edge neighborhood range. The edge neighborhood range covers both sides of the hexagonal vertices circumferentially along the outer contour of the nut. The width of the edge neighborhood range is a preset edge neighborhood width. The preferred range of the preset edge neighborhood width is 10% to 35% of the width of the torque seal mark candidate area. The selection rule for the preset edge neighborhood width is 10% to 20% when the shooting distance is 200 mm to 500 mm, and 20% to 35% when the shooting distance is 500 mm to 1200 mm.

[0101] The torque seal marker fragment set is extracted from the new torque seal marker layer map based on the edge neighborhood range. The torque seal marker fragment set is taken from the connected regions within the new torque seal marker layer map. The fragment outline boundary is recorded for each connected region to obtain the fragment boundary set. The extension length of the fragment boundary set along the intersection line of the component mating surface is statistically calculated to obtain the fragment length set. A lower limit for the fragment length is set to eliminate isolated noise points. The preferred range of the lower limit for the fragment length is 5% to 25% of the width of the torque seal marker candidate area. The selection rule for the lower limit of the fragment length is 5% to 12% when the shooting distance is 200 mm to 500 mm, and 12% to 25% when the shooting distance is 500 mm to 1200 mm. The scale factor in the acquisition information table is read, and the millimeter value corresponding to the lower limit of the fragment length is converted to the pixel domain to obtain the lower limit of the fragment length in pixels. Connected regions with a length not less than the lower limit of the fragment length in pixels are retained as the torque seal marker fragment set.

[0102] Based on the set of torque seal mark segments, cross-edge segment pairs are determined to form a bridging candidate pair set. For each torque seal mark segment, the segment orientation is determined in geometric reference coordinates. The segment orientation is the extension direction of the segment along the intersection line of the component mating surface. For each torque seal mark segment, the segment linewidth is determined. The segment linewidth is the coverage width of the segment in the direction perpendicular to the intersection line of the component mating surface. Orientation difference thresholds and linewidth difference thresholds are set. The preferred range for the orientation difference threshold is 10 degrees to 35 degrees. The selection rule for the orientation difference threshold is 10 degrees to 20 degrees when the width of the torque seal mark candidate area is 4 mm to 12 mm, and 20 degrees to 35 degrees when the width of the torque seal mark candidate area is 10 mm to 25 mm. The preferred range for the linewidth difference threshold is 10% to 40% of the segment linewidth. The selection rule for the linewidth difference threshold is 10% to 25% when the shooting distance is 200 mm to 500 mm, and 25% to 40% when the shooting distance is 500 mm to 1200 mm. Two torque seal mark segments located on both sides of the same edge neighborhood are compared. The combination of the direction difference not exceeding the direction difference threshold and the line width difference not exceeding the line width difference threshold is recorded as a cross-edge segment pair, and the bridging candidate pair set is obtained by summarizing them.

[0103] Bridging is performed across edge segments based on the bridging candidate pair set to generate a continuous torque seal marker band map. Bridging across edge segments refers to performing gap-filling processing on the broken band region between two corresponding torque seal marker segments within the edge neighborhood. This processing defines the spatial range with geometric reference coordinates, the connected objects with the cross-edge segment pairs, and the fill-in scale with a distance threshold. This restores discontinuous segments caused by edge shading, specular reflection, or coating edge discrepancies to a continuous coverage form, avoiding misjudging cross-edge broken bands as hierarchical features caused by damage or misalignment.

[0104] A bridging distance threshold is set to limit the bridging range. The bridging distance threshold is the upper limit of the closest distance between the two ends of the cross-edge segment pair in the geometric reference coordinates. The preferred range for the bridging distance threshold is 0.3 mm to 3 mm. The selection rule for the bridging distance threshold is 0.3 mm to 1 mm when the shooting distance is 200 mm to 500 mm, and 1 mm to 3 mm when the shooting distance is 500 mm to 1200 mm. The scale factor in the acquisition information table is read, and the bridging distance threshold is converted from the millimeter domain to the pixel domain to obtain the bridging distance pixel threshold. The bridging distance pixel threshold is used for comparison with the closest distance of the cross-edge segment pair.

[0105] For each cross-edge segment pair, calculate the nearest distance between the two segment boundaries in geometric reference coordinates. If the nearest distance does not exceed the bridging distance pixel threshold, then fill the gap region between the two segments. The filling range is limited to the edge neighborhood, and the filled coverage area is adjacent to the boundaries of the two segments. After filling all cross-edge segment pairs that meet the bridging distance pixel threshold, update the new torque seal mark layer map to a continuous torque seal mark strip map.

[0106] The effective torque seal mark centerline is extracted based on the continuous torque seal mark strip map, and the zero-reading line is determined on the component mating surface intersection line. The center direction is extracted from the middle position of the continuous torque seal mark strip map along its coverage area to form the effective torque seal mark centerline. The effective torque seal mark centerline is recorded as a centerline coordinate set in geometric reference coordinates. A lower limit for the continuous length of the centerline is set to constrain the usable range of the centerline coordinate set. The preferred range for the lower limit of the continuous length of the centerline is 15% to 60% of the circumferential length of the torque seal mark candidate area. The selection rule for the lower limit of the continuous length of the centerline is 15% to 30% when the continuous torque seal mark strip map covers a shorter circumferential area, and 30% to 60% when the continuous torque seal mark strip map covers a longer circumferential area. The centerline coordinate set that meets the lower limit of the continuous length of the centerline is retained as the effective torque seal mark centerline. Using the position of the component mating surface intersection line in the geometric reference coordinate system as the reading reference line, the direction of the corresponding intersection point of the component mating surface intersection line and the center line of the effective torque seal mark within the candidate area is taken as the reading zero direction. The reading zero line is then determined on the component mating surface intersection line according to the reading zero direction. The center line of the effective torque seal mark and the reading zero line are then output.

[0107] The image determination module obtains the readability conclusion of the torque seal mark based on the new torque seal mark layer map, the old torque seal mark layer map, and the hierarchy map. When the torque seal mark readability conclusion is readable, the torque seal mark deflection angle reading is obtained from the center line of the effective torque seal mark and the zero reading line, and a loosening state determination result is generated. When the torque seal mark readability conclusion is unreadable, the loosening state determination result is not determined.

[0108] Based on the new torque seal mark layer diagram, the old torque seal mark layer diagram, and the hierarchical classification diagram, the readability conclusion of the torque seal mark is obtained. When the readability conclusion of the torque seal mark is readable, the deflection angle reading of the torque seal mark is obtained from the center line of the effective torque seal mark and the zero reading line, and a loosening state judgment result is generated. When the readability conclusion of the torque seal mark is unreadable, the loosening state judgment result is not judged. The following sub-steps can be performed:

[0109] The overlap interference index is obtained by extracting the proportion of double-layer boundaries and counting the number of color segments based on the hierarchical classification map. The hierarchical classification map includes the regional distribution of upper and lower layer classifications. Along the direction of the component mating surface intersection line, adjacent and oriented banded regions of upper and lower layer classifications are identified in the hierarchical classification map, and these banded regions are denoted as double-layer boundary bands. The coverage length of the double-layer boundary bands in the direction of the component mating surface intersection line is compared with the total coverage length of the torque seal mark candidate area image group in the direction of the component mating surface intersection line to obtain the proportion of double-layer boundaries. Then, the number of color segments is counted in the hierarchical classification map according to the segment boundaries corresponding to color similarity. The number of color segments is taken as the number of segments that can be repeated at the same position in the torque seal mark candidate area image group under the shooting order of the distortion-corrected multi-light image group. The proportion of double-layer boundaries and the number of color segments are summarized into the overlap interference index.

[0110] A readable threshold is set based on the data collection information table. The data collection information table includes the shooting distance and shooting time. The readable threshold consists of the upper limit of the double-layer boundary ratio and the upper limit of the number of color segments, used to limit the overlap interference index to an acceptable range. The preferred range for the upper limit of the double-layer boundary ratio is 5% to 25%. The preferred range for the upper limit of the number of color segments is 2 to 6. The selection rule for the upper limit of the double-layer boundary ratio is 5% to 12% when the shooting distance is 200 mm to 500 mm, and 12% to 25% when the shooting distance is 500 mm to 1200 mm. The selection rule for the upper limit of the number of color segments is 2 to 4 when the number of supplementary lighting directions is 2 to 4, and 4 to 6 when the number of supplementary lighting directions is 4 to 8.

[0111] The readability conclusion of the torque seal mark is obtained by comparing the overlap interference index with the readability threshold. The readability conclusion of the torque seal mark refers to the judgment result given on whether the current torque seal mark has the appearance conditions suitable for automatic reading. The value of the readability conclusion of the torque seal mark includes readable and unreadable. The readability conclusion of the torque seal mark is used to constrain whether to output the torque seal mark deflection angle reading and looseness status judgment result, so as to avoid performing reading operation when multiple layers of stacked torque seal marks overlap, double-layer boundary interference, or mixed color distribution exceeds the preset range, thereby reducing the inconsistency of readings caused by the lack of a basis for the selection of reading reference.

[0112] When comparing, both the percentage of the double-layer boundary and its upper limit are compared, as are the number of color segments and their upper limit. The percentage of the double-layer boundary represents the coverage of the double-layer boundary zone along the intersection line of the component mating surface, while the number of color segments represents the number of color segments corresponding to the segment boundaries within the hierarchical classification diagram. When both the percentage of the double-layer boundary does not exceed its upper limit and the number of color segments does not exceed its upper limit, the readability conclusion of the torque seal mark is considered readable. When either the percentage of the double-layer boundary exceeds its upper limit or the number of color segments exceeds its upper limit, the readability conclusion of the torque seal mark is considered unreadable.

[0113] When the torque seal mark readability conclusion is readable, the torque seal mark deflection angle reading is obtained from the center line of the effective torque seal mark and the zero-reading line. The center line of the effective torque seal mark is obtained by bridging the edge segments of the new torque seal mark layer diagram. The zero-reading line is determined at the intersection line of the component mating surfaces. The center line coordinate set of the effective torque seal mark center line is represented by geometric reference coordinates. The direction of the center line coordinate set near the intersection line of the component mating surfaces is taken as the center line direction, and the direction of the zero-reading line in geometric reference coordinates is taken as the zero-position direction. The angle between the center line direction and the zero-position direction is calculated, and the angle is recorded as the torque seal mark deflection angle reading. The reading resolution of the torque seal mark deflection angle reading is taken from 0.5 degrees to 5 degrees. The selection rule for the reading resolution is 0.5 degrees to 2 degrees when the shooting distance is 200 mm to 500 mm, and 2 degrees to 5 degrees when the shooting distance is 500 mm to 1200 mm.

[0114] The loosening status determination result is output based on the readability conclusion of the torque seal mark. When the readability conclusion of the torque seal mark is readable, a loosening status determination result is generated based on the torque seal mark deflection angle reading and the loosening determination threshold. The preferred range for the loosening determination threshold is 3 to 15 degrees. The selection rule for the loosening determination threshold is 3 to 8 degrees for maintenance scenarios where the allowable angle change of the fastening joint is small, and 8 to 15 degrees for maintenance scenarios where the allowable angle change of the fastening joint is large. When the torque seal mark deflection angle reading does not exceed the loosening determination threshold, the loosening status determination result is "not loose". When the torque seal mark deflection angle reading exceeds the loosening determination threshold, the loosening status determination result is "loose". When the readability conclusion of the torque seal mark is unreadable, the loosening status determination result is "not determined". The loosening status determination result is then output.

[0115] Example 2:

[0116] Please see Figure 2 As shown, this embodiment provides an automatic reading method for anti-rotation markings on anti-loosening nuts based on image recognition, including:

[0117] Images from multiple lighting directions are acquired and combined to obtain a multi-light image group;

[0118] Based on the multi-light image group, the intersection line between the outer contour of the nut and the mating surface of the component is located. Geometric reference coordinates are established by the intersection line between the hexagonal vertex and the mating surface of the component. The torque seal mark candidate area image group is then obtained by cropping under the geometric reference coordinates.

[0119] Color similarity, parallel boundaries and specular drift are extracted from the candidate region image group of torque seal markers and fused to obtain a hierarchical classification map. The new torque seal marker layer map and the old torque seal marker layer map are separated from the hierarchical classification map.

[0120] Based on the geometric reference coordinates, the effective torque seal mark centerline is obtained by bridging the cross-edge segment of the new torque seal mark layer diagram, and the zero reading line is determined on the component mating surface intersection line;

[0121] Based on the new torque seal mark layer diagram, the old torque seal mark layer diagram, and the hierarchical classification diagram, the readability conclusion of the torque seal mark is obtained. When the readability conclusion of the torque seal mark is readable, the deflection angle reading of the torque seal mark is obtained from the center line of the effective torque seal mark and the zero reading line, and a loosening state judgment result is generated. When the readability conclusion of the torque seal mark is unreadable, the loosening state judgment result is not judged.

[0122] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0123] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automatic reading method for anti-loosening nut anti-rotation markings based on image recognition, characterized in that, include: Images from multiple lighting directions are acquired and combined to obtain a multi-light image group; Based on the multi-light image group, the intersection line between the outer contour of the nut and the mating surface of the component is located. Geometric reference coordinates are established by the intersection line between the hexagonal vertex and the mating surface of the component. The torque seal mark candidate area image group is then obtained by cropping under the geometric reference coordinates. Color similarity, parallel boundaries and specular drift are extracted from the candidate region image group of torque seal markers and fused to obtain a hierarchical classification map. The new torque seal marker layer map and the old torque seal marker layer map are separated from the hierarchical classification map. Based on the geometric reference coordinates, the effective torque seal mark centerline is obtained by bridging the cross-edge segment of the new torque seal mark layer diagram, and the zero reading line is determined on the component mating surface intersection line; Based on the new torque seal mark layer diagram, the old torque seal mark layer diagram, and the hierarchical classification diagram, the readability conclusion of the torque seal mark is obtained. When the readability conclusion of the torque seal mark is readable, the deflection angle reading of the torque seal mark is obtained from the center line of the effective torque seal mark and the zero reading line, and the loosening state judgment result is generated. If the readability conclusion of the torque seal mark is unreadable, the loose state determination result is not determined.

2. The method for automatically reading the anti-loosening mark of an anti-rotation nut based on image recognition according to claim 1, characterized in that, Methods for obtaining multi-illumination image groups include: At the inspection location, use a camera to aim at the fastening connection pair and record the straight-line distance between the camera and the fastening connection pair to obtain the shooting distance; The lighting sequence of the ring light is set based on the shooting distance to form multiple lighting directions, which are defined by the lighting sequence of the zones. Images are sequentially acquired from multiple lighting directions and in the order of partition illumination. During the acquisition process, the acquisition time of each frame is recorded to obtain the shooting time. Multi-illumination image groups are obtained by summarizing images from multiple illumination directions according to the lighting order of the partitions.

3. The method for automatically reading the anti-rotation mark of an anti-loosening nut based on image recognition according to claim 2, characterized in that, A data acquisition information table is generated based on the shooting distance and shooting time. The data acquisition information table is then aligned with the frames in the multi-light image group according to the order of the lit-up zones.

4. The automatic reading method for anti-loosening nut anti-rotation mark based on image recognition according to claim 3, characterized in that, Methods for obtaining candidate image groups of torque seal markers include: The multi-light image group is sorted frame by frame according to the shooting order of the information collection table, and the area occupied by the fastening connector in the multi-light image group is determined according to the shooting distance. Candidate boundaries are determined based on the common boundaries of each frame in the multi-light image group. Closed boundaries are formed by circumferential tracking of the candidate boundaries. The outer contour of the nut is determined based on the positional consistency of the closed boundaries. Search for intersection lines along the inner side of the nut's outer contour in the region adjacent to the nut's outer contour. Based on the intersection line segments with consistent positions in each frame of the multi-light image group, the intersection line trajectory is obtained by summarizing the intersection line trajectory. The intersection line of the component mating surface is obtained by fitting the intersection line trajectory. The boundary direction changes circumferentially along the outer contour of the nut. Based on the position where the boundary direction changes to reach the preset turning condition, the hexagonal vertex is determined. Geometric reference coordinates are established based on the intersection line between the hexagonal vertex and the mating surface of the component. By cropping each frame of the multi-illumination image group under geometric reference coordinates, a torque seal marker candidate region image group is obtained.

5. The automatic reading method for anti-loosening nut anti-rotation mark based on image recognition according to claim 4, characterized in that, The torque seal marker candidate area image group maintains the shooting order of the multi-light image group.

6. The method for automatically reading the anti-loosening mark of an anti-rotation nut based on image recognition according to claim 3, characterized in that, Methods for obtaining the new torque seal mark layer map and the old torque seal mark layer map include: The candidate image group of torque seal mark is aligned and organized based on geometric reference coordinates. The acquisition time span is calculated based on the acquisition information table. When the acquisition time span exceeds the acquisition time span threshold, multiple consecutive supplementary lighting direction images in the shooting sequence are taken to obtain the aligned image group. A color segmentation map is obtained by dividing continuous regions with similar colors into aligned image groups using a similar color threshold. Parallel boundary segments are identified based on the color segmentation map and the parallel boundary spacing threshold. Based on the range of occurrence of parallel boundary segments, a double-layer boundary intensity map is accumulated. The high-light reflection region is marked based on the double-layer boundary intensity map and the aligned image group. The high-light drift distance is obtained based on the farthest distance of the high-light reflection region under multiple supplementary lighting directions. The high-light drift map is extracted from the torque seal marker candidate area image group based on the high-light drift threshold. Using the color segments of the color segmentation map as the fusion unit, a hierarchical classification map is formed based on preset intensity conditions and specular drift distance. A new torque seal mark layer map is obtained from the upper classification area of ​​the hierarchical classification map, and an old torque seal mark layer map is obtained from the lower classification area of ​​the hierarchical classification map.

7. The method for automatic reading of anti-loosening nut anti-rotation mark based on image recognition according to claim 1, characterized in that, Methods for generating the effective torque seal mark center line and determining the zero reading line include: The new torque seal mark layer diagram is mapped onto the geometric reference coordinates, and the edge neighborhood range is obtained by delineating the circumferential interval corresponding to the hexagonal vertex based on the geometric reference coordinates. Based on the edge neighborhood range, connected regions are extracted from the new torque seal mark layer graph to obtain the torque seal mark fragment set. The torque seal mark fragment set is retained based on the extension length and the lower limit of the fragment length. Based on the torque seal mark fragment set, the fragment orientation and fragment line width are determined in geometric reference coordinates. Based on the direction difference threshold and the line width difference threshold, cross-edge fragment pairs are determined to obtain the bridging candidate pair set. Based on the bridging candidate pair set and the bridging distance threshold, the gap region is filled in the edge neighborhood to obtain a continuous torque seal mark zone map; The effective torque seal mark centerline is obtained by extracting the center direction from the continuous torque seal mark map. The effective torque seal mark centerline is retained based on the lower limit of the continuous length of the centerline. The zero reading direction is determined based on the intersection line of the component mating surface and the effective torque seal mark centerline. The zero reading line is determined on the intersection line of the component mating surface according to the zero reading direction.

8. The method for automatically reading the anti-rotation mark of an anti-loosening nut based on image recognition according to claim 7, characterized in that, The position of the component mating surface intersection line in the geometric reference coordinate system is used as the reading reference line, and the direction of the corresponding intersection point of the component mating surface intersection line and the center line of the effective torque seal mark in the candidate area is taken as the reading zero direction.

9. The method for automatically reading the anti-loosening mark of an anti-rotation nut based on image recognition according to claim 3, characterized in that, Methods for obtaining the torque seal mark deflection angle reading include: Based on the hierarchical classification map, double-layer boundary zones are identified along the direction of the component mating surface intersection line. The coverage length of the double-layer boundary zone in the direction of the component mating surface intersection line is compared with the total coverage length of the torque seal mark candidate area image group in the direction of the component mating surface intersection line to obtain the double-layer boundary ratio. Based on the segmented boundaries in the hierarchical classification map, the number of color segments is counted. The double-layer boundary ratio and the number of color segments are summarized into an overlap interference index. A readable threshold is set based on the collected information table. The readable threshold consists of the upper limit of the proportion of double-layer boundaries and the upper limit of the number of color segments. The readability of the torque seal mark was determined by comparing the overlap interference index with the readability threshold. When the torque seal mark readability conclusion is readable, the centerline direction of the effective torque seal mark centerline is obtained based on the geometric reference coordinates. The direction of the zero reading line under the geometric reference coordinates is taken as the zero position direction. The angle between the centerline direction and the zero position direction is calculated to obtain the torque seal mark deflection angle reading.

10. An automatic reading system for anti-loosening nut anti-rotation markings based on image recognition, used to implement the automatic reading method for anti-loosening nut anti-rotation markings based on image recognition as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire images from multiple lighting directions and summarize them to obtain a multi-light image group; The image cropping module locates the intersection line between the outer contour of the nut and the mating surface of the component based on the multi-light image group. It establishes the geometric reference coordinates by the intersection line between the hexagonal vertex and the mating surface of the component, and cropes the torque seal mark candidate area image group under the geometric reference coordinates. The image differentiation module is used to extract color similarity, parallel boundary and specular drift from the image group of torque seal mark candidate area and fuse them to obtain a hierarchical classification map. The hierarchical classification map is used to separate the new torque seal mark layer map and the old torque seal mark layer map. The image marking module, based on geometric reference coordinates, bridges the new torque seal mark layer map across edge segments to obtain the effective torque seal mark center line, and determines the zero reading line on the component mating surface intersection line; The image determination module obtains the readability conclusion of the torque seal mark based on the new torque seal mark layer map, the old torque seal mark layer map and the hierarchy map. When the readability conclusion of the torque seal mark is readable, the deflection angle reading of the torque seal mark is obtained from the center line of the effective torque seal mark and the zero reading line, and the loosening state determination result is generated. If the readability conclusion of the torque seal mark is unreadable, the loose state determination result is not determined.

Images