Metal key spring punching forming defect on-line vision detection system

Abstract

The application discloses a metal key spring punching forming defect online visual detection system and relates to machine visual detection. The system comprises an image acquisition control module, a contour positioning registration module, an arch surface reflection reference generation module, an edge boundary separation module, a defect residual generation module and a detection result output module. Through acquiring multi-directional light images of the same spring in the same trigger cycle, an arch surface normal reflection model is established according to the left-right oblique mirror image symmetry, and an edge reference area model is established according to the contrast between the backlight boundary and the shadow boundary. Then, a defect residual area is generated and the defect type is judged. The above technical scheme separates the normal imaging signal generated by the spring itself from the defect signal, can distinguish the normal appearance light and shadow from the real defect signal, reduces the misjudgment caused by normal reflection and shadow, and improves the efficiency of the metal key spring punching forming defect detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and in particular to an online visual inspection system for forming defects in metal button spring stamping. Background Technology

[0002] Surface defects in metal stampings are influenced by a combination of factors, such as mold condition, material properties, and real-time fluctuations in process parameters. These factors change frequently during continuous production, making accurate and efficient detection of surface defects a critical aspect of production line quality control. In traditional two-dimensional grayscale image inspection, even with multi-angle light sources and polarized light to suppress specular reflection interference, normal textures, reflections, or shadows on the workpiece surface can easily appear with real defects such as linear grayscale anomalies, planar closed grayscale anomalies, missing material, or burrs, exhibiting extremely similar grayscale characteristics in the two-dimensional grayscale image. This leads to numerous misjudgments in the inspection results. This indicates that relying solely on two-dimensional grayscale information obtained from a single acquisition is insufficient to construct an effective intelligent sensing system; therefore, multimodal perception and information fusion are urgently needed to improve the accuracy of defect identification.

[0003] To address the aforementioned misjudgment issues, some existing technologies attempt to introduce three-dimensional topographic information to assist in defect identification, i.e., by fusing multiple sensing methods to construct a more comprehensive intelligent sensing system. For example, Chinese invention patent CN111624206B discloses a method for detecting metal surface defects based on stereo vision using a linear array camera. This method acquires metal surface images from different perspectives using multiple linear array cameras, and calculates the depth information, motion speed, grayscale correspondence, and normal angle of the metal surface under test. Finally, the three-dimensional information is fused with the two-dimensional grayscale image to detect metal surface defects. The main purpose of this method is to improve the accuracy and reliability of metal surface defect detection by solving the problem that traditional two-dimensional image detection struggles to distinguish between real and false defects.

[0004] However, the aforementioned stereoscopic vision solutions are mainly designed for relatively flat metal sheets or regular surface structures, and their applicability to micro-stamped parts with complex three-dimensional curvature features is significantly limited. Metal button springs are typical micro-metal elastic elements manufactured through precision stamping processes. They are only a few millimeters in size, have a centrally arched arc structure, and their outlines are mostly mirror-symmetrical. Their quality directly affects the button's tactile feedback, conductivity reliability, and lifespan; therefore, detecting their appearance defects is crucial.

[0005] In the actual inspection of metal button springs, all springs must be pre-positioned with their arched surfaces facing upwards to ensure they enter the imaging station in a consistent direction and position. An industrial camera then captures images, and image processing algorithms automatically identify defects. However, due to inherent fluctuations in the stamping process, subtle differences in the curvature of the arched surfaces, edge fit, and burr support between individual springs cause variations in the reflection angle, shadow boundaries, and focal plane position of local areas relative to the camera and light source. These variations result in highlights, shadows, or edge ghosting in the morphology, which overlap with actual stamping defect signals such as linear grayscale anomalies, planar closed grayscale anomalies, missing material, or burrs on the two-dimensional grayscale image. These overlaps are difficult to distinguish, severely impacting the accurate location of defect areas and the accurate evaluation of defect characteristics. Therefore, for metal button springs with mirror-symmetrical outlines, existing technologies still struggle to effectively distinguish between normal imaging signals and defect signals. Summary of the Invention

[0006] This invention provides an online visual inspection system for stamping defects in metal button springs. Addressing the problem that normal imaging signals and actual defects are highly similar in image grayscale features, making direct differentiation through conventional grayscale methods is difficult, the system acquires multi-directional illumination images of the same spring within the same trigger cycle. It then establishes a normal reflection model of the arched surface based on the mirror symmetry of left and right oblique light, and an edge reference region model based on the comparison between the backlight boundary and the shadow boundary. Finally, it generates a defect residual region and determines the defect type, thus separating the normal imaging signal from the defect signal and avoiding misjudging the normal imaging signal as a defect or missing the defect altogether.

[0007] According to the present invention, an online visual inspection system for forming defects in metal button spring stamping is provided, comprising:

[0008] The image acquisition control module is used to control the light sources to light up in a fixed sequence and to enable the camera to acquire multi-directional illumination images within the same trigger cycle; the multi-directional illumination images include front lighting images, backlighting images, and oblique lighting images generated by a pair of mirror-symmetrically arranged light sources;

[0009] The contour localization and registration module is used to extract the contour of the metal button spring with a mirror-symmetric geometric shape, establish a unified spring coordinate system to register multi-directional illumination images, and divide the metal button spring into edge detection zone and arch detection area according to the contour.

[0010] The arch surface reflection reference generation module is used to calculate the mirror residual of each pixel in the arch surface detection area based on the mirror symmetry relationship of the oblique light image. The part of the mirror residual exceeding the mirror residual threshold is marked as the arch surface to be judged area, and the part not exceeding the mirror residual threshold is marked as the arch surface reference area.

[0011] The edge boundary separation module is used to calculate the edge shadow width within the edge detection zone; it marks the area where the boundary abruptness of the metal button spring exceeds the boundary abruptness threshold as the edge to be judged area; and it marks the area where the boundary abruptness does not exceed the boundary abruptness threshold and the edge shadow width does not exceed the upper limit of the edge shadow width as the edge reference area.

[0012] The defect residual generation module is used to remove connected regions with continuously varying gray levels from the area to be judged on the arch surface, and merge the remaining regions to generate the defect residual region.

[0013] The detection result output module is used to output the detection result of the metal button spring based on the defect residual area: if either the image acquisition validity and registration validity verification fails, the result is output as pending re-inspection; if no abnormality occurs in the detection process and the defect residual area is empty, the result is output as OK; otherwise, the result is output as NG and the corresponding defect type is output.

[0014] The beneficial effects achieved by the technical solution provided in the embodiments of the present invention include at least the following: separating the normal imaging signal generated by the morphology of the shrapnel itself from the defect signal, avoiding the misjudgment of normal reflection and shadow as defects or the missed detection of real defects in conventional grayscale detection, and being able to distinguish between normal morphology light and shadow and real defect signals, which helps to reduce misjudgment caused by normal reflection and shadow.

[0015] It eliminates the need for pixel-by-pixel comparison with a fixed template, establishing a normal reflection reference based on the left-right mirror symmetry of the current spring piece. It is adaptable to differences in arc height and curvature between individual spring pieces, facilitating deployment and maintenance on the production line and effectively reducing the debugging and changeover costs of the detection system.

[0016] By dividing the spring sheet into a differentiated detection area for the arched surface and a detection zone for the edge, the arched surface defects and the edge defects can each adopt their own suitable detection strategies, thus avoiding mutual interference between different types of defect signals and improving the targeting and accuracy of defect detection.

[0017] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is an architecture diagram of an online visual inspection system provided according to an embodiment of the present invention;

[0020] Figure 2 This is a diagram showing the arrangement and timing of light sources for multi-light timing acquisition according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of spring registration and contour detection region division according to an embodiment of the present invention;

[0022] Figure 4 This is a diagram of oblique light mirror correspondence and mirror residual threshold discrimination provided by an embodiment of the present invention; Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, system, product, or device that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, systems, products, or devices.

[0025] Example 1: This example provides an online visual inspection system for forming defects in metal button springs. It is suitable for online intelligent inspection scenarios where mirror-symmetrical metal button springs have been coarsely positioned with their arched surfaces facing upwards. The upward arching is achieved by a feeding mechanism such as a vibratory feeder. Before entering the visual inspection system, the metal button springs undergo rough screening in terms of appearance and size by pre-production processes. This screening removes springs with visually identifiable defects such as obvious deformation, large-area crushing, or annular / surface-shaped closed grayscale anomalies, as well as springs exceeding dimensional tolerances. Only springs that have passed rough screening can enter this visual inspection system for fine inspection. After being arranged by the feeding mechanism, the metal button springs enter the flow carrier with their arched surfaces facing upwards in the same orientation. Subsequently, a transfer component moves the springs from the flow carrier to the inspection position, placing them within the camera's field of view. This process is called coarse positioning in industrial inspection. The spring only needs to be within the camera's field of view at the detection position, without the need for micron-level precise positioning through mechanical mechanisms; the precise position and angle deviation of the spring in the image are eliminated by the subsequent contour positioning and registration module through image processing.

[0026] The online visual inspection system has a detection position and is equipped with an industrial camera, a front light source, left and right oblique light sources, a backlight source, and a trigger unit; the system also includes an image processing unit. The detection position holds the spring to be inspected; the industrial camera is positioned directly above the detection position to capture images of the spring; the image processing unit is connected to the industrial camera to receive and process the captured images; the trigger unit triggers each light source to illuminate sequentially according to a preset order and synchronizes the camera exposure. The left and right oblique light sources are respectively positioned on the left and right sides of the detection position, arranged in a mirror-symmetric manner relative to the spring, to form mutually verifiable oblique light images. The left and right oblique light sources are arranged at a preset angle to the surface of the spring; this preset angle can be adjusted according to the spring size, surface curvature, and inspection accuracy requirements. Experiments have verified that when the camera's optical axis is arranged at a small angle to the normal to the spring surface, i.e., the left and right oblique light sources are within an angle range of 30° to 50° with the normal, a superior mirror-symmetric reflection effect can be achieved. The specific value of the incident angle can be adjusted according to actual inspection needs and image quality, and is not limited to a fixed angle. A backlight source is positioned below the detection position, with light passing through it from bottom to top to highlight the backlit boundary of the spring. The detection position uses a transparent dielectric layer, such as a glass plate or transparent acrylic plate, to support the spring. The light emitted from the backlight source passes through the transparent dielectric layer from bottom to top, forming an image of the backlit boundary of the spring in the camera. To reduce specular reflection interference from the surface of the transparent dielectric layer, a polarized plate can be used, or polarization components can be configured in the optical path between the light source and the camera. The acquisition sequence is fixed as: front lighting image, left oblique lighting image, right oblique lighting image, and backlight image. This arrangement ensures that all four images are acquired by the same camera with the spring position unchanged, sharing the same camera coordinate basis. The direct effect of the same position and fixed order is that subsequent differences can be attributed to the illumination state and the spring's imaging response, without introducing additional coordinate changes due to changing the camera or moving the spring.

[0027] like Figure 2 The diagram shown illustrates the light source arrangement and acquisition timing for multi-illumination timing data acquisition. Figure 2 (a) is a schematic diagram of the lateral spatial arrangement. An industrial camera is positioned directly above the detection position to capture images of the spring sheet. A coaxial positive light source is located on the same side of the camera, illuminating the surface of the spring sheet perpendicularly along the camera's optical axis to reveal linear or planar closed grayscale anomalies on the spring sheet surface. Left and right oblique light sources are positioned on the left and right sides of the detection position, respectively, arranged in a mirror-symmetrical manner relative to the spring sheet, to form mutually verifiable oblique light images. A backlight source is positioned below the detection position, with light passing through the detection position from bottom to top to highlight the backlight boundary of the spring sheet. The detection position supports the spring sheet, and the light emitted from the backlight source passes through the transparent dielectric layer from bottom to top, forming the backlight boundary image of the spring sheet in the camera. Figure 2(b) is a relative timing diagram. The four horizontal tracks correspond to four acquisition states: front lighting, left oblique lighting, right oblique lighting, and backlighting. The black dots on the tracks indicate that the camera has completed one image acquisition at that moment. The acquisition order is fixed as front lighting image, left oblique lighting image, right oblique lighting image, and backlighting image. The length of the colored line segments is only used to identify the state and does not represent equal exposure time.

[0028] In online visual inspection of metal button springs, even if the spring has its arched surface facing upwards and has completed coarse positioning, the curvature of the arched surface, the difference in arc height, the local overhang of the edge, and the support of burrs will still change the local reflection angle, shadow width, and focal plane position. This causes the highlights, shadows, or virtual boundaries generated by the normal forming structure to be superimposed on the real defects in the image. The real defects include linear gray-scale anomalies, planar closed gray-scale anomalies, missing material, and burrs. This embodiment does not directly treat gray-scale abrupt changes as defects. Instead, it acquires multi-directional illumination images of the same spring within the same detection trigger. First, it forms a normal imaging reference for the current spring, that is, it uses the mirror symmetry relationship of the left and right oblique light images to determine the arched surface reference area, and uses the comparison between the backlight boundary and the front light or left and right oblique light shadow boundary to determine the edge reference area. Then, it retains the remaining signal that cannot be interpreted by the normal arched surface reflection or the edge reference area as the defect residual area, and outputs the defect type accordingly.

[0029] In this embodiment, the shadow refers to the area of ​​light and dark variation formed on the surface of the spring sheet due to the obstruction or reflection of light by its own micro-geometry, such as the outer contour edge, the punched contour edge, the burr support, and the transition of the arched surface, rather than the shadow projected from the outside of the spring sheet onto other surfaces. This area of ​​light and dark variation can be directly imaged by an industrial camera under direct or oblique lighting and is adjacent to the surface of the spring sheet material.

[0030] At the start of a detection trigger cycle, the trigger unit confirms that the spring is stationary at the same detection position, and that the relative positions between the camera, light source, and spring remain unchanged during this acquisition process. The image processing unit reads the illumination orientation group corresponding to the spring of the current specification. The illumination orientation group is used to specify the type of light source and the lighting order participating in the acquisition.

[0031] like Figure 1 The diagram shows the architecture of an online visual inspection system provided according to an embodiment of the present invention. The inspection process includes an image acquisition and control module, a contour positioning and registration module, an arched surface reflection reference generation module, an edge boundary separation module, a defect residual generation module, and an inspection result output module, with each module executed in series.

[0032] Before image acquisition, the image processing unit performs dark field correction and flat field correction on the industrial camera, and pre-calibrates the illuminance of the front light source, back light source, and left and right oblique light sources to ensure that the average gray value of qualified samples under illumination by each light source within the same trigger cycle is within the same gray range. The pre-calibration is performed during system installation and debugging, and is repeated after the light source is changed or the camera exposure parameters are adjusted.

[0033] Preferably, before performing mirror grayscale comparison, flat-field correction and left and right light source brightness normalization are performed on the left and right oblique lighting images to eliminate the influence of light source attenuation, lens vignetting, and sensor response differences on the mirror residual. Flat-field correction obtains correction coefficients by acquiring dark field images and uniform white board images; left and right light source brightness normalization normalizes the grayscale values ​​of the left and right oblique lighting images to the same brightness reference by acquiring the left and right illumination responses of a standard reflector.

[0034] The image acquisition control module illuminates the front light source, left oblique light source, right oblique light source and back light source in a fixed order within the same trigger cycle when the spring is stationary at the same detection position. The same camera then acquires multi-directional illumination images in a fixed order. In this embodiment, the multi-directional images include front light image, left oblique light image, right oblique light image and back light image.

[0035] If any of the four images in the same trigger cycle fails to be acquired, or if the four images are not acquired when the same spring is at the same detection position, the image processing unit will not perform the defect determination and will output an invalid image acquisition result; the spring will be marked as pending re-inspection to avoid outputting an OK result when the image is incomplete.

[0036] The contour positioning and registration module involves the image processing unit performing grayscale segmentation on the backlight image to obtain the spring sheet material region and extracting the contour; the contour includes the outer contour and the punching contour. For the outer contour, the image processing unit determines the center of the outer contour based on the circumscribed boundary or boundary point set of the material region in the backlight image, and determines the spring sheet plane direction based on the principal axis direction of the outer contour. The principal axis direction is obtained by performing principal component analysis on the boundary point set on the outer contour: using the outer contour center as the origin, constructing the planar coordinate vectors of each boundary point, calculating the covariance matrix of all boundary point coordinate vectors, performing eigenvalue decomposition on the covariance matrix, and taking the direction of the eigenvector corresponding to the largest eigenvalue as the first principal component direction of the outer contour, i.e., the principal axis direction. For an outer contour that is close to a circle, its principal axis direction is determined by the stamping feeding direction. Using the outer contour center as the origin of the spring sheet coordinate system and the principal direction of the outer contour as the first coordinate axis of the spring sheet coordinate system, the current spring sheet coordinate system is established.

[0037] Based on this, the positive direction of each contour is further determined. For the outer contour, the radial outward direction is the positive direction; for the punching contour, the radial inward direction is the positive direction. Under this convention, movement of the backlight boundary in the negative direction is judged as material shortage, and movement in the positive direction is judged as burr.

[0038] For the frontal light image, left oblique light image, and right oblique light image, the image processing unit transforms the pixel positions in the three images to the same spring coordinate system based on the outer contour center and main direction obtained from the backlight image, thus obtaining the registered frontal light image, left oblique light image, and right oblique light image. The registration operation is only used to eliminate the in-plane positional and angular deviations of the spring in the detection position, and does not change the actual source of the grayscale signal in each image.

[0039] If the backlight image cannot segment the effective material area, or cannot obtain the center and main direction of the outer contour of the spring, the image processing unit outputs an invalid contour positioning result and stops the current defect classification of the spring.

[0040] If the punch outline is not detected, but a punch should exist in the standard structure of the current specification spring, the image processing unit outputs an abnormal punch outline and indicates that it needs to be re-inspected. If the current specification spring itself does not have a punch, the generation of the edge detection band related to the punch outline is not performed. If a mirrored pixel falls outside the punch area, the edge detection band, or the effective range of the image, the corresponding pixel will not participate in the arch surface mirror residual judgment. If the proportion of invalid mirrored pixels in the arch surface detection area exceeds the preset allowable proportion, the image processing unit outputs an invalid arch surface reflection reference result and marks the current spring as needing to be re-inspected.

[0041] The image processing unit takes preset distances relative to the inner side of the spring and the outer side of the material along the outer contour of the spring and the punch contour, respectively. These preset distances constitute the edge detection band. The sum of the preset distances is the width of the edge detection band, which is used to cover the outer contour and the punch contour, as well as areas where there may be missing material or burrs. The width of the edge detection band can be pre-calibrated according to the spring specifications and detection accuracy requirements, in pixels.

[0042] During image processing, all length parameters are calculated and compared in pixels, without physical size conversion, to ensure processing speed. Area parameters are judged in the image processing stage using pixel area as the unit, without conversion. When the detection result is output, length parameters are converted to millimeters by multiplying by the camera calibration scale K. , The physical dimensions of the known reference object are in millimeters. This represents the pixel size of the reference object in the image, in pixels. Area parameters are converted to square millimeters by multiplying by K². This conversion only applies to the output of the detection results and does not affect the pixel-level accuracy of threshold judgments during image processing.

[0043] The surface area of ​​the spring sheet material, excluding the edge detection zone and the blank area of ​​the punched hole, is designated as the arched surface detection zone. Both the edge detection zone and the arched surface detection zone are located within the current spring sheet coordinate system, and subsequent calculations regarding the location, length, width, and area of ​​the region are based on this spring sheet coordinate system.

[0044] In this embodiment, the outer contour of the spring sheet and the punch contour are extracted through grayscale segmentation of the backlight image, and the plane direction of the spring sheet is determined by the main direction of the outer contour. The edge boundary separation module scans only along the contour normal direction within the edge detection zone; the defect nature determination and detection result output steps distinguish between edge-type anomalies and arch-type anomalies based on whether the defect residual area contacts the edge detection zone. The normal direction refers to the direction perpendicular to the contour tangent. For the position point corresponding to the current scan line on the spring sheet contour, the image processing unit calculates the tangent direction using the difference method based on the adjacent boundary points on the contour, and then obtains the normal direction based on the 90° perpendicular line of the tangent direction. The normal direction is determined point by point along the circumference of the contour, and the normal direction of each scan line is calculated independently based on the actual position of the scan line on the contour.

[0045] like Figure 3 This is a schematic diagram of the spring registration and contour detection region division provided in an embodiment of the present invention. Figure 3 In (a), the dark blue solid line represents the outer contour and punching contour of the spring piece; the outer circle represents the outer contour, and the inner white circle represents the punching area, the contour of which is the punching contour. Point O represents the center of the spring piece determined by the center of the outer contour, and the horizontal arrow represents the plane direction of the spring piece determined by the main direction of the outer contour; the orange vector and the green arc represent the radial position r and the angular position θ with the center of the spring piece as the reference, respectively. The blue diagonal ring represents the edge detection zone formed by taking preset distances inward and outward along the outer contour and punching contour; the light yellow inner area, excluding the edge detection zone and the blank area of ​​the punching, is the arch surface detection area. Figure 3 (b) Expand the local geometry along the contour normal. The bidirectional arrows on both sides of the contour line represent the preset distances relative to the inner side of the spring and the outer side of the material, respectively. The total width of the preset distances is the width of the edge detection band. Figure 3 (c) Solid lines, long dashed lines, and short dashed lines represent the registered orthogonal light image, left oblique light image, and right oblique light image, respectively. All three share the same spring center O, principal direction, and contour reference, indicating that all surface illumination images have been transformed into the same spring coordinate system. After unifying the center, principal direction, and contour reference, the arch surface reflection reference generation module can establish the current spring's reflection reference within the arch surface detection area according to the radial position r and angular position θ. Through this area division, edge shadows will not be mixed with the arch surface grayscale changes in the same spatial aperture, and the same position in the three images can be consistently referenced in subsequent steps. It should be noted that... Figure 3The illustrated spring is circular, but this is merely an exemplary illustration of the present invention. In practical applications, the outer contour shape of the mirror-symmetrical metal button spring is not limited to a circle; it can also be oval, elliptical, cross-shaped, or square, among other shapes. The punched contour can also be set to a circle, square, or irregular shape according to design requirements. The coordinate system of the present invention is equally applicable to all mirror-symmetrical metal button springs.

[0046] The arch surface reflection reference generation module, within the arch surface detection area, uses the center of the spring sheet as a reference and converts the position of each arch surface pixel into radial and angular positions. The radial position represents the distance from the pixel to the center of the spring sheet, discretized in integer pixel units; that is, all pixels with the same integer pixel radius from the center of the spring sheet constitute the same radial position. The angular position represents the direction of the pixel relative to the first coordinate axis of the spring sheet coordinate system. The image processing unit, within a 90° angular interval on either side of the oblique light incident direction, takes the 90th percentile gray value of all pixels at each radial position within that angular interval as the gray-scale statistical feature of that radial position under that oblique light direction, obtaining the left and right oblique light radial gray-scale distributions for each radial position. For each radial position, only effective material pixels within the arch surface detection area are used in the statistics; pixels in the perforated area and edge detection zone are not included in the arch surface highlight statistics. If the local maximum value of the 90th percentile grayscale within the angular interval and the grayscale difference between the adjacent radial position are less than the preset minimum peak height threshold, the current highlight band positioning of the spring sheet under the oblique light direction is deemed invalid, and the spring sheet is marked for re-inspection. The local maximum value is the grayscale statistical value at a certain radial position, which is simultaneously greater than the grayscale statistical values ​​of the preceding and following radial positions. If multiple local maximum values ​​exist, the local maximum value with the largest 90th percentile grayscale value is selected as the main peak; if multiple local maximum values ​​have the same grayscale value, the local maximum value closest to the center of the spring sheet is selected as the main peak. The preset minimum peak height threshold is determined by the following method: at least 30 qualified samples that have been manually confirmed to be defect-free are collected. Under the same lighting conditions as online inspection, the grayscale difference between the highlight peak position and the adjacent radial position of each qualified sample is calculated according to the above angular interval and 90th percentile grayscale statistical method. The 5th percentile value of this grayscale difference among all samples is taken as the preset minimum peak height threshold.

[0047] Because of differences in the curvature and arc height of the arched surfaces between individual shrapnel pieces, the actual radial position of the highlight band on the arched surface of the shrapnel will shift as a whole with changes in the shrapnel's own shape. For the same shrapnel, when illuminated by left and right oblique light respectively, the position of the highlight band in the two images may be slightly different. This difference is not a defect, but a normal reflection of the individual shrapnel shape. Therefore, directly comparing mirror pixels in the left and right oblique light images pixel by pixel with the center of the shrapnel as a reference will be affected by this highlight band position shift, causing normal arched surface areas to be misjudged as abnormal.

[0048] To eliminate the aforementioned interference, this embodiment first locates the radial position of the highlight band of the current piece under left-side oblique light and right-side oblique light. This radial position of the highlight band is determined by the highlight feature position, which is a feature point in the radial grayscale distribution that reflects the center of the highlight area of ​​the arched surface. The average of the two is taken as the radial position of the piece in the currently detected highlight band, and the left and right radial grayscale distributions are radially aligned according to this position.

[0049] Let the radial position of the current highlight band of the shrapnel be... The radial position of the original highlight band in the left oblique light image is The radial position of the original highlight band in the right oblique light image is The radial offset of the left-side oblique light image is... The radial offset of the right oblique light image is .

[0050] The image processing unit performs two-dimensional coordinate transformations on the left-side oblique light image and the right-side oblique light image respectively: each pixel in the left-side oblique light image is translated radially by ΔL pixels, and each pixel in the right-side oblique light image is translated radially by ΔR pixels; after translation, the radial positions of the highlight bands on the arched surface in both images coincide. The non-integer pixel positions generated after translation are obtained using bilinear interpolation to obtain grayscale values, and the areas outside the image boundaries are filled with zeros. The image processing unit synchronously generates a validity mask for each transformed image, marking the zero-filled areas outside the image boundaries as invalid pixels. This mask corresponds one-to-one with the transformed image.

[0051] When performing a mirror comparison, the image processing unit determines the position of the mirror pixels on the image after radial alignment transformation.

[0052] The radial position of the highlight band reflects the true radial distribution of the highlight area on the arch surface of the current spring sheet, and is used to compensate for the radial offset of the highlight band caused by the difference in the curvature and arc height of the arch surface between different spring sheets. Subsequently, based on this position, the radial gray distribution of the left oblique light and the radial gray distribution of the right oblique light are radially translated and aligned so that the subsequent mirror residual comparison is carried out under the same radial reference and is not affected by the individual differences of the spring sheets.

[0053] The image processing unit identifies the radial positions where the 90th percentile grayscale value reaches a local maximum in both the left-side and right-side oblique light radial grayscale distributions, and uses these positions as the radial positions of the highlight band of the current spring in the corresponding oblique light direction. If there is a slight difference between the radial positions of the highlight band obtained from the left-side and right-side oblique light, the average radial position of the two is used as the radial position of the highlight band of the current spring, and the left-side and right-side oblique light radial grayscale distributions are radially aligned according to this current highlight band radial position. This radial alignment is used to compensate for the overall offset of the highlight band caused by the difference in the current spring's arc height and arch curvature, so that subsequent mirror comparisons are based on the actual imaging state of the current spring, rather than on a fixed template. If the absolute value of the difference in radial position of the highlight band obtained by the left oblique light and the right oblique light exceeds the preset highlight band deviation threshold, the image processing unit determines that the symmetry of the current spring surface is abnormal, outputs the invalid result of the arch surface reflection reference, and marks the spring surface as to be re-inspected, and does not continue to perform the subsequent judgment of the arch surface to be judged area; the above-mentioned preset highlight band deviation threshold is determined in advance based on the calibration result of the qualified sample of the current specification spring surface, and is used to determine whether the current spring surface has an abnormality or detection pose deviation.

[0054] Under the condition that the left and right oblique light sources are arranged in a mirror symmetry with respect to the detection position of the spring, for an arched surface with an ideal surface morphology and no local undulations or defects, the reflected light intensity distribution under the oblique light illumination from the left and the right should have a certain degree of mirror symmetry. That is, the gray level of any point on the arched surface and the gray level of its mirror point about the axis of symmetry should satisfy a specific correspondence in the left and right oblique light images. Specifically, the gray level of a point in the left image should not differ from the gray level of its mirror point at the same position in the right image by more than a mirror residual threshold, and the gray level of the same point in the right image should not differ from the gray level of its mirror point at the same position in the left image by more than a mirror residual threshold. When the arched surface has surface defects such as linear gray level anomalies or planar closed gray level anomalies, the local reflection characteristics of the defective area change, and this mirror symmetry is disrupted, manifested as a significant increase in the gray level difference of the corresponding pixels. Therefore, by calculating the mirror gray level difference of each pixel in the arched surface detection area in the left and right oblique light images and comparing it with a preset mirror residual threshold, it is possible to distinguish between areas conforming to the normal reflection law of the arched surface and areas of potential defects.

[0055] It should be noted that the mirror symmetry judgment in this embodiment mainly targets local asymmetric defects generated during the stamping process. In actual stamping production, defects are usually caused by die damage, local material inhomogeneity, or fluctuations in stamping parameters. These defects often manifest as local linear grayscale anomalies, planar closed grayscale anomalies, material shortages, or burrs, all exhibiting obvious asymmetric spatial distribution characteristics. Completely symmetrical annular defects, such as full-circle planar closed grayscale anomalies or paired symmetrical defects, are extremely rare in actual production. Even if they occur, these defects inevitably lead to local changes in material thickness or surface morphology, appearing as obvious grayscale anomalies in the positive light image. This manifests as an overall grayscale shift across the entire area, rather than a local anomaly. This type of deviation falls under the category of dimensional deviation and should be addressed by the dimensional inspection process, not the surface defect detection scope of this embodiment. Therefore, this embodiment, based on mirror residual comparison, still captures defects through subsequent local grayscale difference judgment and area or length filtering. For anomalies that do not disrupt mirror symmetry, such as perfectly symmetrical annular defects or overall arc height deviations, they should be handled separately by visual inspection or dimensional inspection procedures and are not within the main scope of surface defect detection in this embodiment.

[0056] For any pixel within the arch detection area, the image processing unit determines its mirror image relative to the left and right oblique light directions based on the pre-defined left and right oblique light mirror relationship. The left and right oblique light mirror relationship is determined by the arrangement of the left and right oblique light sources and the camera relative to the detection position, and in the current spring coordinate system, it is represented as a mirror transformation with the spring center as the reference. If the mirror position falls on a non-integer pixel location, bilinear interpolation is used to obtain the corresponding grayscale value. If the mirror position falls within the punched area, the edge detection zone, or outside the effective image range, the pixel does not participate in the current arch normal reflection judgment and is marked as an invalid mirror pixel. When judging valid mirror pixels, it is also necessary to check whether the pixel and its mirror pixel are both valid in the validity mask of their respective transformed images. If the validity mask of any pixel is marked as invalid, meaning the pixel value originates from zero padding, the pixel does not participate in the current arch normal reflection judgment and is marked as an invalid mirror pixel; other pixels are recorded as valid mirror pixels.

[0057] Mirror residual threshold The mirror image residual distribution of the qualified sample within the arch surface detection area is determined based on the left and right oblique light mirror image residual distribution. Specifically, the mirror image residual is calculated for the effective pixels of the arch surface in the qualified sample, and the grayscale difference that can cover the normal highlight shift and normal shadow changes of the arch surface is used as the mirror image residual threshold.

[0058] The mirror residual threshold is determined based on the sum of the mean of the mirror residuals of qualified samples and a safety margin. Qualified samples consist of at least 30 manually verified defect-free spring clips. Images are acquired under the same lighting conditions as online inspection, and the mirror residual of each pixel is calculated. The mean μ and standard deviation σ of the mirror residual values ​​for all qualified samples are obtained, and μ + 3σ is taken as the mirror residual threshold. .

[0059] The mirror residual threshold is used in this embodiment to distinguish between the arch reference area and the arch to be judged area. Its aperture is always the gray level difference between mirror pixels in the left and right relative oblique light images.

[0060] For a mirrored valid pixel, the image processing unit performs two mirrored grayscale comparisons in opposite directions.

[0061] For any mirrored effective pixel p, its mirror symmetric point in the same oblique light image is: Extract pixels from the left oblique light image. The corresponding pixel in the right oblique light image ,in and They have the same physical position in the spring-piece coordinate system, and are all located at the same position as pixel point p. Based on the mirror relationship of left and right oblique light, in the left oblique light image, The mirror symmetric point is In the right oblique light, The mirror symmetric point is .

[0062] First comparison: Pixels from the left oblique light image grayscale The point that is mirror-symmetrical to the right oblique light image grayscale Compare and calculate the difference .

[0063] Second comparison: Take the grayscale value of the pixel in the right oblique light image. The point that is mirror-symmetric to the pixel in the left oblique light image. grayscale Compare and calculate the difference .

[0064] In the two comparisons above, if the grayscale difference is too large in either case, it indicates that there is an anomaly at the pixel position. Therefore, the maximum grayscale difference between the two comparisons is taken as the mirror residual D(p) of the pixel:

[0065] ,

[0066] in, This represents the maximum mirror grayscale difference between the left and right oblique light images at pixel p, expressed in grayscale values. Pixel p is located at the same physical position in both oblique light images.

[0067] like Not exceeding the mirror residual threshold, i.e. This indicates that the brightness change at that location can be explained by the normal mirror reflection of the current shrapnel arch surface under opposing oblique light from the left and right, and the pixel is recorded in the arch surface reference area; if > If the brightness change at that point cannot be explained by the radial position of the current highlight band and the left-right mirror reflection relationship, then the pixel is recorded in the arched surface undetermined area.

[0068] Since the mirror residual D(p) is symmetric about the center of the spring, i.e., D(p) = D(M(p)), D(p) itself cannot distinguish whether the defect is located at p or its mirror point M(p). To eliminate this mirror ambiguity, after forming the arched area to be judged, the image processing unit performs mirror candidate deduplication on pixel pairs that simultaneously satisfy the condition that both p and M(p) are located in the arched area to be judged:

[0069] When both pixel p and its mirror point M(p) are marked as the arched area to be judged, the image processing unit compares their local grayscale differences in the registered positive light image. The local grayscale difference is the average of the absolute values ​​of the differences between the grayscale values ​​of each pixel in the connected region and the average grayscale values ​​of pixels in the surrounding region, where the surrounding neighborhood can be a 3×3 pixel range. If the local grayscale difference at p is greater than the local grayscale difference at M(p), the defect is determined to be located at p, and M(p) is removed from the arched area to be judged; if the local grayscale difference at M(p) is greater than that at p, the defect is determined to be located at M(p), and p is removed from the arched area to be judged; if the local grayscale differences of both are less than the preset mirror candidate deduplication threshold, both are determined to be mirror artifacts and are removed together. If the local grayscale differences of both are equal and not less than the preset mirror candidate deduplication threshold, it is impossible to determine whether the defect is located at p or M(p), both are retained and marked as mirror position undetermined, and the entire piece corresponding to the undetermined position is marked as to be re-inspected.

[0070] The aforementioned preset deduplication threshold for candidate mirror images is determined by the following method: multiple qualified samples that have been manually confirmed to be defect-free are collected, the local grayscale difference distribution of the background area within the arched detection area is calculated, and the 95th percentile value is taken as the preset deduplication threshold for candidate mirror images.

[0071] By using this mirror candidate deduplication rule, any real defect retains only one candidate region at its actual location, while the artifact candidates at its mirror location are eliminated, thus avoiding the doubling of the number of defects and positional shifts.

[0072] Figure 4This is a diagram illustrating the correspondence between oblique light mirror images and the mirror residual threshold discrimination provided by an embodiment of the present invention. Figure (a) is a schematic diagram of the correspondence between mirror pixels p within the arched detection area, where the circle represents the arched detection area and the vertical dashed line is the mirror axis determined relative to the left and right oblique light directions. The image represents the mirror image of pixel p relative to the mirror axis. The double-headed dashed line indicates the spatial correspondence between the two, and the arrows in the two directions represent the left and right oblique light directions, respectively. Figure (b) shows the residual curve of the first comparison, and Figure (c) shows the residual curve of the second comparison. In Figures (b) and (c), the horizontal axis represents the normalized value of the mirror pixel pair order, and the vertical axis represents the relative mirror residual. / and / ,in and The single-mirror residual represents the grayscale difference between two comparisons. ; / and / This represents the ratio of a single mirror residual to the mirror residual threshold. The cyan curve corresponds to the first comparison, the orange curve to the second comparison, the red dashed line is the discrimination threshold line, the red hollow dots mark pixel pairs exceeding the discrimination line, the sloping filled area above the threshold line is the arched area to be judged, and the area below the threshold line is the arched reference area. The two mirror comparisons are performed independently: the relative mirror residuals of the two comparisons do not exceed 1, that is, the single mirror residual of the two comparisons does not exceed the mirror residual threshold. If the light-dark relationship is normal, the corresponding position is interpreted as normal reflection; otherwise, the position enters the arch surface undetermined area.

[0073] The edge boundary separation module and the image processing unit scan line by line along the normal direction of the outer contour of the spring piece and the punch contour within the edge detection zone. For the outer contour, the scanning direction is from the outside of the spring piece to the inside; for the punch contour, the scanning direction is from the blank side of the punch to the side of the spring piece. Each scan line is located in the current spring piece coordinate system and has a corresponding contour position number. The image processing unit records the backlight boundary position on the scan line in the backlight image and records the shadow boundary positions where the grayscale changes from bright to dark or from dark to bright in the registered front light image, the registered left oblique light image, and the registered right oblique light image, respectively.

[0074] For each scan line, the distance along the normal direction between the shadow boundary position and the backlight boundary position in the backlight image is calculated, and this distance is taken as the edge shadow width of the scan line. If multiple shadow boundary positions are obtained on the same scan line in the front lighting image, left oblique lighting image, and right oblique lighting image, the corresponding distances are calculated for each, and the largest effective distance is taken as the edge shadow width of the scan line.

[0075] If no valid shadow boundary can be obtained from the coaxial positive light image, left oblique light image, and right oblique light image at the same scan line position, then the scan line is marked as having a missing shadow boundary. If the number of scan lines with missing shadow boundaries exceeds a preset missing proportion threshold, then the current spring is marked as needing re-inspection. If only some images fail to obtain a valid shadow boundary, then the edge shadow width is calculated from the remaining images that can obtain a valid shadow boundary.

[0076] The preset missing proportion threshold is determined by the following method: Collect 30 qualified samples that have been manually confirmed to be without defects, and count the proportion of the number of scan lines in each sample whose shadow boundary extraction failed due to insufficient local lighting, smooth edge or imaging noise to the total number of scan lines in the edge detection zone of that sample. Take the 95th percentile value of this proportion for all samples as the preset proportion threshold.

[0077] Let the one-dimensional coordinates of the backlight boundary in the backlight image on a certain scan line be... The shadow boundary in a normal or oblique light image has a one-dimensional coordinate of [coordinate missing] on the scan line. The width of the edge shadow of the scan line. for:

[0078] ,

[0079] in, This represents the distance between the shadow boundary and the backlight boundary along the scan line direction, in pixels.

[0080] The upper limit of the edge shadow width is determined based on the width of the positive and oblique shadows at the outer contour and punched contour of at least 30 qualified samples. For samples with normal edge adhesion and no missing material or burrs, the image processing unit calculates the distance between the shadow boundary and the backlight boundary along the contour normal direction. For the outer contour and punched contour, the mean of this distance for the corresponding samples is added to three times the standard deviation as the upper limit of the edge shadow width to cover most normal edge overhangs, curved surface transitions, and illumination shadow extensions. This parameter is only used to determine whether the edge brightness variation belongs to normal shadows and is not used as the basis for measuring the area of ​​burrs or missing material.

[0081] The image processing unit further compares the change in the backlight boundary position on the current scan line with the change in the backlight boundary position of adjacent scan lines or within a local neighborhood. If the edge shadow width does not exceed the upper limit of the edge shadow width, and the backlight boundary in the backlight image does not shrink or bulge beyond the boundary abrupt change threshold relative to adjacent scan lines, then the brightness change at that location is recorded as an edge reference region. The edge reference region represents the normal brightness expansion caused by the outer contour and punch contour edges, arc transitions, or illumination angles, and is not output as a real material defect or burr. Adjacent scan lines do not only refer to two physically adjacent actual scan lines, but also include the reference contour position obtained by statistical fitting of the backlight boundary positions of adjacent scan lines. Specifically, the image processing unit first performs RANSAC fitting statistics on the backlight boundary positions of all scan lines on the current contour, uses this statistical value as the boundary position of the reference contour for the local region, and then calculates the difference between the current scan line and the reference contour. The calculation method for this difference is the same as the calculation method for the difference between adjacent scan lines. In this sense, adjacent scan lines should be understood as equivalent adjacent positions including the reference contour.

[0082] RANSAC fitting employs different contour model types to adapt to the contour shape of the current specification spring, where the contour shape is preset in the product specification configuration. In spring reference contour fitting, the image processing unit randomly samples from the backlight boundary positions of all scan lines through 500 iterations, with the random seed for random sampling fixed as a constant. For each set of sampled points, a contour model is fitted, and the number of boundary points within 1 pixel of the model is counted as the inlier count. The model with the most inliers is selected as the candidate reference contour. If the difference in the number of inliers between the model with the most inliers and the model with the second most inliers does not exceed 5% of the total number of boundary points, the reference contour fitting result is deemed unstable, and the current spring is marked for re-inspection. Finally, the reference contour position is obtained by refitting based on the selected set of inliers. If the proportion of inliers to the total number of boundary points is less than 50%, the current spring contour is deemed to have a large-scale anomaly, the reference contour fitting is unreliable, and the current spring is marked for re-inspection.

[0083] If the backlight boundary in the backlight image is recessed relative to the reference contour and the recess exceeds the boundary abrupt change threshold, the image processing unit records the corresponding recessed area as a region to be judged. The boundary abrupt change threshold is automatically calculated based on the boundary segmentation noise level of the backlight image. The specific calibration method is as follows: at least 30 qualified samples that have been manually confirmed to be defect-free are collected. For all outer contour scan lines of each sample, the positional deviation of the backlight boundary relative to the reference contour at each scan line is calculated. The positional deviations of all scan lines of all samples are summarized, and the standard deviation of the boundary position fluctuation is calculated. Three times the standard deviation is taken as the upper limit of the boundary fluctuation. If three times the standard deviation of the boundary position fluctuation is less than the minimum allowable grayscale anomaly size of the product, then three times the standard deviation of the boundary position fluctuation is taken as the boundary abrupt change threshold; if three times the standard deviation of the boundary position fluctuation is greater than or equal to the minimum allowable grayscale anomaly size of the product, then it is determined that normal boundary fluctuation and minimum defect cannot be reliably distinguished under the current imaging conditions, and the spring is marked as needing re-inspection. This threshold is used to determine whether the recess or protrusion of the backlight boundary relative to the reference contour belongs to a real material boundary change. If the boundary change does not exceed the boundary abrupt change threshold, the output will not be missing material or burrs due to the slight jitter of the backlight contour; if the boundary change exceeds the boundary abrupt change threshold, it will enter the boundary judgment signal area: if it shrinks inward, it will enter the shrinkage judgment area; if it protrudes outward, it will enter the protrusion judgment area.

[0084] Let the position of the backlight boundary at the current scan line be... Boundary change at the current scan line for:

[0085] ,

[0086] in, ∈S, where S represents the set of scan lines covered by all valid regions. This indicates the displacement of the current backlight boundary relative to the reference contour, in pixels.

[0087] like ≤ If so, then the current boundary change is determined to be within a controllable threshold; where This is the threshold for boundary mutation, expressed in pixels.

[0088] when > If so, it is determined that a boundary change has occurred, and the direction of the boundary change is further determined: if If the spring retracts relative to the adjacent scan line, it is determined to be a retraction signal to be judged; if If the material protrudes outward relative to the adjacent scan line, it is determined as a protrusion signal to be judged.

[0089] For the outward-protruding region to be judged, its area for:

[0090] ,

[0091] in, The set of scan lines covered by the burr body region. This refers to the position where the backlight boundary protrudes outwards along the j-th scan line; This is the reference contour position on scan line j, i.e., the normal contour position when there is no protrusion. The backlight boundary positions of all scan lines on the current contour are obtained by RANSAC fitting statistics to eliminate the influence of local material shortages and burr protrusions on the reference position. The arc length of the contour between adjacent scan lines; the calculated result is the pixel area of ​​the burr body, in pixels².

[0092] If the backlight boundary in the backlight image protrudes outward relative to the adjacent scan line and the protrusion exceeds the boundary abrupt change threshold, the image processing unit records the corresponding protruding material area as the protrusion to be judged region. Regions adjacent to this protruding region but which only show an increase in shadow width without further protrusion of the backlight boundary are not included in the protrusion to be judged region.

[0093] For the region to be judged after shrinking, its area for:

[0094] ,

[0095] in, The set of scan lines covering the area of ​​material shortage; The position where the backlight boundary is recessed inward on the j-th scan line; the calculated result is the pixel area of ​​the missing material region, in pixels².

[0096] If a valid shadow boundary cannot be obtained in a certain illumination image within the edge detection zone, but a stable backlight boundary can be obtained in the backlight image, the image processing unit calculates the edge shadow width based on the other images that can obtain shadow boundaries. If a valid shadow boundary cannot be obtained in the front light image, left oblique light image, and right oblique light image, then the scan line does not form an edge reference area, and the determination of missing material or burrs is still based on whether the backlight boundary exceeds the boundary abrupt change threshold. If an isolated single-point jump occurs in the backlight boundary within a local area, and the jump does not form a continuous boundary contraction or protrusion, then the jump is not output as a missing material or burr on its own.

[0097] Before the defect residual generation module is executed, the image processing unit first performs imaging quality verification on the registered positive light image. The imaging quality verification includes at least one of the following: (1) Sharpness verification: Taking the arch detection area as the calculation area, calculate the Laplacian variance of the pixel gray value in the area. If the value is lower than the preset sharpness threshold, it is determined that the image is out of focus, and the current spring is marked as to be re-inspected; (2) Saturated pixel verification: Statistically count the proportion of pixels in the image whose gray value reaches the upper limit of saturation. In this embodiment, it is an 8-bit image with an upper limit of saturation of 255. If the proportion exceeds the preset saturation proportion threshold, it is determined that the image is overexposed, and the current spring is marked as to be re-inspected; (3) Glare verification: Statistically count the distribution of gray gradient modulus values ​​in local areas of the image. If the contrast between the high brightness area and the neighborhood exceeds the preset glare threshold, which is 200 in this embodiment, it is determined that there is local glare, and the current spring is marked as to be re-inspected.

[0098] The sharpness threshold, saturation ratio threshold, and glare threshold are calibrated using qualified samples or predetermined based on the characteristics of the image sensor. If any item in the image quality verification fails, the current spring will not perform subsequent defect residual generation and defect classification, and will directly output "awaiting re-inspection".

[0099] The defect residual generation module and the image processing unit first process the arched surface region to be judged. The pixels in the arched surface region to be judged include at least the following components:

[0100] (1) Real defects: Linear gray-scale anomalies and planar closed gray-scale anomalies disrupt the mirror symmetry of the arch surface, causing the mirror residual to exceed the threshold. (2) Normal highlight band edge: The different descent speeds on the left and right sides of the gray-scale gradient end cause the mirror residual to exceed the threshold. (3) Isolated noise: Sensor noise causes the mirror residual to exceed the threshold.

[0101] Of the three types of pixels mentioned above, the normal highlight band edge belongs to the normal shape of the spring sheet, and the isolated noise belongs to the physical noise of the imaging system. Both are non-defect components that were misjudged as the signal to be judged by the arch surface. The reasons for their misjudgment are as follows: the normal highlight band edge causes the mirror symmetry to be destroyed by the normal curvature change, resulting in the mirror residual exceeding the mirror residual threshold. It is subsequently eliminated by judging the continuous change of gray level; the isolated noise causes the local mirror residual to exceed the mirror residual threshold due to random fluctuations of the sensor. It is subsequently eliminated by filtering by area and length.

[0102] For this region, the image processing unit identifies the connected regions adjacent to the arch surface and the corresponding arch surface reference region. A connected region refers to a set of interconnected pixels within the arch surface region to be judged, which is extracted from the binary image of the arch surface region to be judged using a connected component analysis algorithm. A single arch surface region to be judged may contain one or more unconnected connected regions. For each connected region, it checks whether there are any pixels belonging to the arch surface reference region within the 8-neighborhood of the edge pixels of the connected region. If so, the connected region is determined to be adjacent to the arch surface reference region.

[0103] For connected regions adjacent to the arched reference region, the sampling direction is determined using a distance transformation method: the shortest distance from each pixel within the connected region to the adjacent normal region is calculated, and the sampling direction is taken along the gradient direction of the shortest distance, ensuring that the sampling path traverses the narrowest cross section of the connected region. If the grayscale sequences of the connected region show contradictory determinations in different directions, such as one direction being determined as continuous and another as abrupt change, the direction of abrupt change is prioritized, and the connected region is retained.

[0104] The image processing unit extracts the grayscale sequence of its edge pixels along the direction from the normal region to the connected region. , ,… (n≥3). If the sequence shows a monotonically increasing or monotonically decreasing trend, and the difference between adjacent gray levels is... The maximum ratio among (i≥1) does not exceed the adjacent difference ratio threshold R, i.e. If the gray level of the connected region changes continuously, it is marked as a candidate region of continuous change and not deleted for the time being; otherwise, it is retained.

[0105] To prevent adjacent differences When the value is 0, division by zero occurs. When calculating the ratio of adjacent grayscale differences, if... If the grayscale difference ratio at that location exceeds the threshold R, it is considered a grayscale abrupt change. Here, ε is a preset positive decimal, such as 1 grayscale level. If di = di + 1 = 0, it means that the grayscale level of that segment is completely flat and does not constitute an abrupt change; it is directly determined as a continuous change.

[0106] Taking a certain connected region to be judged as an example, the grayscale sequence of the edge pixels of the connected region to be judged is 200, 195, 185, 175, with adjacent differences of 5, 10, 10, and difference ratios of 2.0 and 1.0 respectively. Neither exceeds the preset threshold R=3, and the sequence is monotonically decreasing. Therefore, this connected region is included in the continuously changing candidate region. If the grayscale sequence is 210, 200, 120, 115, with adjacent differences of 10, 80, 5, and difference ratios of 8.0 and 16.0 respectively, both exceeding R=3, then this connected region is retained.

[0107] The adjacent difference ratio threshold R is determined by calibration as follows: At least 30 qualified samples, manually confirmed to be defect-free, are collected. Multiple pixel grayscale sequences are extracted from the edge region of the normal highlight band within the arched surface inspection area. The adjacent grayscale difference ratio of each sequence is calculated, and the 95th percentile of all difference ratio distributions is taken as the value of R. This calibration process is repeated when the product is redesigned or when lighting conditions change.

[0108] If the grayscale sequence of a connected region satisfies monotonicity and the ratio of adjacent differences does not exceed the threshold R of adjacent differences, then the connected region belongs to normal highlight or normal shadow transition and does not constitute a real defect. It will be deleted after subsequent local grayscale difference verification and confirmation; otherwise, it will be retained.

[0109] Through this process, the edges of the normal highlight band, the normal shadow transition, and the continuous reflection changes caused by the difference in arc height will not enter the defect residual area of ​​the current shrapnel, and only the areas with gray-scale abrupt changes will be retained as valid candidates.

[0110] If the grayscale sequence shows a sudden change, the connected region is retained and enters the subsequent joint filtering stage based on area, continuous length, and local grayscale difference. In the joint filtering, if the area of ​​the region is less than the minimum effective defect area, the continuous length is less than the minimum effective defect length, and the local grayscale difference is less than the preset local grayscale difference threshold, it is determined as isolated noise and deleted.

[0111] It should be noted that, for connected regions marked as candidate regions of continuous change, this embodiment does not unconditionally determine them as normal specular transitions. The image processing unit further examines the local grayscale difference of this connected region in the registered positive-light image in subsequent steps. If the local grayscale difference of the connected region exceeds a preset local grayscale difference threshold, it indicates that although the grayscale change in this region is smooth, its overall grayscale deviates from the grayscale level of the surrounding normal region, belonging to a true gradual change defect such as a shallow, closed grayscale anomaly or a gradual concavity, and is therefore retained. If the local grayscale difference does not exceed the threshold, it indicates that the grayscale change in this region is consistent with the surrounding normal region, belonging to a normal specular edge, and the candidate region of continuous change is deleted from the arched surface to be judged region.

[0112] For connected regions that are not adjacent to the arch reference area, they do not undergo continuous grayscale change judgment and directly enter the subsequent area and length filtering stage.

[0113] For the brightness variation areas within the edge detection zone, the image processing unit deletes the portion overlapping with the edge reference area and retains the inward and outward bulge areas to be judged. These inward and outward bulge areas are formed based on the changes in the backlight boundary in the backlight image; therefore, their area and boundaries do not increase due to the expansion of surrounding positive or oblique light shadows. The brightness variation areas within the edge detection zone also include non-defect signals caused by isolated single-point jumps in the backlight boundary due to sensor noise. Although this isolated noise is initially marked as a brightness variation area, its backlight boundary jump does not form a continuous inward or outward bulge; it is subsequently eliminated based on the continuity of boundary changes and is not output as a missing material or burr.

[0114] If the width of the edge shadow exceeds the upper limit of the edge shadow width but the backlight boundary does not exceed the threshold of the boundary mutation amount, then this area will not be recorded in the edge reference area, nor in the area to be judged for material shortage or protrusion. Instead, it will be treated as an undefined anomaly by the defect residual generation module and participate in the subsequent merging process.

[0115] For the remaining arched surface regions to be judged after the above processing, the image processing unit calculates the local gray-level difference, continuous length, and area of ​​each connected region in the registered orthophoto image.

[0116] Local grayscale difference is the average of the absolute values ​​of the differences between the grayscale values ​​of each pixel in the connected region and the average grayscale value of the pixels in the surrounding region of the connected region, where the surrounding neighborhood can be a range of 3×3 pixels.

[0117] Local grayscale difference is determined by the following formula:

[0118] ,

[0119] Where S represents the connected region, N represents the surrounding neighborhood formed by extending the connected region outwards by a preset pixel range, g(p) is the gray value of pixel p within the connected region, g(q) is the gray value of pixel q within the surrounding neighborhood, |S| is the total number of pixels in the connected region, and |N| is the total number of pixels in the surrounding neighborhood. ΔG represents the degree of local gray-level deviation of the connected region relative to its surrounding background, expressed in gray levels. When ΔG is less than a preset local gray-level difference threshold, it is determined that there is no significant gray-level difference between the region and the surrounding background, indicating that no significant gray-level difference has been formed between the connected region and the surrounding background. The gray-level change in this region is insufficient to constitute a valid defect feature, and combined with the area and continuous length conditions, it is deleted as isolated noise.

[0120] The local grayscale difference threshold calibration method involves collecting at least 30 qualified samples and calculating the local grayscale difference distribution of the background area within the arch surface detection zone, taking the 95th percentile value as the threshold. The minimum effective defect area and minimum effective defect length are determined according to the minimum detectable defect size specified in the product inspection standard. If the area of ​​a certain region is smaller than the minimum effective defect area, the continuous length is smaller than the minimum effective defect length, and the local grayscale difference is smaller than the preset local grayscale difference threshold, then that region is judged as isolated noise and deleted; the remaining regions are treated as arch surface defect residual regions. Subsequently, the image processing unit merges the arch surface defect residual regions, the inward-shrinking to-be-judged regions, and the outward-protruding to-be-judged regions to form the defect residual regions.

[0121] It should be noted that isolated noise manifests differently in the arched surface judgment area and the edge judgment signal area, but both are nondeterministic outputs of the sensor or imaging system, rather than actual defects on the surface of the spring sheet. Isolated noise in the arched surface judgment area is filtered using area and continuous length thresholds; isolated noise in the edge judgment signal area is excluded by judging the continuity of boundary changes, that is, isolated boundary jumps that do not form continuous inward or outward contractions are not considered as material shortages or burrs.

[0122] In the detection result output module, the image processing unit calculates the location, length, width, area, and whether it is within the edge detection zone or the arched surface detection area for each defect residual region. The location is the coordinate of the geometric center of the defect residual region in the spring coordinate system; the length is the length of the continuous edge of the defect residual region; the width is the maximum width perpendicular to the length direction; and the area is the effective pixel area within the defect residual region.

[0123] The upper limit of crack width is determined according to product inspection standards, acceptance standards or sample calibration results, and is used to distinguish between continuous linear gray-scale anomalies and planar closed gray-scale anomalies in the residual area of ​​arch surface defects.

[0124] If the same region within the edge detection band simultaneously meets the following two conditions:

[0125] (1) The edge shadow width exceeds the upper limit of the edge shadow width; (2) The indentation or protrusion of the backlight boundary relative to the adjacent scan line exceeds the boundary abrupt change threshold.

[0126] The change in the backlight boundary is used as the primary criterion for judgment. The affected area is retained as either an inward-shrinking or outward-protruding region to be judged, and is not classified as an undefined anomaly based solely on exceeding the shadow width limit. This is because the backlight boundary in the backlight image reflects the change in the actual material boundary. An increase in shadow width may be caused by burr protrusion or localized suspension, which are accompanying phenomena of edge anomalies. Ignoring the actual change in the backlight boundary due to exceeding the shadow width limit will lead to the missed detection of actual material shortages or burr defects. Only when the backlight boundary does not exhibit inward shrinkage or outward protrusion exceeding the boundary abrupt change threshold, and the edge shadow width exceeds the upper limit of the edge shadow width, is the affected area not included in the edge reference region, but rather treated as an undefined anomaly by the defect residual generation module for subsequent merging processing.

[0127] If the defect residual region is located within the arch surface detection area, and its length is not less than the minimum effective defect length, its width is not greater than the upper limit of the crack width, and it appears as a continuous linear gray-level abrupt change in the registered positive light image, and this linear region exhibits a consistent gray-level anomaly direction in both left and right oblique light images, and the difference in gray-level gradient amplitude on both sides of the normal direction of the linear region does not exceed a preset symmetry threshold, then it is determined to be a defect gray-level anomaly. If the defect residual region is located within the arch surface detection area, and its area is not less than the minimum effective defect area, its width is greater than the upper limit of the crack width, and the backlight boundary does not show corresponding inward or outward contraction, and the boundary closure degree of this planar region is not lower than a preset closure degree threshold, and it exhibits gray-level anomalies in both positive light and left and right oblique light images, then the image processing unit outputs that the defect is a defect gray-level anomaly. Defect residual regions that do not meet the above conditions are output as undefined anomalies, and their position and area are retained.

[0128] The preset symmetry threshold is determined by the following method: collect 30 qualified samples that have been manually confirmed to be defect-free and a known linear gray-scale defect sample, take the average gradient of 3 pixels on each side along the normal direction of the defect area, calculate the absolute value of the difference between the average gradients on both sides; take the 95th percentile value of this difference of all qualified samples as the preset symmetry threshold.

[0129] The preset closure threshold is determined by the following method: collect 30 qualified samples that have been manually confirmed to be defect-free and known surface closure gray-scale abnormal defect samples, calculate the boundary closure of each defect area or candidate area respectively, and define the closure as the ratio of the circumference of an equal-area circle to the perimeter of the area; take the 5th percentile value of the closure of all qualified samples as the preset closure threshold.

[0130] If the defect is determined to be an anomaly in grayscale, further determine the type of defect: Let the length of the defect residual region be L; the width be W, representing the maximum width of the defect residual region perpendicular to the length direction; the area be A; and the minimum effective defect length be... The minimum effective defect area is The upper limit of the crack width is The morphological classification of the defect residual region is determined according to the following conditions:

[0131] When L≥ W≤ Furthermore, when all three conditions are met—continuous linear gray-level abrupt changes—the defect is output as a linear gray-level anomaly, such as a shallow crack; when A ≥ W> Furthermore, the backlight boundary has no inward or outward retraction, and the output of this defect is a planar closed grayscale abnormality, such as an indentation.

[0132] Through the above processing, the unshaped anomaly is not only output as the current test result of the spring, but also provides statistical basis for the stability monitoring of the molding process and the evaluation of the mold condition.

[0133] When a spring has no defect residual area, the image processing unit outputs "OK" for the spring. When at least one defect residual area is identified as missing material, burr, linear grayscale anomaly, planar closed grayscale anomaly, or unshaped anomaly, the image processing unit outputs "NG" for the spring and outputs the category, location, length, width, and area of ​​each defect. If a defect residual area exists but its category does not meet the classification conditions for missing material, burr, linear grayscale anomaly, or planar closed grayscale anomaly, the detection result output module outputs "unshaped anomaly" and retains the location and area of ​​the area. Unshaped anomalies are not deleted as normal reflections or normal shadows for subsequent quality review or process statistics. For defect residual areas identified as unshaped anomalies, the image processing unit further performs the following processing:

[0134] (1) Extract the multi-light image sub-images of the region, including the local image regions at the corresponding positions in the front light, left oblique light, right oblique light and backlight images, and associate these sub-images with the detection results of the current shrapnel and store them in the quality database.

[0135] (2) Statistically analyze the frequency of occurrence of unshaped abnormalities in the current batch, their spatial distribution on the surface of the spring sheet, and their geometric morphological characteristics. When the cumulative occurrence ratio of unshaped abnormalities in the same batch exceeds the preset abnormality accumulation threshold, output a molding process abnormality warning.

[0136] (3) For a single spring piece where the area of ​​unshaped abnormality exceeds the preset large area abnormality threshold, even if the specific defect type cannot be determined, the spring piece is still judged as NG and an alarm signal is output to prompt manual review.

[0137] The test results are sent to the display unit, quality recording unit, or subsequent sorting unit on the production line. However, in this embodiment, the defect judgment is based on the defect residual area and defect classification results formed by the image processing unit.

[0138] Through the above processing, the same spring sheet obtains a front light image, a left-right relative oblique light image, and a back light image within the same detection trigger cycle. The back light image is used to establish the real material boundary and the unified spring sheet coordinates. The left-right relative oblique light image is used to generate the arched reference area of ​​the current spring sheet based on the radial position of the current highlight band and the mirror residual threshold. The distance between the back light boundary and the front light and oblique light shadow boundaries is used to distinguish the edge reference area, missing material, and burr body. Finally, the defect judgment is not based on the original grayscale abrupt change, but on the defect residual area after deducting the normal arched reflection and the edge reference area, thereby reducing the interference of arch curvature, arc height difference, local edge suspension, and burr support on the real stamping forming defect signal.

[0139] In this embodiment, if an anomaly occurs in the detection process, i.e., the image acquisition validity or contour positioning validity verification fails, the corresponding invalid state is output first, followed by a state pending re-inspection. If the validity verification passes and a defective residual region exists, NG is output. OK is output only when the validity verification passes and the defective residual region is empty. The aforementioned empty defective residual region includes the case where candidate signals are completely deleted after area or length filtering. The state pending re-inspection indicates that the current spring needs to be manually or offline reviewed. After review, the final state should be OK or NG, and pending re-inspection is not retained as the final state.

[0140] When product specifications change, light sources are replaced, or camera parameters change, or when the false detection rate or missed detection rate exceeds the warning value during continuous operation, recalibration must be performed. Furthermore, when the average area of ​​the defect residual region of 30 consecutive spring pieces exceeds three times the average area of ​​30 qualified samples of that specification, a molding process anomaly warning is issued; when the area of ​​a single defect residual region exceeds 10% of the arch surface detection area, a large-area anomaly warning is triggered and marked for re-inspection. If the average area of ​​qualified samples is 0 or less than the preset minimum statistical accuracy, the preset default value is used instead. The above triggering conditions are basic operations for daily maintenance and quality control of production line equipment; their specific observation methods and warning values ​​are determined based on the actual production line configuration and will not be elaborated here.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An online visual inspection system for forming defects in metal button spring stamping, characterized in that, include: The image acquisition control module is used to control the light sources to light up in a fixed sequence and to enable the camera to acquire multi-directional illumination images within the same trigger cycle; the multi-directional illumination images include front light images, backlight images, and oblique light images generated by a pair of mirror-symmetrically arranged light sources; The contour localization and registration module is used to extract the contour of the metal button spring with a mirror-symmetric geometric shape, establish a unified spring coordinate system to register multi-directional illumination images, and divide the metal button spring into an edge detection zone and an arched surface detection area according to the contour. The arch surface reflection reference generation module is used to calculate the mirror residual of each pixel in the arch surface detection area based on the mirror symmetry relationship of the oblique light image. The part of the mirror residual exceeding the mirror residual threshold is marked as the arch surface to be judged area, and the part not exceeding the mirror residual threshold is marked as the arch surface reference area. The edge boundary separation module is used to calculate the edge shadow width within the edge detection zone; mark the area where the boundary abruptness of the metal button spring exceeds the boundary abruptness threshold as the edge to be judged area; and mark the area where the boundary abruptness does not exceed the boundary abruptness threshold and the edge shadow width does not exceed the upper limit of the edge shadow width as the edge reference area. The defect residual generation module is used to remove connected regions with continuously changing gray levels from the area to be judged on the arch surface, and to remove connected regions whose area, continuous length, and local gray level difference do not meet the corresponding thresholds, and to merge the remaining regions to generate the defect residual region. The detection result output module is used to output the detection results of the metal button spring based on the defect residual area: if either the image acquisition validity and registration validity verification fails, it will output that it needs to be re-inspected. If no abnormality occurs in the detection process and the defect residual area is empty, the detection result is OK; otherwise, the detection result is NG and the corresponding defect type is output.

2. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, The steps for dividing the edge detection zone and the arched surface detection area are as follows: Based on the outline, offset by a preset distance to its inner and outer sides respectively, and use the sum of the preset distances of the offset area as the width of the edge detection band to form an edge detection band; if the metal button spring of the current specification requires a punch, the outline information also includes the punch outline; if the punch outline is not detected in the backlight image, the punch missing abnormality is output. The edge detection zone is used to cover the contour and the area where defects may occur; the arched surface detection area is the remaining area on the surface of the spring sheet after removing the edge detection zone and the blank area for punching.

3. The online visual inspection system for forming defects in metal button spring stamping according to claim 2, characterized in that, In the contour positioning and registration module, the center of the outer contour extracted from the backlight image is used as the origin of the spring coordinate system, and the main direction of the outer contour is used as the first coordinate axis of the spring coordinate system to establish the current spring coordinate system; and the front light image and the oblique light image are both registered to the spring coordinate system.

4. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, In the arched surface reflection reference generation module, the two oblique light images are radially aligned based on the radial position of the current highlight band of the spring piece. The two oblique light images are the mirror-symmetrically arranged oblique light images acquired by the image acquisition control module, and are respectively denoted as the left oblique light image and the right oblique light image. After radial alignment, the gray level of the pixel in the left oblique light image is compared with the gray level of its mirror pixel in the right oblique light image for the first time, and the gray level of the pixel in the right oblique light image is compared with the gray level of its mirror pixel in the left oblique light image for the second time. Both comparisons produce a gray level difference value, and the larger gray level difference between the two comparisons is taken as the mirror residual of the pixel.

5. The online visual inspection system for forming defects in metal button spring stamping according to claim 4, characterized in that, The radial position of the highlight band is obtained by the following method: Within the arch detection area, each pixel is converted into radial and angular positions, and the gray-level statistical features of the two mirror-symmetrical oblique light images at different radial positions are statistically analyzed. Based on the gray-scale statistical characteristics, the highlight feature position of each radial gray-scale distribution is determined. The average of the highlight feature positions of two mirror-symmetrical oblique light radial gray-scale distributions is taken as the radial position of the highlight band, and the two distributions are aligned according to this position.

6. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, The edge shadow width is calculated as follows: scan lines are set one by one along the normal direction of the contour within the edge detection band, and each scan line has a corresponding contour position number; for each scan line within the edge detection band, the backlight boundary position in the backlight image and the shadow boundary position in the oblique light image are obtained; the maximum effective distance along the normal direction between the shadow boundary position and the backlight boundary position in the positive light and oblique light images that produce shadows on the same scan line is taken as the edge shadow width of the scan line.

7. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, The boundary abrupt change is generated by the abrupt change of the boundary of the metal button spring relative to the reference contour. The boundary abrupt change includes inward shrinkage and outward protrusion. If the backlight boundary shrinks inward relative to the adjacent scan line and the shrinkage exceeds the boundary abrupt change threshold, the corresponding shrinkage area is recorded as the inward shrinkage to be judged area. If the backlight boundary in the backlight image protrudes outward relative to the adjacent scan line and the protrusion exceeds the boundary abrupt change threshold, the corresponding protruding material area is recorded as the outward protrusion to be judged area. The reference contour is obtained by fitting the backlight boundary position of each scan line.

8. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, The specific methods for generating the defect residual region include: Several connected regions are extracted from the arch surface to be judged region; for each connected region, it is checked whether there are pixels belonging to the arch surface reference region within the neighborhood range of the edge pixels of the connected region. If there are, it is determined that the connected region is adjacent to the arch surface reference region. For a connected region adjacent to the arched reference region, the grayscale sequence of edge pixels is extracted along the direction from the arched reference region to the connected region. If the grayscale sequence shows a monotonically increasing or monotonically decreasing trend, and the maximum ratio between adjacent grayscale differences in the sequence does not exceed the adjacent difference ratio threshold, then the grayscale of the connected region is determined to be continuously changing, and it is marked as a candidate region for continuous change and temporarily retained. Otherwise, the grayscale of the connected region is determined to be abruptly changing and is retained. For connected regions that are not adjacent to the arch surface reference area, they are directly retained; For the arch surface connected regions retained after the above processing, the local gray level difference of each connected region is calculated in the registered positive light image; if its area is less than the minimum effective defect area, its continuous length is less than the minimum effective defect length, and its local gray level difference is less than the local gray level difference threshold, then it is removed. The remaining arch surface region to be judged is merged with the edge region to be judged to generate the defect residual region.

9. The online visual inspection system for forming defects in metal button spring stamping according to claim 7, characterized in that, When the defect residual region is not empty, the defect type corresponding to each defect residual region is determined according to the position, geometry and backlight boundary change direction of the defect residual region in the spring coordinate system. The defect types include: defects located within the edge detection zone and whose backlight boundary is recessed relative to the spring sheet are identified as material shortage; defects located within the edge detection zone and whose backlight boundary protrudes relative to the spring sheet are identified as burrs; defects located within the arched surface detection area and exhibiting grayscale anomalies in the registered positive light image, left oblique light image, and right oblique light image, if exhibiting continuous linear grayscale abrupt changes, are identified as linear grayscale anomalies; defects exhibiting planar grayscale abrupt changes are identified as planar closed grayscale anomalies; defects that do not meet any of the above morphological conditions are identified as undefined anomalies.

10. The online visual inspection system for forming defects in metal button spring stamping according to claim 1, characterized in that, Before the image acquisition and control module executes, the feeding mechanism first unifies the metal button springs so that the arched surface faces upwards, and then the conveying mechanism sends the springs to the detection position so that the springs are within the camera's field of view, completing the coarse positioning; in the coarse positioning state, the position and angle deviation of the springs in the detection position are subsequently eliminated by the contour positioning and registration module through image registration.

Images