Hardware stamping defect visual detection method and system based on image technology

Abstract

The invention relates to the technical field of visual inspection, in particular to a hardware stamping defect visual inspection method and system based on an image technology. The method comprises the following steps of: setting a visual inspection work station at the downstream of a stamping line, and combining an automatic transmission track, an electric clamp, a laser alignment module, a light source combination, an industrial camera and a PLC (Programmable Logic Controller) control removal device to realize precise positioning and multi-angle image acquisition of hardware, identify a flow mark strip area in an image, quantitatively judge the severity of a flow mark, and improve the quality of the hardware. And unqualified products are automatically judged and are sorted by the rejecting device. According to the method, an intelligent system based on integration of visual inspection and automatic control is constructed, efficient, accurate and standardized automatic identification and grading judgment of hardware flow mark defects are achieved, and the quality management and control level and the production line intelligent capacity are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a visual inspection method and system for stamping defects in hardware parts based on image technology. Background Technology

[0002] Flow marks are a common surface defect in the stamping process, often caused by factors such as uneven material flow, die wear, and insufficient lubrication. Although they sometimes do not affect structural strength, they seriously affect the appearance and functional integrity of the product. Especially in industries with extremely high requirements for surface quality, such as automobiles, electronics, and home appliances, exceeding the standard can directly lead to customer returns, fines, or damage to brand reputation. Therefore, accurate identification of flow marks has become the key to quality control.

[0003] Currently, in the stamping process of metal parts, the detection of most surface defects, such as flow marks, is mostly carried out by methods such as manual visual inspection, single-angle visual inspection, or manual inspection with auxiliary lighting. However, the stamping production line has a fast cycle time, and the speed of manual inspection is difficult to match the production rhythm. Moreover, long-term inspection can easily cause visual fatigue, leading to missed or misjudged flow mark defects, affecting quality stability. Furthermore, a single perspective cannot fully cover the surface of the part, especially for metal parts with curved or uneven structures, where some flow marks are easily overlooked due to angle obstruction or uneven lighting. At the same time, traditional algorithms have limited ability to identify texture defects (such as flow mark streaks) and cannot quantify and classify the severity of defects based on features such as image grayscale and texture intensity. Summary of the Invention

[0004] Therefore, it is necessary to provide a visual inspection method and system for stamping defects of hardware parts based on image technology to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above objectives, a visual inspection method for stamping defects in hardware parts based on image technology is provided. This method includes a visual inspection workstation, an automatic transfer track, a miniature rotary table, and a PLC-controlled rejection device. The PLC-controlled rejection device has a built-in high-speed input module connected to the PLC program. The visual inspection workstation has a built-in laser alignment module and an automatic positioning device. The method includes the following steps: Step S1: Establish a vision inspection workstation downstream of the stamping production line, and set up the vision inspection workstation and automatic transfer track. Use the automatic transfer track to send each stamped metal part into the inspection area of ​​the vision inspection workstation. Step S2: Fix the hardware in the inspection area using an electric clamp; install an industrial camera on the fixed hardware and configure a combination of a ring light source and a slanted strip light source; Step S3: Detect the position signal of the hardware part by the photoelectric sensor on the vision inspection workstation, and trigger the camera to acquire images of the hardware part under the light source combination to obtain a set of hardware part images from multiple angles; Step S4: Identify the flow mark stripe area in the hardware image set and determine the severity of the flow mark stripe area; mark the hardware with the flow mark severity greater than the preset severity threshold as unqualified hardware, and automatically sort the unqualified hardware by the PLC-controlled rejection device.

[0006] This specification provides a visual inspection system for stamping defects in hardware parts based on image technology, used to perform the aforementioned visual inspection method for stamping defects in hardware parts based on image technology. The visual inspection system for stamping defects in hardware parts based on image technology includes: The transmission module is used to establish a vision inspection workstation downstream of the stamping production line, and to set up the vision inspection workstation and the automatic transmission track. The automatic transmission track is used to send each stamped metal part into the inspection area of ​​the vision inspection workstation. A fixing module is used to fix the hardware in the inspection area by electric clamps; an industrial camera is mounted on the fixed hardware and configured with a combination of ring light source and oblique strip light source; The image acquisition module is used to detect the position signal of the hardware parts through the photoelectric sensor on the vision inspection workstation, and trigger the camera to acquire images of the hardware parts under the light source combination to obtain a set of hardware parts images from multiple angles. The defect identification module is used to identify flow mark stripe areas in the hardware part image set and determine the severity of flow marks in the flow mark stripe areas; hardware parts with flow mark severity greater than a preset severity threshold are marked as unqualified hardware parts, and the PLC-controlled rejection device automatically sorts the unqualified hardware parts.

[0007] The beneficial effects of this invention are as follows: I. By establishing a visual inspection workstation downstream of the stamping production line, and combining it with an automatic transfer track, electric clamps, photoelectric sensors, industrial cameras, and high-performance light sources, the system achieves fully automated inspection of flow mark defects on metal parts. The system can accurately locate the position of the metal parts and automatically complete multi-angle image acquisition, ensuring that the inspection results do not rely on human experience. This greatly improves the efficiency, standardization, and consistency of inspection, effectively avoiding human error and omissions.

[0008] Second, a combination of a ring light source and an oblique strip light source is used, and a stable brightness is achieved in advance by turning on the first light source before camera exposure, ensuring uniform illumination and clear texture in the image. At the same time, a miniature rotating stage equipped with a servo motor allows the camera to acquire images from multiple angles (0°, 30°, 60°, 90°) sequentially, realizing multi-dimensional visual modeling of flow marks on the surface of hardware parts, improving the comprehensiveness and accuracy of inspection, and is especially suitable for parts with complex curved surface structures.

[0009] Third, by introducing multiple physical quantities such as grayscale gradient change rate, strip width, length, edge sharpness, and strip number, a three-level (minor, moderate, and severe) flow mark grading evaluation system was established to achieve quantitative judgment of defects. This method provides a traceable, verifiable, and adjustable defect judgment standard, facilitating subsequent quality traceability and process optimization, and further improving the scientific nature and transparency of product quality control.

[0010] Fourth, the system employs a PLC control architecture to uniformly schedule the light source, camera, sensor, and rejection device. Through preset high-speed input / output logic, it achieves automatic linkage of key processes such as photoelectric sensing, camera triggering, and rejection judgment. Simultaneously, the system features an adaptive positioning correction mechanism for hardware component positional deviations, making it suitable for hardware components of different sizes and shapes. It possesses good versatility and scalability, facilitating rapid deployment and upgrades across multiple production lines. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the steps in a visual inspection method for stamping defects in hardware parts based on image technology. Figure 2 for Figure 1 A detailed flowchart illustrating the implementation steps of step S3. Figure 3 This is a physical image of flow marks on the surface of a hardware part, which is a visual inspection method for stamping defects of hardware parts based on image technology. Figure 4 This is a visual inspection scene diagram for a visual inspection method of stamping defects in hardware parts based on image technology. The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0012] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0013] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0014] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0015] To achieve the above objectives, please refer to Figures 1 to 4 A visual inspection method for stamping defects in hardware parts based on image technology includes a visual inspection workstation, an automatic transfer track, a miniature rotary table, and a PLC-controlled rejection device. The PLC-controlled rejection device has a built-in high-speed input module connected to the PLC program. The visual inspection workstation has a built-in laser alignment module and an automatic positioning device. The method includes the following steps: Step S1: Establish a vision inspection workstation downstream of the stamping production line, and set up the vision inspection workstation and automatic transfer track. Use the automatic transfer track to send each stamped metal part into the inspection area of ​​the vision inspection workstation. In one embodiment, reference Figure 3 as well as Figure 4 To achieve automated quality inspection of stamped hardware parts, a vision inspection workstation is installed at the end of the stamping production line. The vision inspection workstation includes an industrial camera for image acquisition, a ring light source for illumination, an image processing module, and a data transmission and control unit connected thereto.

[0016] To achieve continuous inspection without human intervention, an automatic transfer track is installed between the stamping equipment and the vision inspection workstation. The automatic transfer track can take the form of a belt conveyor, a slide rail pushing mechanism, or a robotic arm gripping and transferring structure, etc. Preferably, a linear motor transfer device equipped with a positioning and limiting structure is used to ensure the stability of the hardware parts during transport and to prevent rotation and deviation.

[0017] After each stamping operation completes the forming of a metal part, the automatic transport track starts under preset timing control, precisely conveying the metal part to the inspection area in the vision inspection workstation. To ensure the stability and consistency of image acquisition, a fixed fixture or support platform is set below the inspection area to position the metal part, and an industrial camera is positioned directly above it or at a specific angle to capture the key forming areas of the metal part.

[0018] Step S2: Fix the hardware in the inspection area using an electric clamp; install an industrial camera on the fixed hardware and configure a combination of a ring light source and a slanted strip light source; In one embodiment, considering that the hardware may experience slight shaking or displacement during transport to the inspection area, an electric clamp is installed in the inspection area of ​​the visual inspection workstation to quickly and accurately fix the hardware in order to ensure positional stability and image consistency during the imaging process. The electric clamp preferably employs a double-sided symmetrical clamping structure, which can automatically adjust the clamping force according to the size and shape of the hardware, avoiding displacement due to insufficient clamping or damage to the hardware due to excessive clamping.

[0019] The electric clamp is linked to the transport track. After the hardware part enters the inspection area and completes its initial positioning, the clamp initiates a clamping action based on sensor feedback signals, firmly fixing the hardware part to the standard position on the inspection platform. This fixed position has been pre-calibrated according to the focal length and field of view of the industrial camera to ensure that the acquired images have good geometric consistency and resolution.

[0020] An industrial camera is mounted directly above the fixed hardware to capture high-definition images. To improve image clarity and the recognizability of feature edges, a combined light source system is configured around the industrial camera, comprising a ring light source and oblique strip light sources.

[0021] Among them, the ring light source is installed around the industrial camera lens to provide uniform, shadowless vertical illumination, which is suitable for detecting vertical reflection features such as surface scratches and uneven defects; the oblique strip light source is arranged on both sides of the detection area, illuminating the surface of the hardware at a certain angle, enhancing the contrast between light and dark of the edge contour and minor deformation of the parts, which is beneficial for the subsequent image processing module to identify defects such as contour offset and breakage.

[0022] Step S3: Detect the position signal of the hardware part by the photoelectric sensor on the vision inspection workstation, and trigger the camera to acquire images of the hardware part under the light source combination to obtain a set of hardware part images from multiple angles; In one embodiment, to achieve automated control and precise timing coordination of image acquisition, at least one photoelectric sensor is configured in the visual inspection workstation to monitor in real time whether the hardware component accurately reaches the inspection area. The photoelectric sensor preferably adopts a through-beam or reflective structure and is installed on one side or above / below the inspection area. When the hardware component enters the designated inspection position, the sensor emits a positioning signal, which serves as the trigger condition for subsequent camera image acquisition.

[0023] Based on the precise fixing of the hardware parts by the aforementioned electric clamp, the control module will immediately start the industrial camera's acquisition task after receiving the positioning signal from the photoelectric sensor. At this time, the pre-set light source combination will be turned on simultaneously, including a ring light source and an oblique strip light source, to ensure that the illumination intensity and direction during the imaging process meet the detection requirements.

[0024] To obtain more comprehensive inspection information, the system can control the industrial camera or the hardware component itself to adjust its posture, allowing the camera to acquire images of the hardware component from multiple angles and construct a multi-angle image dataset. For example, a rotating platform combined with a fixed camera can be used to rotate the hardware component around its vertical axis to acquire images from different sides; or a multi-camera arrangement can be used, where multiple industrial cameras simultaneously capture images from different angles while the hardware component remains stationary, thereby simultaneously acquiring frontal views, side views, oblique angle views, and other images.

[0025] The image acquisition can be set to continuous frame mode or fixed frame trigger mode. The specific number of frames and angles acquired depend on the complexity of the hardware structure and the inspection requirements. After acquisition, the image data will be sent to the image processing module for subsequent defect identification and parameter analysis.

[0026] Step S4: Identify the flow mark stripe area in the hardware image set and determine the severity of the flow mark stripe area; mark the hardware with the flow mark severity greater than the preset severity threshold as unqualified hardware, and automatically sort the unqualified hardware by the PLC-controlled rejection device.

[0027] In one embodiment, the multi-angle hardware image set acquired in step S3 can be preprocessed, including denoising, contrast enhancement, and edge extraction, to improve the visual clarity of the flow mark stripes. Subsequently, a deep learning-based target detection algorithm or a traditional image segmentation method (such as a threshold-based region growing algorithm, or an algorithm combining edge detection and morphological processing) is used to accurately identify the flow mark stripe region.

[0028] After identifying the flow mark streaks, the system further extracts parameters such as texture features, grayscale distribution, and area size of the region. Combined with a pre-trained evaluation model or expert rules, the severity index of the flow marks is quantified. This index comprehensively considers the length, width, concentration, and coverage of the flow marks, thus reflecting the degree of impact of the flow marks on the surface quality of the hardware. A preset severity threshold is determined based on the production process and quality standards. If the severity index of a hardware component exceeds this threshold, the system automatically classifies the component as a defective product. This judgment result, along with the unique identification information of the corresponding hardware component, is transmitted to the PLC control unit.

[0029] After receiving the non-conforming mark, the PLC control unit issues a rejection command in real time according to the linkage control program with the rejection device, driving the robotic arm or air blowing device to remove the non-conforming hardware from the conveyor track, thus completing the automatic sorting operation.

[0030] As an example of the present invention, reference is made to Figure 2 As shown, step S3 in this example includes: Step S31: Fix a pair of through-beam photoelectric sensors at the entrance of the visual inspection workstation, wherein the transmitting end and receiving end of the through-beam photoelectric sensors are horizontally aligned and the beam height is level with the upper edge of the hardware. Step S32: Adjust the spacing between the through-beam photoelectric sensors; connect the output signal of the through-beam photoelectric sensors to the high-speed input module of the PLC-controlled rejection device, and set the confirmation logic for the position signal in the PLC program; Step S33: After receiving the photoelectric sensor's arrival signal, the PLC program outputs two control signals in succession, including a first light source turn-on signal and a second camera trigger signal. Step S34: Based on the first light source activation signal and the second camera trigger signal, perform multi-angle image acquisition on the hardware to obtain a multi-angle image set of the hardware.

[0031] In one embodiment, a high-sensitivity through-beam photoelectric sensor is selected. The transmitting and receiving ends form a detection channel through an infrared or laser beam. The installation position is precisely adjusted so that the beam height is aligned with the upper edge of the stamped metal part, ensuring that the beam is accurately cut off when the metal part enters the detection area, thereby generating a positioning signal. The sensor is fixed with a shockproof bracket to ensure stability during long-term operation.

[0032] After installation, the sensor spacing is adjusted to cover the maximum width of the entire hardware component, preventing signal detection errors caused by variations in component dimensions. The sensor output signals are transmitted to the high-speed input module of the PLC controller via shielded cables to ensure real-time and accurate signal acquisition. Logical judgment conditions are set in the PLC program; a signal interruption (beam beam blocked) must be continuously and stably detected for a certain period before confirming the hardware component is accurately in place, avoiding misjudgments due to malfunctions. After receiving the confirmed hardware component placement signal, the PLC outputs the first light source activation signal sequentially according to a preset time sequence, activating the first light source (such as a ring light) configured in the vision inspection workstation to ensure uniform and sufficient illumination of the hardware component surface. Subsequently, a second camera trigger signal is output to drive the industrial camera to begin image acquisition. Both signals are connected to the light source and camera control units via relays or solid-state relays to ensure safe and stable signal transmission.

[0033] The visual inspection workstation initiates a multi-angle acquisition scheme based on control signals. Multi-angle acquisition can be achieved by controlling a rotating platform to adjust the angle of the hardware component, or by using a multi-camera system to simultaneously capture images, providing comprehensive coverage of the hardware component's surface from different perspectives. The image acquisition system synchronously adjusts the brightness and angle of the light source to ensure stable image quality free from interference such as shadows and reflections. The acquired multi-angle images are transmitted in real-time to the image processing module for subsequent defect detection and quality analysis.

[0034] In another embodiment, for example, on a stamping hardware production line, a pair of OMRON E3JK-5DM1 through-beam photoelectric sensors are installed at the entrance of the vision inspection workstation. The transmitter and receiver of these sensors are mounted horizontally, adjusted to a height of 500mm from the conveyor platform, level with the upper edge of the hardware component, to ensure the beam is accurately blocked when the hardware component reaches the inspection area. The photoelectric sensor spacing is adjusted to 300mm to cover hardware components with a maximum width of 280mm, ensuring effective inspection of hardware components of different sizes. The sensor output is an NPN normally open switching signal, connected via shielded twisted-pair cable to the high-speed input module of a Siemens S7-1500 series PLC, with an input signal sampling period of 1ms. The PLC program is configured to determine that the hardware component is in position if the photoelectric sensor signal interruption lasts for more than 10ms, filtering out mechanical jitter and false triggering. Upon receiving the arrival signal, the PLC first outputs a control signal to turn on the Hikvision ring LED light source in the vision inspection workstation. The light source brightness is set to 1200 lumens, ensuring uniform and flicker-free illumination. Then, after a 20ms delay, the PLC outputs an industrial camera trigger signal, driving the Basler ace acA1920-40gm industrial camera mounted directly above the inspection area to begin image acquisition. Camera parameters are set to 1920×1200 resolution, 800μs exposure time, and 40fps frame rate. To achieve multi-angle acquisition, the hardware is fixed on a rotary servo motor-driven turntable. The turntable control program, synchronously issuing rotation commands from the PLC, rotates the hardware every 45°, acquiring images from eight angles. After each angle acquisition, the industrial camera trigger signal is activated by the PLC every 500ms to ensure clear and blur-free images. The acquired multi-angle images are transmitted in real-time to an image processing server equipped with an NVIDIA RTX3060 GPU for subsequent flow mark detection and severity assessment.

[0035] Preferably, the configuration process for the first light source activation signal and the second camera trigger signal includes: Configure the light source combination according to the first light source activation signal: drive the drivers of the ring light source and the oblique strip light source to ensure that the light source reaches the preset stable brightness 5–10ms before the camera exposure; Configure the camera according to the second camera trigger signal: set the exposure time to 500–1000µs, the gain to 0–12dB, the trigger mode to hardware trigger single frame, and send a single pulse through the camera's external trigger interface to trigger the camera to start acquisition.

[0036] In one embodiment, the configuration process of the first light source activation signal and the second camera trigger signal includes: configuring the light source combination according to the first light source activation signal, specifically by driving the drivers of the ring light source and the oblique strip light source to ensure that the light source reaches a preset stable brightness 5–10 ms before camera exposure. This process uses PLC to control the switching output of the light source drivers to achieve timely lighting and stable brightness of the light source, avoiding image quality fluctuations caused by insufficient brightness at the moment of light source startup. The camera is configured according to the second camera trigger signal, specifically by setting the exposure time of the industrial camera to 500–1000 microseconds (µs), adjusting the gain to the range of 0–12 dB, and setting the trigger mode to hardware trigger single-frame mode. The camera receives a single-pulse signal from the PLC through an external trigger interface, triggering the camera to complete single-frame image acquisition, ensuring the consistency and stability of the acquired image.

[0037] In practical applications, the PLC can use a Siemens S7-1500 series controller, which sends an activation signal to the Hikvision ring LED light source driver via its digital output module. The PLC program logic ensures that the light source activation signal is sent 5ms in advance. Both the ring light source and the angled strip light source drivers receive this signal, and the light reaches a preset brightness stable state within this time. Simultaneously, the industrial camera can be a Basler ace acA1920-40gm model. The camera configuration software sets the exposure time to 800µs, the gain to 8dB, and the trigger mode to external hardware trigger single-frame mode. The PLC digital output module sends a single-pulse signal to the camera's external trigger interface via a dedicated trigger line, triggering the camera to complete a high-quality image acquisition. In the PLC program design, the first light source activation signal is sent, followed by a 10ms wait to ensure light source stabilization, and then the second camera trigger signal is sent. This achieves optimal lighting conditions during image acquisition, ensuring the image is free of blur and underexposure, meeting the requirements for subsequent defect detection.

[0038] Preferably, the multi-angle image acquisition of the hardware based on the first light source activation signal and the second camera trigger signal includes: The servo motor on the miniature rotary table is driven by the first light source activation signal and the second camera trigger signal, so that it rotates the camera sequentially according to 0°, 30°, 60° and 90°. At each angle, the PLC outputs two control signals again and executes step S34.

[0039] In one embodiment, a servo motor mounted on a miniature rotary table is driven by a first light source activation signal and a second camera trigger signal, causing the rotary table to rotate sequentially at a preset angle sequence of 0°, 30°, 60°, and 90°. Whenever the rotary table rotates to a designated angle position, the PLC controller outputs two control signals, triggering the light source activation and camera acquisition respectively, executing the multi-angle image acquisition operation in step S34. This method achieves comprehensive imaging of hardware parts from different angles, enhancing the coverage and accuracy of defect detection.

[0040] In practical applications, a miniature rotary table driven by a Panasonic MSMD022P1S servo motor can be used. The rotary table is mounted on the inspection platform of the vision inspection workstation, supporting the metal parts being inspected. The PLC uses a Siemens S7-1500 series controller. The PLC program presets a rotation angle sequence of 0°, 30°, 60°, and 90°, sequentially controlling the servo driver via pulse commands to precisely position the rotary table at these angles. After the rotary table is positioned, the PLC sequentially outputs a first light source activation signal, ensuring the light source illuminates 10ms in advance. Then, it outputs a second camera trigger signal, driving a Basler ace acA1920-40gm industrial camera mounted above the inspection platform to complete single-frame image acquisition. After each angle image acquisition is completed, the PLC controls the rotary table to rotate to the next angle, repeating the light source activation and camera triggering process until all preset angles are acquired. The entire process features high control precision, fast acquisition efficiency, and ensures the multi-angle integrity and acquisition quality of the image data.

[0041] Preferably, in step S4, identifying the flow mark stripe area in the hardware part image and determining the severity of the flow mark stripe area includes: For any image of a hardware component, calculate the pixel gradients in the horizontal and vertical directions. The gradient magnitude and direction of each pixel are determined based on the pixel gradient. For each pixel, within its neighborhood window, count the number of pixels with the same direction as the pixel and whose gradient difference is less than the threshold, and obtain the pixels with the same direction. Calculate the texture intensity of each pixel in the hardware part image, and filter candidate strip points in the hardware part image by combining pixels with consistent orientation; Identify the flow mark areas in the hardware part image based on the candidate strip points, and determine the severity of the flow marks in the flow mark areas.

[0042] In one embodiment, for any image of a hardware component, the pixel gradients in the horizontal and vertical directions are first calculated. Commonly used operators include the Sobel operator or the Prewitt operator, which are used to extract image edge and texture variation features. Based on the calculated horizontal and vertical gradients, the gradient magnitude (i.e., gradient strength) and gradient direction (angle) of each pixel are determined. The gradient magnitude reflects the edge strength of the pixel, and the gradient direction reflects the direction of the texture or stripes.

[0043] For each pixel, the number of pixels within its neighborhood window (e.g., a 3×3 or 5×5 pixel range) that share the same gradient direction and have a gradient magnitude difference less than a preset threshold is counted; these are called orientation-consistent pixels. This statistic helps capture texture continuity and filter out noise interference. The texture intensity of each pixel in the image is calculated, which can be obtained based on statistical features such as the mean local gradient magnitude or the gray-level co-occurrence matrix (GLCM). Combining the distribution of orientation-consistent pixels, candidate stripe points that meet the requirements of texture intensity and orientation consistency are selected; these points are initially identified as regions where flow mark stripes may exist.

[0044] Based on spatial clustering and connectivity analysis of candidate strip points, the boundary range of the flow mark strip region is determined. Then, by quantifying indicators such as the mean gradient intensity, area size, and connectivity length within this region, the severity index of the flow mark is calculated. The severity of the flow mark is compared with a preset threshold to complete the quantitative evaluation of the flow mark quality on the surface of the hardware part.

[0045] In a practical system, the above process can be implemented using Python combined with the OpenCV library. Taking an image of a hardware part as an example, the horizontal and vertical gradients are calculated using a 3×3 Sobel operator to obtain the gradient magnitude and direction of each pixel. For each pixel, a neighborhood window of 5×5 is defined, and the threshold is set to a gradient magnitude difference of no more than 20 (grayscale units). The number of pixels in the neighborhood with a directional difference of no more than 15 degrees and a magnitude difference that meets the threshold is counted. Texture intensity is obtained by calculating the average local gradient magnitude, and pixels with a texture intensity greater than 50 and more than 12 pixels with the same direction are selected as candidate stripe points. Connectivity analysis is performed on the candidate stripe points, and the largest connected region is extracted as the flow mark stripe region. Its area and average gradient intensity are calculated as severity indicators. The preset severity threshold is an area greater than 1000 pixels and an average gradient intensity greater than 60. Hardware parts exceeding this threshold are marked as having severe flow marks.

[0046] Preferably, the flow mark stripe regions in the hardware part image are identified based on candidate stripe points, including: For each candidate strip point, extract the pixel neighborhood within a preset range centered on it; Based on the extension direction of the candidate strip points, extract several adjacent slice images at equal intervals in the pixel neighborhood; In each adjacent slice image, the candidate strip point at the corresponding position is located, and its gray value and brightness value are recorded as the intensity temporal sequence of the pixel point as the image sequence changes; The change amplitude of the intensity temporal sequence of each candidate strip point is calculated to obtain the temporal stability value of the pixel intensity change; Sort and filter the stability values ​​of all candidate strip points to obtain a set of stable candidate points; Connect spatially adjacent stable candidate points to form a stable strip region candidate zone; The flow mark stripe region is identified by detecting whether the candidate stripe in the stable stripe region is consistent with the trend of the extension direction of the candidate stripe points.

[0047] In one embodiment, for each candidate stripe point, a pixel neighborhood (e.g., 7×7 or 11×11 pixels) within a preset range is extracted as the local analysis region for that point. Based on the extension direction of the candidate stripe point (determined by its gradient direction), several adjacent slice images (i.e., several parallel pixel rows or columns along the extension direction) are extracted at equal intervals within this pixel neighborhood to observe the spatial continuity characteristics of the pixel. The candidate stripe point at the corresponding position is located in each adjacent slice image, and its grayscale and brightness values ​​are recorded to form an intensity temporal sequence of the pixel's variation with the image sequence.

[0048] For each candidate strip point, calculate the magnitude of change in intensity over time (e.g., the difference between the maximum and minimum values, standard deviation, etc.) to obtain the temporal stability value of the intensity change of that pixel. Sort and filter all candidate strip point stability values, removing points with drastic temporal changes and poor stability to obtain a stable candidate point set.

[0049] Spatially adjacent stable candidate points are connected through adjacency relationships to form stable stripe region candidate bands. The overall extension direction of the stable stripe region candidate band is checked to see if it matches the original extension direction trend of the candidate stripe points. If they match, the stable stripe region is confirmed as a flow mark stripe region in the hardware part image.

[0050] In practical applications, assuming an image of a hardware component has a resolution of 1920×1200 pixels, candidate strip points are obtained through prior gradient and direction filtering. For each candidate point, an 11×11 pixel neighborhood can be extracted with the center point as the core. Based on the gradient direction θ of this point, the neighborhood is divided into 5 adjacent slices (slice width is 3 pixels) along the θ direction, and the pixel values ​​at the corresponding positions in these slices are sampled respectively.

[0051] Grayscale values ​​are collected to form an intensity time series, and the standard deviation σ of the series is calculated as a time series stability index. All candidate points are sorted in ascending order of σ, and pixels with a standard deviation less than 15 are selected to form a stable candidate point set. An 8-neighborhood connectivity algorithm is used to connect adjacent stable candidate points into connected regions, filtering out isolated regions with an area less than 50 pixels, and retaining larger connected blocks as candidate bands for stable strip regions.

[0052] Principal component analysis (PCA) is used to calculate the principal direction of candidate bands in stable strip regions, and the angle between the direction and the average gradient direction of the original candidate band points is compared. If the angle is less than 20°, the direction is considered to be consistent, and the region is confirmed as an effective flow mark strip region.

[0053] Preferably, the indicators for judging the severity of flow marks in the flow mark streak area include: If the grayscale gradient change rate of the flow mark strip area is less than 15 grayscale values / mm; the strip width is less than or equal to 2.0mm and the continuous strip length is less than or equal to 15mm; the edge sharpness is less than or equal to 10 contrast units / mm; and the number of flow mark strips in the same detection surface is less than or equal to 3, then the flow mark strip area is judged as a slight flow mark. If the grayscale gradient change rate of the flow mark strip area is between 15 and 35 grayscale values / mm; the strip width is between 2.0 mm and 5.0 mm, or the continuous strip length is between 15 mm and 30 mm; the edge sharpness is between 10 and 20 contrast units / mm; and the number of flow mark stripes in the same detection surface is between 4 and 6, then the flow mark strip area is judged as a moderate flow mark. If the grayscale gradient change rate of the flow mark strip area is greater than 35 grayscale values / mm; the strip width is greater than 5.0mm and the continuous length of the strip exceeds 30mm; the edge sharpness is greater than 20 contrast units / mm; and the number of flow mark stripes in the same detection surface is more than 6, then the flow mark strip area is judged as a severe flow mark.

[0054] In one embodiment, if the grayscale gradient change rate of the flow mark strip area is less than 15 grayscale values / mm; the strip width is less than or equal to 2.0 mm, and the continuous strip length is less than or equal to 15 mm; the edge sharpness is less than or equal to 10 contrast units / mm; and the number of flow mark strips in the same detection surface is less than or equal to 3, then the flow mark strip area is determined to be a slight flow mark. If the grayscale gradient change rate of the flow mark strip area is between 15 and 35 grayscale values / mm; the strip width is between 2.0 mm and 5.0 mm, or the continuous strip length is between 15 mm and 30 mm; the edge sharpness is between 10 and 20 contrast units / mm; and the number of flow mark strips in the same detection surface is between 4 and 6, then the flow mark strip area is determined to be a moderate flow mark. If the grayscale gradient change rate of the flow mark strip area is greater than 35 grayscale values / mm; the strip width is greater than 5.0mm and the continuous length of the strip exceeds 30mm; the edge sharpness is greater than 20 contrast units / mm; and the number of flow mark stripes in the same detection surface is more than 6, then the flow mark strip area is judged as a severe flow mark.

[0055] In the visual inspection system for hardware parts, the grayscale gradient change rate of the acquired flow mark stripe image area is calculated. This is done by converting the image pixel grayscale change to the actual physical size, with the unit being grayscale value / mm. The stripe width and continuous length are measured using image processing algorithms. The stripe width is measured in pixels and combined with camera calibration to obtain millimeters, while the continuous length is calculated from the length of the connected region of the stripe pixels. The edge sharpness index is calculated based on the grayscale contrast change rate of the stripe edge, with the unit being contrast units / mm. The calculation method is the average of the absolute values ​​of the grayscale gradient at the stripe edge. The number of flow mark stripes within the same inspection surface is obtained statistically through image segmentation techniques and connected component analysis.

[0056] In another embodiment, for example, a certain strip region has a grayscale gradient change rate of 12 grayscale values / mm, a strip width of 1.8mm, a continuous length of 14mm, an edge sharpness of 8 contrast units / mm, and a total of 2 flow mark stripes within the detection surface, thus it is judged as a slight flow mark. Another strip region has a grayscale gradient change rate of 25 grayscale values / mm, a strip width of 3.5mm, a continuous length of 20mm, an edge sharpness of 15 contrast units / mm, and a total of 5 flow mark stripes within the detection surface, thus it is judged as a moderate flow mark. Another strip region has a grayscale gradient change rate of 40 grayscale values / mm, a strip width of 6.2mm, a continuous length of 35mm, an edge sharpness of 25 contrast units / mm, and a total of 7 flow mark stripes within the detection surface, thus it is judged as a severe flow mark.

[0057] Preferably, connecting spatially adjacent stable candidate points includes: For each stable candidate point, examine its neighboring pixels in 8 directions. If a neighboring pixel is also a stable candidate point, it means that the two points are spatially adjacent. The 8 directions include up and down, left and right, and the four diagonal directions. Starting from any stable candidate point, find all stable candidate points that are directly or indirectly adjacent to it and group them into the same connected region; For all stable candidate points, check whether they have been assigned to a connected region. If not, establish a new connected region starting from that point using the method described above. Each connected region is a candidate stable strip region.

[0058] In one embodiment, for each stable candidate point, its neighboring pixels in eight directions are checked. These eight directions include up and down, left and right, and the four diagonal directions. If a neighboring pixel is also a stable candidate point, then the stable candidate point is considered to be spatially adjacent to its neighboring pixel. Starting from any stable candidate point that has not been assigned a connected region, all stable candidate points that are directly or indirectly adjacent to it are recursively searched and these points are assigned to the same connected region. All stable candidate points are traversed. If a point has not yet been assigned a connected region, the above connected region search process is repeated with that point as the starting point to establish a new connected region. Each connected region forms a stable strip region candidate strip.

[0059] In practical systems, a pixel-based 8-neighborhood connectivity algorithm can be used to achieve spatial connectivity of stable candidate points. The set of stable candidate points is stored in the form of a binary graph, where "1" represents a stable candidate point and "0" represents a non-candidate point. For each pixel with a value of "1" in the graph, the values ​​of pixels in its eight surrounding directions are checked sequentially to see if they are also "1". If they are, a connection is established. A depth-first search (DFS) or breadth-first search (BFS) algorithm is used to traverse the image to find all connected stable candidate point sets, marking them as the IDs of the same connected region. For example, this can be achieved using Python combined with OpenCV. The function performs connected component analysis to obtain the number and size of each stable stripe candidate region. All connected regions are then filtered, with isolated regions smaller than 50 pixels being removed, and larger regions retained for subsequent flow mark stripe analysis. This method ensures the effective organization of the spatial relationships of stable candidate points, providing a foundation for subsequent stripe morphology feature extraction and severity determination.

[0060] Preferably, before fixing the hardware in the inspection area using an electric clamp, the process further includes: The laser alignment module scans and identifies the detection area to determine whether the hardware currently placed in the detection area is accurately in the preset detection reference position. The obtained hardware position information is compared with the preset detection benchmark position. If the result indicates that there is a positional deviation in the hardware, the automatic positioning device is activated to correct the position of the hardware.

[0061] In one embodiment, in a visual inspection workstation on a stamping production line, a Keyence IL-3000 series laser displacement sensor module can be used as a laser alignment module, installed directly above the inspection area to scan the surface of the hardware parts. The position information of the hardware parts collected by the laser module is transmitted in real time to a Siemens S7-1500 series PLC via an industrial Ethernet network. The PLC has preset detection reference position coordinates (X0, Y0, Z0). The PLC program compares the position measured by the laser module with the reference position. If a position deviation in the X or Y direction is detected to exceed ±0.5mm, it is determined that there is a position deviation in the hardware parts. After the position deviation is confirmed, the PLC drives the high-precision stepper motor automatic positioning device installed under the conveyor table to start, perform fine-tuning actions, adjust the position of the conveyor track, and restore the position of the hardware parts to the preset reference. After the position correction is completed, the PLC triggers the laser module to scan and confirm again. After confirmation, it sends a clamping command to the electric clamp to complete the fixing of the hardware parts.

[0062] Most importantly, determining whether the hardware currently placed in the testing area is accurately positioned at the preset testing reference position also includes: Based on the laser alignment module, spatial scanning data within the detection area is collected, and the boundary contour features of the hardware are extracted to generate hardware contour feature data. Calculate the geometric deviation between the current posture of the hardware and the preset detection benchmark based on the hardware contour feature data, and generate posture deviation calculation data. The attitude deviation calculation data is compared and analyzed with thresholds to determine whether the position, angle and rotation axis of the hardware parts exceed the tolerance range, and the deviation judgment result data is generated. Based on the deviation judgment result, the system outputs a positioning pass / fail message or a deviation warning.

[0063] In one embodiment, a three-dimensional spatial scanning data of the detection area is acquired based on a laser alignment module. A multi-line laser contour sensor (such as the Keyence LJ-X8000 series) performs a high-resolution scan of the hardware component to obtain its boundary contour features within the detection area, generating hardware component contour feature data. Image analysis algorithms are used to fit the contour feature data, extracting key boundary segments, positioning holes, bends, or curvature edges of the hardware component. Based on the detection reference posture defined in the CAD template or reference drawing, the geometric deviation between the current hardware component posture and the reference is calculated, yielding a result including positional offset. Attitude deviation calculation data, including angle deviations (Pitch, Yaw, Roll).

[0064] Multidimensional tolerance analysis is performed on the attitude deviation calculation data to compare whether it exceeds the set thresholds (e.g., ±0.5mm position tolerance, ±1.0° angle tolerance), generating deviation judgment result data, including: position qualified / offset, whether the attitude rotation angle exceeds the tolerance, and whether the rotation axis deviates from the design direction. Based on the deviation judgment result data, the system outputs prompts: if all deviations are within the tolerance range, a "positioning qualified" signal is output; if any deviation exceeds the tolerance, a "deviation exceeds limit" prompt is output, driving the automatic positioning module to correct the deviation, or triggering an alarm to prompt manual intervention.

[0065] Of particular importance, starting from any stable candidate point, finding all stable candidate points that are directly or indirectly adjacent to it also includes: Starting from any stable candidate point, randomly select an unvisited stable candidate point as the current starting seed point to generate stable candidate seed point data; Based on the stable candidate seed point data, the adjacency of stable candidate points within its neighborhood is determined, an initial set of adjacency points is constructed, and initial adjacency mapping data is generated. Based on the initial adjacency mapping data, recursively search for all stable candidate points that are directly or indirectly connected to the current seed point, and continuously expand the adjacency set to generate connected region expansion data. The extended data of the connected regions are uniquely identified and included in a unified set of connected regions to generate the current connected region data. At the same time, the set of unprocessed points is updated to prepare for the search of the next connected region.

[0066] In one embodiment, from the set of stable candidate points, a stable candidate point that has not yet been visited is randomly selected as the current starting seed point, its two-dimensional coordinate position (x0, y0) is recorded, and stable candidate seed point data SeedPoint(x0, y0) is generated. An eight-neighborhood (up, down, left, right + four diagonals) is defined, and the surrounding neighborhood of the seed point SeedPoint is scanned to determine whether the surrounding pixels are also stable candidate points; if so, the adjacent point is added to the initial adjacent point set N0, generating initial adjacency mapping data. .

[0067] Based on a breadth-first search (BFS) or depth-first search (DFS) strategy, starting from the current seed point, adjacency determination and adjacency mapping construction are repeatedly performed for each adjacent point, recursively expanding the set of adjacent points until it cannot be expanded further, generating connected region extended data ConnectedSet_i. The expanded connected region ConnectedSet_i is assigned a unique identifier RegionID_i, and this set of data is included in a unified connected region set AllRegions. Simultaneously, all points within this region are deleted from the stable candidate point set, and the unprocessed point set RemainingPoints is updated to prepare for the next seed point selection.

[0068] Repeat the above steps until all stable candidate points have been processed.

[0069] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0070] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A visual inspection method for stamping defects in hardware parts based on image technology, characterized in that, The method comprises a visual inspection workstation, an automatic transfer track, a miniature rotary table, and a PLC-controlled rejection device. The PLC-controlled rejection device has a built-in high-speed input module connected to the PLC program. The visual inspection workstation has a built-in laser alignment module and an automatic positioning device. The method includes the following steps: Step S1: Establish a vision inspection workstation downstream of the stamping production line, and set up the vision inspection workstation and automatic transfer track. Use the automatic transfer track to send each stamped metal part into the inspection area of ​​the vision inspection workstation. Step S2: Fix the hardware in the inspection area using an electric clamp; install an industrial camera on the fixed hardware and configure a combination of a ring light source and a slanted strip light source; Step S3: Detect the position signal of the hardware part by the photoelectric sensor on the vision inspection workstation, and trigger the camera to acquire images of the hardware part under the light source combination to obtain a set of hardware part images from multiple angles; Step S4: Identify the flow mark stripe area in the hardware image set and determine the severity of the flow mark stripe area; mark the hardware with the flow mark severity greater than the preset severity threshold as unqualified hardware, and automatically sort the unqualified hardware by the PLC-controlled rejection device.

2. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Fix a pair of through-beam photoelectric sensors at the entrance of the visual inspection workstation, wherein the transmitting end and receiving end of the through-beam photoelectric sensors are horizontally aligned and the beam height is level with the upper edge of the hardware. Step S32: Adjust the spacing between the through-beam photoelectric sensors; connect the output signal of the through-beam photoelectric sensors to the high-speed input module of the PLC-controlled rejection device, and set the confirmation logic for the position signal in the PLC program; Step S33: After receiving the photoelectric sensor's arrival signal, the PLC program outputs two control signals in succession, including a first light source turn-on signal and a second camera trigger signal. Step S34: Based on the first light source activation signal and the second camera trigger signal, perform multi-angle image acquisition on the hardware to obtain a multi-angle image set of the hardware.

3. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 2, characterized in that, The configuration process for the first light source activation signal and the second camera trigger signal includes: Configure the light source combination according to the first light source activation signal: drive the drivers of the ring light source and the oblique strip light source to ensure that the light source reaches the preset stable brightness 5–10ms before the camera exposure; Configure the camera according to the second camera trigger signal: set the exposure time to 500–1000µs, the gain to 0–12dB, the trigger mode to hardware trigger single frame, and send a single pulse through the camera's external trigger interface to trigger the camera to start acquisition.

4. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 2, characterized in that, Multi-angle image acquisition of hardware components based on the first light source activation signal and the second camera trigger signal includes: The servo motor on the miniature rotary table is driven by the first light source activation signal and the second camera trigger signal, so that it rotates the camera sequentially according to 0°, 30°, 60° and 90°. At each angle, the PLC outputs two control signals again and executes step S34.

5. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 1, characterized in that, Step S4 involves identifying flow mark band areas in the hardware part image and determining the severity of flow marks in these areas, including: For any image of a hardware component, calculate the pixel gradients in the horizontal and vertical directions. The gradient magnitude and direction of each pixel are determined based on the pixel gradient. For each pixel, within its neighborhood window, count the number of pixels with the same direction as the pixel and whose gradient difference is less than the threshold, and obtain the pixels with the same direction. Calculate the texture intensity of each pixel in the hardware part image, and filter candidate strip points in the hardware part image by combining pixels with consistent orientation; Identify the flow mark areas in the hardware part image based on the candidate strip points, and determine the severity of the flow marks in the flow mark areas.

6. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 5, characterized in that, Based on the candidate stripe points, the flow mark stripe regions in the hardware part image were identified as including: For each candidate strip point, extract the pixel neighborhood within a preset range centered on it; Based on the extension direction of the candidate strip points, extract several adjacent slice images at equal intervals in the pixel neighborhood; In each adjacent slice image, the candidate strip point at the corresponding position is located, and its gray value and brightness value are recorded as the intensity temporal sequence of the pixel point as the image sequence changes; The change amplitude of the intensity temporal sequence of each candidate strip point is calculated to obtain the temporal stability value of the pixel intensity change; Sort and filter the stability values ​​of all candidate strip points to obtain a set of stable candidate points; Connect spatially adjacent stable candidate points to form a stable strip region candidate zone; The flow mark stripe region is identified by detecting whether the candidate stripe in the stable stripe region is consistent with the trend of the extension direction of the candidate stripe points.

7. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 5, characterized in that, The indicators for judging the severity of flow marks in flow mark streaks include: If the grayscale gradient change rate of the flow mark strip area is less than 15 grayscale values / mm; the strip width is less than or equal to 2.0mm and the continuous strip length is less than or equal to 15mm; the edge sharpness is less than or equal to 10 contrast units / mm; and the number of flow mark strips in the same detection surface is less than or equal to 3, then the flow mark strip area is judged as a slight flow mark. If the grayscale gradient change rate of the flow mark strip area is between 15 and 35 grayscale values / mm; the strip width is between 2.0 mm and 5.0 mm, or the continuous strip length is between 15 mm and 30 mm; the edge sharpness is between 10 and 20 contrast units / mm; and the number of flow mark stripes in the same detection surface is between 4 and 6, then the flow mark strip area is judged as a moderate flow mark. If the grayscale gradient change rate of the flow mark strip area is greater than 35 grayscale values / mm; the strip width is greater than 5.0mm and the continuous length of the strip exceeds 30mm; the edge sharpness is greater than 20 contrast units / mm; and the number of flow mark stripes in the same detection surface is more than 6, then the flow mark strip area is judged as a severe flow mark.

8. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 6, characterized in that, Connecting spatially adjacent stable candidate points includes: For each stable candidate point, examine its neighboring pixels in 8 directions. If a neighboring pixel is also a stable candidate point, it means that the two points are spatially adjacent. The 8 directions include up and down, left and right, and the four diagonal directions. Starting from any stable candidate point, find all stable candidate points that are directly or indirectly adjacent to it and group them into the same connected region; For all stable candidate points, check whether they have been assigned to a connected region. If not, establish a new connected region starting from that point using the method described above. Each connected region is a candidate stable strip region.

9. The visual inspection method for stamping defects of hardware parts based on image technology according to claim 1, characterized in that, Before securing the hardware in the inspection area using an electric clamp, the following steps are also included: The laser alignment module scans and identifies the detection area to determine whether the hardware currently placed in the detection area is accurately in the preset detection reference position. The obtained hardware position information is compared with the preset detection benchmark position. If the result indicates that there is a positional deviation in the hardware, the automatic positioning device is activated to correct the position of the hardware.

10. A visual inspection system for stamping defects in hardware parts based on image technology, characterized in that, For performing the image-based visual inspection method for stamping defects in hardware parts as described in claim 1, the image-based visual inspection system for stamping defects in hardware parts comprises: The transmission module is used to establish a vision inspection workstation downstream of the stamping production line, and to set up the vision inspection workstation and the automatic transmission track. The automatic transmission track is used to send each stamped metal part into the inspection area of ​​the vision inspection workstation. A fixing module is used to fix the hardware in the inspection area by electric clamps; an industrial camera is mounted on the fixed hardware and configured with a combination of ring light source and oblique strip light source; The image acquisition module is used to detect the position signal of the hardware parts through the photoelectric sensor on the vision inspection workstation, and trigger the camera to acquire images of the hardware parts under the light source combination to obtain a set of hardware parts images from multiple angles. The defect identification module is used to identify flow mark stripe areas in the hardware part image set and determine the severity of flow marks in the flow mark stripe areas; hardware parts with flow mark severity greater than a preset severity threshold are marked as unqualified hardware parts, and the PLC-controlled rejection device automatically sorts the unqualified hardware parts.

Images