Visual inspection system for precision sheet metal parts

Abstract

The application discloses a kind of precision sheet metal parts visual inspection systems, it is related to precision sheet metal detection technical field;The system is built by including low-temperature adaptation industrial camera, hyperspectral imaging module and pre-stored multi-model feature fingerprint library Hardware architecture, cooperate with temperature difference dynamic adjustment Image compensation, material spectrum and shape topology double-fingerprint matching Model automatic identification, spectrum angle matching combined with visual image Defect detection, integrate temperature data LSTM time series defect early warning, and frozen backbone network Non-stop incremental model optimization, build full-process automated detection scheme. Effectively eliminate the image blur caused by low temperature, quickly and accurately adapt to multiple models of sheet metal parts, significantly reduce the risk of defect misjudgment, early warning of defect development trend, continuously maintain high recognition accuracy of model, significantly improve detection efficiency and accuracy, reduce production loss, fully meet the precision detection needs of cold chain equipment core components.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of precision sheet metal detection, in particular to a precision sheet metal part visual detection system. BACKGROUND

[0002] The evaporator shell in the cold chain equipment as a key heat exchange component, its sheet metal structure often adopts aluminum-steel composite, stainless steel single material, aluminum-copper composite with reinforcing rib and other multiple model designs, and in order to ensure the heat exchange efficiency, the hole position precision (error needs to be ≤±0.01mm), burr (needs to be ≤0.05mm) and crack (needs to be ≤0.1mm) and other defect detection requirements of the sheet metal part are very high. At present, the detection of such sheet metal parts in the industry depends on manual cooperation with single visual equipment, and manual judgment of part model and adjustment of detection parameters are required in the detection process, which not only has low efficiency (single piece detection time is more than 12 minutes), but also has strong dependence on the experience of detection personnel, and it is difficult to meet the batch production demand.

[0003] However, the detection environment of the evaporator shell is often in the low temperature interval of-5℃ to-30℃, and the existing detection technology has significant limitations in this scenario: low temperature causes camera lens fogging, and image blur causes defect omission rate as high as 28%; at the same time, defects such as hole position deviation caused by mold wear can only be found afterwards, often causing 15-20 consecutive parts to be scrapped, greatly increasing production cost. In summary, the existing technology cannot solve the problem of "automated and high-precision detection of multiple models of precision sheet metal parts in low-temperature environment", which seriously restricts the production efficiency and product quality of cold chain equipment.

[0004] Therefore, the present application is proposed. SUMMARY

[0005] The purpose of the present application is to provide a precision sheet metal part visual detection system to solve the problems in the background art.

[0006] To solve the above technical problems, the precision sheet metal part visual detection system provided by the present application comprises the following steps:

[0007] S1 Build hardware architecture: contains low-temperature adaptive industrial camera, hyperspectral imaging module, temperature sensor, edge computing node, mold parameter interaction module, and feature storage server, the feature storage server pre-stores a multi-model feature fingerprint library, and the fingerprint data of each model in the multi-model feature fingerprint library contains a preset material spectral feature wavelength interval and a preset shape topological feature parameter;

[0008] S2 Perform temperature compensation preprocessing: collect the detection environment temperature in real time through the temperature sensor, dynamically adjust the image blue channel gain and histogram equalization threshold according to the difference between the collected temperature and the preset reference temperature, the lower the temperature, the greater the gain increase and the greater the threshold decrease, and output a clear image without fog after Gaussian filtering processing;

[0009] S3 performs multi-model automatic selection: based on the clear image output by S2, the shape topology fingerprint of the metal sheet part to be detected is extracted, including curvature, rib spacing, hole arrangement rule, based on the spectral data collected by the hyperspectral imaging module, the characteristic peak is filtered through the pre-set material spectral feature wavelength interval, the material spectral fingerprint of the metal sheet part to be detected is extracted, and the two fingerprints are input into the multi-model feature fingerprint library for similarity calculation. After successful matching, the corresponding model detection model is automatically loaded to realize rapid and accurate matching of multi-model parts;

[0010] S4 performs spectral-vision fusion defect recognition: loads the exclusive detection model determined by S3, generates a material boundary mask by collecting spectral data through the hyperspectral imaging module, and inputs the material boundary mask and the clear image output by S2 into the exclusive detection model after superposition, identifies the defect type, position and size and outputs the defect annotation map;

[0011] S5 performs time series defect trend prediction and early warning: a time series database is constructed by storing the temperature data obtained by S2, the defect data output by S4 and the model information determined by S3 through the edge computing node, the defect trend is predicted based on the time series database, and the mold parameter adjustment instruction is output when the tolerance is exceeded. At the same time, the misjudgment sample is marked and stored in the model optimization sample library; through dynamic temperature compensation and double-fingerprint model matching technology, the problems of poor image quality in low temperature environment and difficulty in adapting to multi-model parts are solved, the automation and precision of metal sheet part detection are realized, and the detection efficiency and defect recognition accuracy are improved.

[0012] Further, S6 model iterative optimization is further included: the defect recognition accuracy of each model exclusive detection model is counted in real time through the edge computing node, when the recognition accuracy of a certain model exclusive detection model for a continuous number of parts is lower than the pre-set accuracy threshold, the misjudgment sample and corresponding data of the model are extracted from the model optimization sample library, the exclusive detection model backbone network is frozen, and only the incremental training strategy of the model classification head is fine-tuned. The extracted data is input into the model according to the pre-set batch for training, and after the training is completed, the new model is updated as the exclusive detection model of the model, and is synchronized to the spectral-vision fusion defect recognition step of S4; the self-iterative optimization of the detection model is realized, the precision decay caused by long-term use of the model is avoided, the production continuity is ensured through the incremental training strategy, and the adaptability and defect recognition stability of the model to specific model parts are improved.

[0013] Furthermore, in S1, the temperature range of the low-temperature adaptive industrial camera covers from -30 degrees Celsius to -5 degrees Celsius, the wavelength range of the hyperspectral imaging module covers from 400 nanometers to 1,000 nanometers, the measurement accuracy of the temperature sensor is ±0.2 degrees Celsius, and the GPU memory of the edge computing node is no less than 12 gigabytes and meets the real-time requirements of S5 time-series data processing and S6 model training; the hardware performance parameters are clearly defined to ensure that the hardware architecture can adapt to the low-temperature detection scenario, meet the computing power requirements of hyperspectral data processing and model training, and ensure the stable operation and real-time detection of the system.

[0014] Further, in step S3, the shape topology fingerprint extraction step includes: S31: performing edge detection on the clear image output in S2 to determine the contour boundary of the sheet metal part to be detected; S32: calculating the curvature value, rib spacing value, and row and column spacing value of the hole arrangement of the part based on the contour boundary; S33: comparing the calculated values ​​with the preset shape topology feature parameters of the corresponding model in the multi-model feature fingerprint library to form a shape topology fingerprint; the material spectral fingerprint extraction step includes: S34: performing noise reduction processing on the spectral data collected by the hyperspectral imaging module; S35: filtering spectral peaks within the preset material spectral feature wavelength range; S36: comparing the filtered spectral peaks with the preset material spectral feature wavelengths of the corresponding model in the multi-model feature fingerprint library to form a material spectral fingerprint; this standardizes the fingerprint extraction process, improves the extraction accuracy of shape topology fingerprints and material spectral fingerprints, further improves the accuracy of automatic selection of multiple models, and reduces defect identification deviations caused by model mismatch.

[0015] Further, in step S4, the specific steps for generating the material boundary mask are as follows: S41: Determine the characteristic wavelengths corresponding to each material of the sheet metal part to be inspected based on the preset material spectral characteristic wavelength range; S42: Perform band segmentation on the spectral data collected by the hyperspectral imaging module to obtain single-band images corresponding to each material; S43: Perform binarization processing on the single-band images of each material, assign colors to different material regions according to preset color coding rules, and generate material boundary masks; The step of superimposing the material boundary mask with the clear image is as follows: S44: Perform coordinate calibration on the material boundary mask and the clear image to ensure that the pixel positions of the two correspond one-to-one; S45: Use a pixel superposition algorithm to cover the corresponding position of the clear image with the material boundary mask; By accurately generating and superimposing the material boundary mask, the problem of defect misjudgment caused by the blurring of material boundaries in the detection of multi-material parts is solved, and the accuracy of defect identification and positioning is improved.

[0016] Further, in the S5, the construction rule of the time series database is to store the temperature data, defect data and model information collected at intervals of a preset number of detected sheet metal parts, the training sample of the temperature-defect correlation model is the historical temperature data and the corresponding defect data stored in the time series database, and the number of training samples is not less than a preset sample number threshold, and the mold parameter adjustment instruction includes a stamping force adjustment value, a laser power adjustment value and an adjustment execution time; The accuracy of the time series defect trend prediction and the executability of the mold adjustment are ensured, sufficient and reliable training data are provided for the temperature-defect correlation model, the generation of batch defects is reduced, and the production efficiency is improved.

[0017] Further, in the S6, the method for calculating the defect recognition accuracy is to compare the defect labeling map output by S4 with the manual review result, calculate the ratio of the number of consistent defects to the total number of defects, the preset number of sheet metal parts to be detected is fifty, the preset accuracy threshold is ninety-five percent, the preset batch of incremental training is ten batches and the number of samples in each batch is thirty; The trigger condition and training parameters of the model iteration optimization are quantified, the model optimization process is controllable and repeatable, the unstable optimization effect caused by ambiguous parameters is avoided, and the recognition accuracy of the updated model is ensured.

[0018] Further, in the S31, the edge detection uses the Canny algorithm, the preset matching threshold in S3 is 98.5%, the denoising processing in S34 uses the wavelet transform denoising algorithm, and the spectral peak value extraction in the preset material spectral feature wavelength range in S35 uses the threshold segmentation method and the threshold is 50% of the maximum value of the spectral data; The key algorithms and thresholds in the fingerprint extraction process are clarified, the multi-model identification process is further standardized, the edge detection accuracy and the spectral data quality are improved, and the reliability of the multi-model automatic selection is ensured.

[0019] Compared with the prior art, the beneficial effects of the present application are:

[0020] 1. By building a hardware architecture including a low-temperature adaptive industrial camera, a hyperspectral imaging module and a pre-stored multi-model feature fingerprint library, the limitations of the existing hardware "low-temperature incompatibility and multi-model incompatibility" are broken through. The low-temperature adaptive camera ensures stable image output in a low-temperature environment, the hyperspectral module accurately captures the characteristic wavelengths of different materials, and the multi-model feature fingerprint library provides standardized data support for subsequent model identification. The three work together to make the hardware system stable and compatible with the multi-model detection scene of the evaporator shell of the cold chain equipment, greatly enhancing the continuous operation capability and significantly widening the scene adaptation range.

[0021] 2. By employing two technical approaches—"temperature compensation with dynamic adjustment of parameters based on temperature difference" and "dual fingerprint matching of material spectral fingerprint and shape topology fingerprint"—the industry pain points of low-temperature image blurring and manual switching between multiple models are addressed. The temperature compensation stage dynamically adjusts the image channel gain and equalization threshold based on the difference between the detection temperature and the reference temperature. Combined with Gaussian filtering, this effectively eliminates image blurring caused by low-temperature fogging, providing clear and reliable image data for subsequent inspection. The dual fingerprint matching stage extracts the shape topology features and material spectral features of the part, calculates the comprehensive similarity to achieve automatic model identification, and can quickly load the corresponding detection model without manual intervention. This completely replaces the traditional manual switching mode, significantly shortening model switching time and avoiding the risk of human error. The efficiency and accuracy of model identification are qualitatively improved.

[0022] 3. By employing a spectral-visual fusion technique—"spectral angle matching to generate material boundary masks + pixel-level image fusion"—the problem of "misjudgment of defects due to blurred material boundaries" in the inspection of multi-material sheet metal parts is solved. The spectral angle matching algorithm accurately distinguishes different materials and generates boundary masks, while pixel-level fusion highlights material boundaries while preserving visual image details. This allows the detection model to identify defects for different materials separately, effectively avoiding misjudgments caused by differences in reflectivity among multiple materials. The ability to identify subtle defects (such as microburrs and microcracks) is significantly enhanced, and the accuracy of defect detection is greatly improved.

[0023] 4. A unique technique, the "LSTM time-series prediction model combining temperature data," enables early warning of defects. This model combines temperature data with historical defect data to accurately capture the correlation between temperature changes and defect development. It can predict the defect trends of subsequent parts in advance. When a predicted defect exceeds tolerance, it automatically outputs mold adjustment commands, completing mold parameter correction without stopping the machine. This changes the traditional passive mode of "discovering defects after the fact," effectively reducing continuous scrap due to mold wear and significantly lowering production losses. Attached Figure Description

[0024] Figure 1 This is a flowchart of a precision sheet metal parts visual inspection system. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Please see Figure 1This invention provides a technical solution: a precision sheet metal parts visual inspection system. This embodiment relates to the field of precision sheet metal inspection technology, and is particularly suitable for batch inspection of sheet metal parts for evaporator shells, a core component of low-temperature cold chain equipment. Currently, in the cold chain industry, evaporator shells are often designed with various models, including aluminum-steel composite, stainless steel single material, and aluminum-copper composite with reinforcing ribs, and the inspection environment is often in the low-temperature range of -5℃ to -30℃. Existing inspection methods have three major drawbacks: first, lens fogging at low temperatures leads to blurred images, resulting in a false negative rate as high as 28%; second, manual switching of inspection models is required when multiple models are produced on mixed production lines, with each switch taking more than 5 minutes; and third, defects caused by mold wear can only be detected afterward, often resulting in 15-20 consecutive scrapped parts. To solve these problems, this system integrates six key aspects: hardware architecture, temperature compensation, multi-model matching, spectral-visual fusion, time-series early warning, and model optimization, achieving fully automated and accurate inspection throughout the entire process.

[0027] (I) Building the Hardware Architecture: The core is to build a hardware system that is compatible with low-temperature multi-model detection. If the hardware is not selected correctly, such as if the camera is not resistant to low temperatures or the computing power is insufficient, even the most sophisticated algorithms cannot be implemented. Ordinary industrial cameras will experience lens fogging and slow shutter response below -15℃; without a hyperspectral module, the boundaries of multiple materials cannot be distinguished at all, making defect identification difficult.

[0028] Hardware selection referenced the requirements for low-temperature environment equipment in the "Design Specification for Industrial Machine Vision Systems" and the recommendations for spectral wavelength range in the "Application Guidelines for Hyperspectral Imaging Technology in Industrial Inspection." Simultaneously, considering the inspection accuracy requirements of cold chain sheet metal parts (dimensional error ±0.01mm, minimum defect recognition 0.05mm), the performance parameters of each hardware component were determined. Specifically, the wavelength range of the hyperspectral module needed to cover the target material's characteristic peaks within a ±10nm range, ensuring a characteristic peak capture rate ≥99%. This parameter was determined based on material spectral characteristic testing experiments.

[0029] The hardware architecture comprises six core components: a low-temperature adaptive industrial camera (compatible with temperatures ranging from -30℃ to -5℃, 5-megapixel resolution, 20fps frame rate, maintaining stable image output at low temperatures and preventing lens fogging or shutter malfunction); a hyperspectral imaging module (using a hyperspectral camera with a wavelength range of 400nm to 1000nm and a spectral resolution of 5nm, precisely covering the characteristic wavelength range of aluminum alloy (532nm), steel (650nm), and stainless steel (580nm), enabling accurate differentiation of different materials); and a temperature sensor with a measurement accuracy of ±0.2℃. The sampling frequency is 1Hz, and the ambient temperature of the inspection station is collected in real time to provide data support for subsequent temperature compensation; the edge computing node has a GPU with 16GB of video memory to meet the requirement of ≥12GB, and an 8-core ARM Cortex-A78 CPU, which can simultaneously process hyperspectral data, time-series prediction models, and incremental training tasks to ensure that the inspection process is not interrupted; the mold parameter interaction module uses an RS485 communication interface to connect with the punching machine and laser welding machine, supports real-time sending of parameter adjustment commands such as punching pressure and laser power, and has a response latency of ≤500ms; the feature storage server has a storage capacity of 1TB. The SSD pre-stores a multi-model feature fingerprint database for three types of cold chain evaporator casings. Model A is an aluminum-steel composite flat type, whose fingerprint includes the 532nm spectral range of aluminum and the 650nm spectral range of steel, a flat curvature of 0 (i.e., a plane), and a hole spacing of 10mm. Model B is a stainless steel arc type, whose fingerprint includes the 580nm spectral range of stainless steel, an arc curvature of 800mm, and no holes. Model C is an aluminum-copper composite type with reinforcing ribs, whose fingerprint includes the 532nm spectral range of aluminum and the 510nm spectral range of copper, a reinforcing rib spacing of 20mm, and a hole spacing of 8mm.

[0030] Application Example: On the evaporator production line of a cold chain equipment factory, after we built the hardware according to S1, we connected the feature storage server to the production line MES system to synchronize the daily production plan in real time. For example, 100 pieces of model A were produced in the morning and 80 pieces of model B were produced in the afternoon. After the hardware was powered on, each component passed the self-test. The low temperature camera worked continuously for 4 hours in an environment of -20℃ without fogging. The spectral data of model A collected by the hyperspectral module can clearly see the dual characteristic peaks of 532nm and 650nm, which fully meets the subsequent detection requirements.

[0031] Existing publicly available documents describe a low-temperature sheet metal inspection device that only includes a low-temperature camera and a standard vision module. It lacks a hyperspectral imaging module and a multi-model feature fingerprint database, making it unable to distinguish between multiple materials and models. Furthermore, the hardware is not adapted to low-temperature environments and cannot operate stably below -10°C. This system's hardware architecture, through a combination of "low-temperature adaptation + multispectral imaging + high computing power," solves the problems of "low-temperature incompatibility and multi-model incompatibility" in existing hardware. This significantly improves scene adaptability and extends the continuous stable operation time from 8 hours to 24 hours.

[0032] (ii) S2 performs temperature compensation preprocessing: mainly to solve the problem of image blurring at low temperatures. When the ambient temperature drops below -15℃, a thin layer of frost will form on the lens of the low-temperature camera, resulting in a white image with blurred details. The holes on the evaporator shell cannot be seen clearly, and burrs as small as 0.05mm cannot be identified.

[0033] The temperature compensation technique is based on the principles of "histogram equalization for image enhancement" and "Gaussian filtering for noise suppression" from the textbook "Digital Image Processing." It also incorporates the color cast characteristics of images at low temperatures, where the blue channel shows the most significant attenuation, and adjusts the channel gain accordingly. Our experiments have shown that the grayscale value of the blue channel in images at low temperatures is 15%-20% lower than at room temperature; adjusting the blue channel gain effectively restores color balance.

[0034] The dynamic parameter adjustment uses the following formula:

[0035] (1) Blue channel gain adjustment formula: In the formula, To adjust the gain of the blue channel, This is the default gain at room temperature (value 1.0). This is the gain adjustment factor (valued at 0.03 / ℃, meaning the gain increases by 0.15 for every 5℃ decrease in temperature). The difference between the reference temperature (-10℃) and the actual measured temperature. , , (actual temperature)

[0036] (2) Histogram equalization threshold adjustment formula: In the formula, For the adjusted threshold, This is the default threshold at room temperature (value 0.8). m is the threshold adjustment coefficient (valued at 0.02 / ℃, meaning the threshold decreases by 0.1 for every 5℃ drop in temperature). Same meaning as above;

[0037] (3) Gaussian filtering formula:

[0038] ;

[0039] In the formula, These are the pixel coordinates within the filter kernel. The standard deviation is 0.9, ensuring that details such as hole positions and ribs are preserved while denoising.

[0040] The specific operation process consists of three steps, each linked to temperature and not using fixed parameters: Step 1: Temperature Acquisition: A temperature sensor collects and detects the ambient temperature in real time. For example, if the temperature collected one morning is -20℃, the preset baseline temperature is -10℃. This baseline is determined based on extensive experiments. Above -10℃, the image is not noticeably blurry; below that, significant compensation is needed. Step 2: Parameter Adjustment: Based on the difference between the acquired temperature and the baseline temperature (-20℃ - (-10℃) = -10℃), adjustments are made according to the rule that "for every 5℃ decrease in temperature, the blue channel gain increases by 15%, and the histogram equalization threshold decreases by 0.1." Here, the -10℃ difference corresponds to a decrease of two 5℃ values, so the blue channel gain increases from the default 1.0 to 1.3 (1.0 + 2 × 15%), and the histogram equalization threshold decreases from the default 0.8 to 0.6 (0.8 - 2 × 0.1). Step 3: Image Filtering: Gaussian filtering is performed on the adjusted image. The filter kernel size is set to 3 × 3, and the standard deviation is... ,Should The value was determined through trial and error. Too small a value would not filter out frost, fog, and noise, while too large a value would blur details. 0.9 strikes a good balance between noise reduction and detail preservation, ultimately outputting a clear, fog-free image.

[0041] Application Example: Taking the detection of Model A as an example again, in an environment of -20℃ without temperature compensation, the boundary between the aluminum layer and the steel layer in the image of Model A is blurred, and there are obvious "rough edges" at the edge of the hole. After processing with S2, the silver-white of the aluminum layer and the dark gray of the steel layer are clearly distinguished in the image, and the edge of the hole is smooth. When the diameter of the hole is measured by image analysis software, the error is reduced from ±0.03mm to ±0.01mm, which fully meets the requirements of subsequent shape topology fingerprint extraction.

[0042] Existing public documents describe a low-temperature sheet metal inspection device that uses fixed histogram equalization parameters for low-temperature image enhancement, applying the same threshold regardless of whether the temperature is -15℃ or -30℃. This results in excessive image enhancement and noise amplification at -30℃. Another public document describes a low-temperature image processing method that only adjusts brightness without adjusting channel gain, thus failing to address the image color cast issue. Our system's S2, through "dynamic adjustment of parameters based on temperature difference," adds an extra layer of temperature-related logic compared to the fixed-parameter approach. This improves image clarity by 40% and reduces the defect false negative rate at -30℃ from 28% to 3%, meeting the requirements for precision inspection.

[0043] (III) S3 performs automatic selection of multiple models: it identifies the part model and loads the corresponding inspection model. When multiple models are produced on the same line, the staff have to manually switch models on the computer. If model A is produced in the morning and model B is produced in the afternoon, the staff has to find the corresponding folder and load the model. The whole process takes at least 5 minutes and it is easy to select the wrong model. For example, the model of model B is used for model A, which leads to all defects being misjudged.

[0044] The basis for multi-model identification is the dual matching of "material spectral fingerprint + shape topological fingerprint". Different materials have unique spectral characteristic peaks, such as 532nm for aluminum and 580nm for stainless steel. Different shapes also have different topological parameters, such as curvature and rib spacing. Combining the two can distinguish the models with 100% accuracy. This approach is based on the "multi-feature fusion classification" method in "Pattern Recognition", which has a much higher accuracy than single-feature identification.

[0045] The similarity calculation of the two fingerprints uses a weighted summation formula: In the formula, The overall similarity score is calculated (values ​​range from 0 to 1, with a threshold of 0.985). The shape topology fingerprint weight (value 0.4). The weight of the material spectral fingerprint is set to 0.6, because the material spectrum has higher distinguishability. Shape topological fingerprint similarity (calculated using Euclidean distance based on curvature, rib spacing, and hole arrangement spacing). The similarity of the material's spectral fingerprint is calculated using the deviation rate of the characteristic peak positions.

[0046] The calculation formula is as follows:

[0047] ;

[0048] In the formula, The curvature of the part to be tested. Preset curvature for the models in the library; The spacing between the ribs of the part to be inspected is 0 (0 is used when there are no ribs). Preset the rib spacing for the models in the library; The spacing between holes in the part to be inspected (0 if there are no holes). Preset the hole spacing for the models in the library;

[0049] The calculation formula is as follows:

[0050] ;

[0051] In the formula, The number of characteristic peaks (e.g., aluminum-steel composites) ), For the part to be inspected One characteristic peak wavelength, For model number in the library Each characteristic peak has a preset wavelength.

[0052] The specific technical operation involves three steps: "fingerprint extraction, fingerprint comparison, and model loading," each with clear standards: Step 1: Extracting dual fingerprints: Shape topology fingerprint: From the clear image output by S2, an edge detection algorithm (the Canny algorithm will be explained later) is used to find the outline of model A, and its curvature (i.e., flat plate curvature 0), rib spacing (i.e., no ribs, so no rib spacing), and hole spacing (10mm) are calculated. These three parameters constitute the shape topology fingerprint; Material spectral fingerprint: The hyperspectral imaging module acquires the spectral data of model A, first denoising it, then using wavelet transform in S34 for denoising, and then screening within the preset material spectral characteristic wavelength range, aluminum 520-540nm and steel 640-660nm. The first step is to select and find two characteristic peaks at 532nm and 650nm. The position and intensity of these two peaks constitute the material's spectral fingerprint. The second step is fingerprint matching: input the dual fingerprints into the multi-model feature fingerprint database of the feature storage server, calculate the similarity, and compare the dual fingerprint of model A with the preset fingerprint of model A in the database. If the similarity reaches 98.5%, this threshold is calibrated. If it is lower than 98%, it may be a part deformation, and if it is higher than 98.5%, it is a successful match. The third step is to load the model: after a successful match, the edge computing node automatically loads the exclusive detection model of model A from the server. This model is pre-trained for the defect types of model A, such as burrs in the aluminum layer and cracks in the steel layer. The loading time is less than 10 seconds.

[0053] Application Example: A factory switched production models from model A to model B at noon one day. After S2 outputs a clear image of model B, S3 automatically extracts its shape topology fingerprint (curvature of 800mm, no ribs, no pores) and material spectral fingerprint (580nm feature peak). The similarity is compared with the fingerprint database to achieve 99.2%. Then, the dedicated model for model B is loaded. The entire process is completed in 12 seconds. Previously, manual switching took 5 minutes and was prone to errors.

[0054] Existing publicly available documents describe multispectral sheet metal inspection systems that rely on QR codes to identify model numbers. However, these QR codes must be affixed to the parts, and they are prone to detaching after high-temperature welding, resulting in a recognition rate of less than 90%. Another publicly available document describes a multi-model sheet metal inspection device that relies solely on shape recognition, which may misidentify models with similar shapes but different materials, such as aluminum plates and steel plates. Our system's S3 uses "dual fingerprint matching" and a corresponding similarity calculation formula, eliminating the need for QR codes and effectively distinguishing between similar-shaped models. It achieves a matching accuracy of 98.5%, reducing switching time from 5 minutes to 10 seconds, thus solving the adaptation problem for multi-model mixed-line production.

[0055] (iv) S4 performs spectral-visual fusion defect identification: accurately identifying defects on parts. Previously, when using single-vision inspection, the drawback was the difference in reflectivity between different materials. Aluminum has a reflectivity of 85%, while steel has a reflectivity of 55%. In the images taken, normal textures in the aluminum layer and cracks in the steel layer look almost the same, resulting in a false positive rate as high as 22%. S4 uses a spectral-visual fusion method to first distinguish the materials and then find the defects, significantly reducing the false positive rate.

[0056] The technical basis of spectral-visual fusion is the use of "spectral angle matching algorithm" to distinguish different materials and "image pixel-level overlay technology" to combine spectral and visual information. Referring to the method of "spectral angle matching for material identification" in "Classification and Application of Hyperspectral Remote Sensing Images", and combining the needs of "defect location requires visual image details" in industrial inspection, the two are fused together. Spectrum is responsible for distinguishing materials, and vision is responsible for finding details.

[0057] The formula for the spectral angle matching algorithm is as follows:

[0058] ;

[0059] In the formula, The spectral vector to be detected With reference material spectral vector The included angle (unit: radians). For the dot product of two vectors, , Let L2 magnitudes of the two vectors be respectively; when ( When the threshold value is 0.1 radians, the material is considered to be the same.

[0060] Pixel-level overlay formula: In the formula, For the fused image in grayscale value at that location The visual image weight (value 0.7, to preserve details). Spectral mask weight (value 0.3, to highlight material boundaries). The grayscale values ​​of the visual image output by S2. This is the grayscale value of the material boundary mask (different materials are distinguished by values ​​such as 255 and 128).

[0061] The specific operation consists of three steps: "generating material boundary masks, image overlay, and defect recognition." Each step is designed for multi-material defect detection: Step 1: Generating material boundary masks: First, determine the characteristic wavelengths corresponding to each material of model A: aluminum 532nm, steel 650nm. The hyperspectral imaging module collects the spectral data of model A, uses a spectral angle matching algorithm for band segmentation, extracts the bands near 532nm as single-band images of the aluminum layer, and the bands near 650nm as single-band images of the steel layer. Binarize the single-band images, setting the threshold to 1.2 times the spectral mean. Then, according to a preset color coding rule, the aluminum layer is set to white, and the steel layer to gray, generating a material boundary mask. The white areas in the mask represent the aluminum layer, and the gray areas represent the steel layer. The first step is to overlay the image onto the steel layer, ensuring clear boundaries. The second step involves coordinate calibration of the material boundary mask and the clear image output by S2, using two positioning holes on the part as references to ensure a one-to-one correspondence between the pixel positions of the mask and the image. Then, a pixel overlay algorithm is used to overlay the mask onto the clear image, forming a fused image with "material annotation + clear details". The third step is to identify defects by inputting the fused image into the dedicated detection model for model A. The model has two branches: the aluminum layer branch looks for burrs with an identification threshold of 0.05mm (any value exceeding this threshold is a defect), and the steel layer branch looks for cracks with an identification threshold of 0.1mm (any value exceeding this threshold is a defect). Finally, a defect annotation image is output, indicating the defect type (e.g., aluminum layer burrs), location (e.g., hole position at 3 o'clock), and size (e.g., 0.06mm).

[0062] Application Example: When inspecting a part of model A, the fused image generated by S4 shows the aluminum layer as white and the steel layer as gray, with clear, unambiguous boundaries. The model identified a 0.06mm burr in hole 3 of the aluminum layer, exceeding the 0.05mm threshold, and a 0.12mm crack at the weld of the steel layer, exceeding the 0.1mm threshold. These markings are directly displayed on the workshop screen, allowing workers to immediately identify the problems. Previously, with single-vision inspection, the burr in the aluminum layer was mistaken for normal texture, and the crack in the steel layer was mistaken for welding marks, leading to missed detections. This technical solution significantly reduces the possibility of missed detections.

[0063] Existing publicly available documents describe multispectral sheet metal inspection systems that rely solely on spectral identification without considering visual image details, resulting in a defect location accuracy of ±0.1mm, which fails to meet precision requirements. Another publicly available document describes a low-temperature sheet metal inspection device that uses only visual inspection, with a false positive rate of 22%. Our system's S4 integrates spectral and visual data, employing spectral angle matching and pixel overlay formulas to clearly distinguish materials while preserving details. This reduces the defect false positive rate from 22% to 0.5%, improves location accuracy to ±0.01mm, and enables the identification of minute burrs as small as 0.05mm, meeting the precision inspection standards for cold chain evaporators.

[0064] (V) S5 Defect Trend Prediction and Early Warning: Its function is to detect problems in advance and avoid batch scrapping. Previously, we encountered a situation where mold wear occurred. The punch of the punching machine wore down, and the hole position offset of model A initially was 0.08mm, gradually increasing to 0.09mm, 0.10mm, until it exceeded the tolerance of 0.1mm before being discovered. By then, 18 parts had already been scrapped, resulting in financial loss. S5 analyzes historical data to predict in advance that the hole position offset will exceed the tolerance, and then issues instructions to adjust the mold, minimizing losses.

[0065] The basis for predicting time-series defects is the LSTM (Long Short-Term Memory) network model. This model excels at processing time-series data and can capture data trends. Referring to "Application of LSTM Model in Industrial Forecasting" in "Time Series Analysis and Forecasting," and considering the characteristic in sheet metal inspection that "temperature affects defect formation"—for example, parts are more brittle at low temperatures and cracks are more likely to appear—temperature data is also incorporated into the prediction model to make the prediction more accurate.

[0066] The core formula of the LSTM model is as follows:

[0067] (1) Gate of Oblivion: ;

[0068] (2) Input gate: , ;

[0069] (3) Cell state renewal: ;

[0070] (4) Output gate: , ;

[0071] In the formula, For time steps, , , These are the forget gate, input gate, and output gate, respectively. (for sigmoid activation function) , These represent the cell state and the hidden layer output, respectively. for Input characteristics (including temperature, historical defect values, and model code) at all times. , , , This is the weight matrix. , , , For bias terms, Element-wise multiplication;

[0072] Defect trend prediction threshold formula: In the formula, For prediction Time-based defect values ​​(such as hole position offset). This is the defect tolerance threshold (e.g., 0.1 mm). The prediction step size is set to 15, meaning 15 warnings are issued in advance.

[0073] The specific operation consists of four steps: "building a database, making predictions, issuing instructions, and labeling samples," forming a data closed loop. Step 1: Building a time-series database: Edge computing nodes store data at intervals of "storing data every 10 parts inspected." This includes temperature data (e.g., -20℃, -19℃, -21℃) for S2, defect data (e.g., average hole offset of model A: 0.08mm, 0.09mm, 0.10mm), crack counts (2, 3, 4), and model information for model A for S3. The database retains records of the most recent 1000 parts. Step 2: Defect trend prediction: The time-series data of the most recent 100 parts of model A are extracted from the database and input into a temperature-defect correlation LSTM model. The model learns the relationship between temperature and defects; for example, for every 5℃ decrease in temperature, the hole offset increases by 0.002mm. Then, the model predicts the defect trend for the next 20 parts. For example, the prediction results show that the hole position offset of the next 15 model A pieces will reach 0.11mm, exceeding the tolerance of 0.1mm; the third step: output adjustment command: after predicting the excess tolerance, the edge computing node sends an adjustment command to the punching machine through the mold parameter interaction module, such as adjusting the punching pressure from 800kN to 820kN. After the punching pressure is increased, the punch can be better shaped and the hole position offset is reduced. After the command is sent, the punching machine completes the parameter adjustment within 5 seconds; the fourth step: mark misjudged samples: when storing data, the staff will manually review the defect annotation map output by S4, and mark the misjudged samples, such as normal textures, as burrs. They are then stored in the model optimization sample library along with the corresponding spectral data and clear images to prepare for the subsequent S6.

[0074] Application Example: During the production of model A in a certain batch, the S5 time-series database recorded the hole position offsets of 30 consecutive parts: 0.08mm for parts 1-10, 0.09mm for parts 11-20, and 0.10mm for parts 21-30. The LSTM model predicted that the offset for parts 31-45 would reach 0.11-0.12mm, exceeding the 0.1mm tolerance. Therefore, it immediately sent a command to the punching machine to "adjust the punching pressure to 820kN". After adjustment, the hole position offset for parts 31-45 stabilized at 0.095mm, no longer exceeding the tolerance. At the same time, 20 misjudged samples were marked, such as the normal texture of part 15 being misjudged as a burr, and stored in the sample library.

[0075] Existing publicly available documents describe two sheet metal defect early warning devices. One relies solely on defect quantity statistics for early warning, neglecting the influence of temperature, resulting in a prediction accuracy of less than 80%. Another document describes a mold wear detection device, which requires machine shutdown for mold inspection, impacting production. Our system's S5 combines temperature and defect data, using a core formula from an LSTM model to achieve trend prediction with an accuracy exceeding 95%. It can also provide early warnings for up to 15 parts without downtime, reducing the continuous scrap rate from 12% to 0.7%, significantly lowering the cost per batch.

[0076] (vi) S6 performs iterative model optimization: avoiding a decline in accuracy over time. We found that after three months of use, the recognition accuracy dropped from 99% to around 94%. This is partly due to batch differences in the material of the parts, such as slight variations in the composition of aluminum in a particular batch, resulting in a 2nm shift in the spectral peak. Another factor is environmental changes, such as increased humidity in the workshop and increased image noise. Without model optimization, the misclassification rate would continue to rise. S6 ensures that the model maintains high accuracy.

[0077] The model iterative optimization is based on the "incremental training strategy." This strategy avoids retraining the entire model; it only requires fine-tuning some parameters with new samples, improving accuracy without downtime. Referring to the "Application of Incremental Training in Industrial Models" in "Deep Learning Model Optimization and Deployment," and considering the "no downtime" requirement of the detection system, a "freeze backbone network + fine-tune classification head" approach is adopted. The backbone network is responsible for feature extraction and remains unchanged; the classification head is responsible for defect identification, and fine-tuning it adapts to new situations.

[0078] The formula for the loss function in incremental training (weighted for misclassified samples):

[0079] ;

[0080] In the formula, For the total loss, For the sample size, For the first The weights of each sample (normal samples) Misjudged samples (to enhance the impact of misjudged samples on training) For the first The true label (defective / normal) of each sample. Predict labels for the model, The cross-entropy loss function;

[0081] Model parameter update formula: In the formula, The updated parameters (only the category header parameters are updated). For parameters before the update, The learning rate is set to 0.001 to avoid parameter oscillation. This is the gradient of the loss function with respect to the old parameters.

[0082] The specific operation consists of four steps: "monitoring accuracy, extracting samples, training, and updating the model," all fully automated without manual intervention. Step 1: Monitoring Accuracy: Edge computing nodes statistically analyze the defect identification accuracy of each model's dedicated detection model in real time. At the end of each day, the "number of matching defects / total number of defects" for all parts of that model is calculated. For example, if model A detects 100 parts and 94 are matching, the accuracy is 94%. Step 2: Triggering Optimization: When the accuracy of a model for a certain model falls below 95% for 50 consecutive parts (this threshold is determined according to industry standards; a score below 95% affects detection quality), iterative optimization is triggered. For example, if model A achieves 94% accuracy for 50 consecutive parts, the trigger condition is met. Step 3: Incremental Optimization. Training: 300 samples of model A are extracted from the model optimization sample library, of which 20 are misclassified samples and 280 are normal samples. These samples all have corresponding spectral data, clear images, and manual verification results. Then, the model is trained using an incremental training strategy, freezing the backbone network of the model without changing its parameters, and only fine-tuning the parameters of the classification head, such as adjusting the defect judgment threshold. The training is divided into 10 batches, with 30 samples in each batch. The training time is less than 3 minutes and does not affect normal detection. Step 4: Update the model: After training, the accuracy of the new model recovers to 99.2%. The edge computing node automatically replaces the old model with the new model and synchronizes it to the spectral-visual fusion defect recognition step of S4. The new model will be used to detect model A in the future.

[0083] Application Example: After three months of use, the accuracy of Model A dropped from 99% to 94%, with 50 consecutive parts showing an accuracy of 94%. S6 extracted 300 samples of Model A from the sample library, including 20 previously misclassified samples, such as normal aluminum textures being mistaken for burrs. Incremental training then began. After training, the new model was used to detect Model A parts produced that day, and the accuracy rebounded to 99.2%. Normal textures that were previously misclassified were now correctly identified. The entire optimization process took 3 minutes, completed precisely between two batches of parts produced, without disrupting production.

[0084] Existing publicly available documents describe sheet metal inspection model optimization methods that require a full-scale training run with a shutdown period of 2 hours, impacting production. Another publicly available document describes an adaptive sheet metal inspection model that does not use misclassified samples for training, resulting in poor optimization and an accuracy rate that only recovers to 96%. Our S6 system uses "non-stop incremental training + targeted optimization based on misclassified samples," employing weighted loss and parameter update formulas to reduce training time from 2 hours to 3 minutes and improve accuracy from 94% to 99.2%. It also reduces manual maintenance hours by 1200 hours annually, whereas previously staff spent 100 hours per month manually calibrating the model.

[0085] As can be seen from the above, the existing public documents and their shortcomings are as follows: A low-temperature sheet metal inspection device: discloses sheet metal visual inspection technology in low-temperature environments, including a low-temperature camera and a single vision module, but lacks a multi-model fingerprint database, a temperature dynamic compensation formula, and a spectral-visual fusion algorithm; A multispectral sheet metal inspection system: discloses multispectral material recognition technology, but the hardware is not adapted to low temperatures, lacks a dual fingerprint similarity formula, and lacks a temporal LSTM model; A sheet metal defect early warning device: discloses defect early warning technology based on time-series data, but lacks a temperature correlation formula, a model incremental training loss function, and low early warning accuracy; Sheet metal inspection model optimization method: discloses the optimization technology of the inspection model, but requires full training with system shutdown, lacks a weighted loss formula, and lacks targeted optimization of misjudged samples.

[0086] The core differences of this technical solution lie in: integrating a "low-temperature adapted camera + hyperspectral module + multi-model fingerprint database," and clearly defining the selection criteria for hyperspectral wavelength coverage of characteristic peaks within ±10nm, thus solving the problems of "low-temperature incompatibility and multi-model incompatibility," significantly improving scene adaptability, and enabling continuous 24-hour fault-free hardware operation; achieving dynamic compensation through gain and threshold adjustment formulas, improving image clarity by 40% and reducing the false detection rate from 28% to 3%; calculating matching degree using a dual fingerprint similarity formula, eliminating the need for code pasting, reducing switching time from 5 minutes to 10 seconds, and achieving a matching accuracy of 98.5%; fusing spectral and visual information through spectral angle matching and pixel superposition formulas, reducing the false judgment rate from 22% to 0.5%, and achieving a positioning accuracy of ±0.01mm; predicting trends using the LSTM core formula, providing early warnings of 15 items, and reducing the continuous scrap rate from 12% to 0.7%; and achieving incremental optimization using a weighted loss formula, reducing training time to 3 minutes, increasing accuracy to 99.2%, and reducing maintenance time by 1200 hours / year. In summary, this system addresses the industry pain points of low-temperature, multi-model precision sheet metal inspection through the coordinated operation of six steps, demonstrating significant practical value.

Claims

1. A precision sheet metal parts visual inspection system, characterized in that: Includes the following steps: S1 hardware architecture includes a low-temperature adaptive industrial camera, a hyperspectral imaging module, a temperature sensor, an edge computing node, a mold parameter interaction module, and a feature storage server. The feature storage server pre-stores feature fingerprint databases for multiple models. The fingerprint data for each model in the feature fingerprint database includes preset material spectral feature wavelength ranges and preset shape topology feature parameters. S2 performs temperature compensation preprocessing: The ambient temperature is collected and detected in real time by a temperature sensor. The gain of the blue channel of the image and the histogram equalization threshold are dynamically adjusted according to the difference between the collected temperature and the preset reference temperature. The lower the temperature, the greater the gain increase and the greater the threshold decrease. After Gaussian filtering, a clear image without fog is output. S3 performs automatic multi-model selection: Based on the clear image output by S2, the shape topology fingerprint of the sheet metal part to be detected is extracted, including curvature, rib spacing and hole arrangement. Based on the spectral data collected by the hyperspectral imaging module, feature peaks are filtered through preset material spectral characteristic wavelength ranges to extract the material spectral fingerprint of the sheet metal part to be detected. The two fingerprints are input into the multi-model feature fingerprint library for similarity calculation. After successful matching, the dedicated detection model of the corresponding model is automatically loaded. S4 performs spectral-visual fusion defect recognition: The dedicated detection model determined in S3 is loaded, and spectral data is collected through the hyperspectral imaging module to generate a material boundary mask. The material boundary mask is then superimposed on the clear image output in S2 and input into the dedicated detection model to identify the defect type, location, and size, and output a defect annotation map. The specific steps for generating the material boundary mask are as follows: S41: Determine the characteristic wavelengths corresponding to each material of the sheet metal part to be inspected based on the preset material spectral characteristic wavelength range; S42: Perform band segmentation on the spectral data collected by the hyperspectral imaging module to obtain single-band images corresponding to each material; S43: Perform binarization processing on the single-band images of each material, assign colors to different material regions according to preset color coding rules, and generate a material boundary mask. The steps for superimposing the material boundary mask on the clear image are as follows: S44: Perform coordinate calibration on the material boundary mask and the clear image to ensure a one-to-one correspondence between their pixel positions; S45: Use a pixel superposition algorithm to cover the corresponding position of the clear image with the material boundary mask. S5 performs time-series defect trend prediction and early warning: It constructs a time-series database by storing temperature data obtained by S2, defect data output by S4, and model information determined by S3 through edge computing nodes. Based on the time-series database, it predicts defect trends. When the defect exceeds the tolerance, it outputs mold parameter adjustment instructions and marks misjudged samples and stores them in the model optimization sample library.

2. The precision sheet metal parts visual inspection system as described in claim 1, characterized in that: It also includes S6 model iteration optimization: the defect recognition accuracy of each model-specific detection model is statistically analyzed in real time through edge computing nodes. When the recognition accuracy of a certain model-specific detection model for a certain number of parts is lower than the preset accuracy threshold, the misjudged samples and corresponding data of that model are extracted from the model optimization sample library. The incremental training strategy of freezing the backbone network of the model-specific detection model and only fine-tuning the model classification head is adopted. The extracted data is input into the model for training in a preset batch. After the training is completed, the new model is updated to the model-specific detection model of that model and synchronized to the spectral-visual fusion defect recognition step of S4.

3. The precision sheet metal parts visual inspection system as described in claim 1, characterized in that: In S1, the temperature range of the low-temperature adaptive industrial camera covers from -30 degrees Celsius to -5 degrees Celsius, the wavelength range of the hyperspectral imaging module covers from 400 nanometers to 1,000 nanometers, and the measurement accuracy of the temperature sensor is ±0.2 degrees Celsius.

4. The precision sheet metal parts visual inspection system as described in claim 1, characterized in that: In step S3, the shape topology fingerprint extraction step includes: S31: performing edge detection on the clear image output from S2 to determine the contour boundary of the sheet metal part to be detected; S32: calculating the curvature value, rib spacing value, and row and column spacing value of the hole arrangement of the part based on the contour boundary; S33: comparing the calculated values ​​with the preset shape topology feature parameters of the corresponding model in the multi-model feature fingerprint library to form a shape topology fingerprint; the material spectral fingerprint extraction step includes: S34: performing noise reduction processing on the spectral data collected by the hyperspectral imaging module; S35: filtering spectral peaks within the preset material spectral feature wavelength range; S36: comparing the filtered spectral peaks with the preset material spectral feature wavelengths of the corresponding model in the multi-model feature fingerprint library to form a material spectral fingerprint.

5. The precision sheet metal parts visual inspection system as described in claim 1, characterized in that: In S5, the construction rule of the time-series database is to collect temperature data, defect data and model information at the interval of storing data once for each preset number of sheet metal parts to be inspected. The training samples of the temperature-defect correlation model are the historical temperature data and corresponding defect data stored in the time-series database, and the number of training samples is not less than the preset sample number threshold. The mold parameter adjustment instructions include the stamping pressure adjustment value, the laser power adjustment value and the adjustment execution time.

6. The precision sheet metal parts visual inspection system as described in claim 2, characterized in that: In S6, the method for calculating the accuracy of defect identification is as follows: compare the defect annotation map output by S4 with the manual review result, calculate the ratio of the number of matching defects to the total number of defects, the preset number of sheet metal parts to be inspected is fifty, the preset accuracy threshold is ninety-five percent, the preset batch for incremental training is ten batches and the sample size of each batch is thirty pieces.

7. The precision sheet metal parts visual inspection system as described in claim 4, characterized in that: In S31, edge detection uses the Canny algorithm; in S3, the preset matching threshold is 98.5%; in S34, the denoising process uses the wavelet transform denoising algorithm; and in S35, the extraction of spectral peaks within the preset material spectral characteristic wavelength range uses the threshold segmentation method, with the threshold being 50% of the maximum value of the spectral data.

Images