Visual inspection system for precise sheet metal parts

Abstract

The invention discloses a precision sheet metal part visual detection system, and relates to the technical field of precision sheet metal detection. According to the system, a hardware architecture comprising a low-temperature adaptive industrial camera, a hyperspectral imaging module and a pre-stored multi-model feature fingerprint database is built; according to the method, image compensation of temperature difference value dynamic adjustment, model automatic identification of material spectrum and shape topology double-fingerprint matching, defect detection of combining spectrum angle matching with visual images, LSTM time sequence defect early warning integrated with temperature data and non-stop incremental model optimization of a frozen backbone network are matched, and a whole-process automatic detection scheme is constructed. Image blurring caused by low temperature is effectively eliminated, multiple types of sheet metal parts are quickly and accurately adapted, the defect misjudgment risk is greatly reduced, the defect development trend is warned in advance, the high recognition precision of the model is continuously maintained, the detection efficiency and precision are remarkably improved, the production loss is reduced, and the precision detection requirement of core components of cold chain equipment is fully met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of precision sheet metal detection, in particular to a precision sheet metal part visual detection system. BACKGROUND

[0002] The evaporator shell in the cold chain equipment as a key heat exchange component, its sheet metal structure often adopts aluminum-steel composite, stainless steel single material, aluminum-copper composite with reinforcing rib and other multiple model designs, and in order to ensure the heat exchange efficiency, the hole position precision (error needs to be less than or equal to ±0.01mm), burr (needs to be less than or equal to 0.05mm) and crack (needs to be less than or equal to 0.1mm) and other defect detection requirements are very high. At present, the detection of such sheet metal parts in the industry relies on manual operation with single visual equipment, and manual judgment of part models and adjustment of detection parameters are required during the detection process, which not only has low efficiency (single piece detection time is more than 12 minutes), but also has strong dependence on the experience of detection personnel, and it is difficult to meet the batch production demand.

[0003] However, the detection environment of the evaporator shell is often in the low temperature interval of-5℃ to-30℃, and the existing detection technology has significant limitations in this scenario: low temperature causes camera lens fogging, and image blur causes defect omission rate as high as 28%; at the same time, defects such as hole position deviation caused by mold wear can only be found afterwards, often causing 15-20 consecutive parts to be scrapped, greatly increasing production costs. In summary, the existing technology cannot solve the problem of "automated and high-precision detection of multiple types of precision sheet metal parts in low-temperature environment", which seriously restricts the production efficiency and product quality of cold chain equipment.

[0004] Therefore, the present application is proposed. SUMMARY

[0005] The purpose of the present application is to provide a precision sheet metal part visual detection system to solve the problems in the background art.

[0006] To solve the above technical problems, the precision sheet metal part visual detection system provided by the present application comprises the following steps: S1 Build hardware architecture: including low-temperature adaptive industrial camera, hyperspectral imaging module, temperature sensor, edge computing node, mold parameter interaction module, and feature storage server, the feature storage server pre-stores a multi-model feature fingerprint library, and the fingerprint data of each model in the multi-model feature fingerprint library includes a preset material spectral feature wavelength interval and a preset shape topological feature parameter; S2 Perform temperature compensation preprocessing: collect the detection environment temperature in real time through the temperature sensor, dynamically adjust the image blue channel gain and histogram equalization threshold according to the difference between the collected temperature and the preset reference temperature, the lower the temperature, the greater the gain increase and the greater the threshold decrease, and output a clear image without fog after Gaussian filtering processing; S3 performs multi-model automatic selection: based on the clear image output by S2, the shape topology fingerprint of the metal sheet part to be detected is extracted, including curvature, rib spacing, hole arrangement rule, based on the spectral data collected by the hyperspectral imaging module, the characteristic peaks are filtered through the preset material spectral feature wavelength interval, the material spectral fingerprint of the metal sheet part to be detected is extracted, and the two fingerprints are input into the multi-model feature fingerprint library for similarity calculation. After successful matching, the corresponding model detection model is automatically loaded to realize rapid and accurate matching of multi-model parts; S4 performs spectral-vision fusion defect recognition: loads the exclusive detection model determined by S3, generates a material boundary mask by collecting spectral data through the hyperspectral imaging module, and inputs the material boundary mask and the clear image output by S2 into the exclusive detection model after superposition, recognizes the defect type, position and size and outputs the defect annotation map; S5 performs time sequence defect trend prediction and early warning: a time sequence database is constructed by storing the temperature data obtained by S2, the defect data output by S4 and the model information determined by S3 through the edge computing node, the defect trend is predicted based on the time sequence database, and the mold parameter adjustment instruction is output when the tolerance is exceeded. At the same time, the misjudgment sample is marked and stored in the model optimization sample library; through dynamic temperature compensation and double-fingerprint model matching technology, the problems of poor image quality in low temperature environment and difficulty in adapting to multi-model parts are solved, the automation and precision of metal sheet part detection are realized, and the detection efficiency and defect recognition accuracy are improved.

[0007] Further, S6 model iterative optimization is further included: the defect recognition accuracy of each model exclusive detection model is counted in real time through the edge computing node, when the recognition accuracy of a certain model exclusive detection model for a continuous preset number of parts is lower than a preset accuracy threshold, the misjudgment sample and corresponding data of the model are extracted from the model optimization sample library, the exclusive detection model backbone network is frozen, and only the incremental training strategy of the model classification head is fine-tuned. The extracted data is input into the model according to the preset batch for training, and after the training is completed, the new model is updated as the exclusive detection model of the model, and is synchronized to the spectral-vision fusion defect recognition step of S4; the self-iterative optimization of the detection model is realized, the precision decay caused by long-term use of the model is avoided, the production continuity is ensured through the incremental training strategy, and the adaptability and defect recognition stability of the model to specific model parts are improved.

[0008] Furthermore, in S1, the temperature range of the low-temperature adaptive industrial camera covers from -30 degrees Celsius to -5 degrees Celsius, the wavelength range of the hyperspectral imaging module covers from 400 nanometers to 1,000 nanometers, the measurement accuracy of the temperature sensor is ±0.2 degrees Celsius, and the GPU memory of the edge computing node is no less than 12 gigabytes and meets the real-time requirements of S5 time-series data processing and S6 model training; the hardware performance parameters are clearly defined to ensure that the hardware architecture can adapt to the low-temperature detection scenario, meet the computing power requirements of hyperspectral data processing and model training, and ensure the stable operation and real-time detection of the system.

[0009] Further, in step S3, the shape topology fingerprint extraction step includes: S31: performing edge detection on the clear image output in S2 to determine the contour boundary of the sheet metal part to be detected; S32: calculating the curvature value, rib spacing value, and row and column spacing value of the hole arrangement of the part based on the contour boundary; S33: comparing the calculated values ​​with the preset shape topology feature parameters of the corresponding model in the multi-model feature fingerprint library to form a shape topology fingerprint; the material spectral fingerprint extraction step includes: S34: performing noise reduction processing on the spectral data collected by the hyperspectral imaging module; S35: filtering spectral peaks within the preset material spectral feature wavelength range; S36: comparing the filtered spectral peaks with the preset material spectral feature wavelengths of the corresponding model in the multi-model feature fingerprint library to form a material spectral fingerprint; this standardizes the fingerprint extraction process, improves the extraction accuracy of shape topology fingerprints and material spectral fingerprints, further improves the accuracy of automatic selection of multiple models, and reduces defect identification deviations caused by model mismatch.

[0010] Further, in step S4, the specific steps for generating the material boundary mask are as follows: S41: Determine the characteristic wavelengths corresponding to each material of the sheet metal part to be inspected based on the preset material spectral characteristic wavelength range; S42: Perform band segmentation on the spectral data collected by the hyperspectral imaging module to obtain single-band images corresponding to each material; S43: Perform binarization processing on the single-band images of each material, assign colors to different material regions according to preset color coding rules, and generate material boundary masks; The step of superimposing the material boundary mask with the clear image is as follows: S44: Perform coordinate calibration on the material boundary mask and the clear image to ensure that the pixel positions of the two correspond one-to-one; S45: Use a pixel superposition algorithm to cover the corresponding position of the clear image with the material boundary mask; By accurately generating and superimposing the material boundary mask, the problem of defect misjudgment caused by the blurring of material boundaries in the detection of multi-material parts is solved, and the accuracy of defect identification and positioning is improved.

[0011] Further, in the S5, the construction rule of the time series database is to store the temperature data, defect data and model information collected at intervals of a preset number of detected sheet metal parts, the training sample of the temperature-defect correlation model is the historical temperature data and the corresponding defect data stored in the time series database, and the number of training samples is not less than a preset sample number threshold, and the mold parameter adjustment instruction includes a stamping force adjustment value, a laser power adjustment value and an adjustment execution time; The accuracy of the time series defect trend prediction and the executability of the mold adjustment are ensured, sufficient and reliable training data are provided for the temperature-defect correlation model, the generation of batch defects is reduced, and the production efficiency is improved.

[0012] Further, in the S6, the method for calculating the defect recognition accuracy is to compare the defect labeling map output by S4 with the manual review result, calculate the ratio of the number of consistent defects to the total number of defects, the preset number of sheet metal parts to be detected is fifty, the preset accuracy threshold is ninety-five percent, the preset batch of incremental training is ten batches and the number of samples in each batch is thirty; The trigger condition and training parameters of the model iteration optimization are quantified, the model optimization process is controllable and repeatable, the unstable optimization effect caused by ambiguous parameters is avoided, and the recognition accuracy of the updated model is ensured.

[0013] Further, in the S31, the edge detection uses the Canny algorithm, the preset matching threshold in S3 is ninety-eight point five percent, the denoising processing in S34 uses the wavelet transform denoising algorithm, and the spectral peak extraction in the preset material spectral feature wavelength range in S35 uses the threshold segmentation method and the threshold is fifty percent of the maximum value of the spectral data; The key algorithms and thresholds in the fingerprint extraction process are clarified, the multi-model identification process is further standardized, the edge detection accuracy and the spectral data quality are improved, and the reliability of the multi-model automatic selection is ensured.

[0014] Compared with the prior art, the beneficial effects of the present application are: 1. By building a hardware architecture including a low-temperature adaptive industrial camera, a hyperspectral imaging module and a pre-stored multi-model feature fingerprint library, the limitations of the existing hardware "low-temperature incompatibility and multi-model incompatibility" are broken through. The low-temperature adaptive camera ensures stable image output in a low-temperature environment, the hyperspectral module accurately captures the characteristic wavelengths of different materials, and the multi-model feature fingerprint library provides standardized data support for subsequent model identification. The three work together to make the hardware system stable and compatible with the multi-model detection scene of the evaporator shell of the cold chain equipment, greatly enhancing the continuous operation capability and significantly widening the scene adaptation range.

[0015] 2. Through the two technical means of "temperature difference dynamic adjustment parameter temperature compensation" and "material spectrum fingerprint + shape topology fingerprint double fingerprint matching", the industry pain points of low temperature image blur and multi-model manual switching are solved respectively. The temperature compensation link dynamically adjusts the image channel gain and equalization threshold according to the difference between the detection temperature and the reference temperature, and then combines the Gaussian filter to effectively eliminate the image blur caused by low temperature fog, and provide clear and reliable image data for subsequent detection; The double fingerprint matching link extracts the shape topology features and material spectrum features of the part, calculates the comprehensive similarity to realize automatic identification of the model, and can quickly load the corresponding detection model without manual intervention, completely replacing the traditional manual switching mode, not only greatly shortening the model switching time, but also avoiding the risk of manual misjudgment, The efficiency and accuracy of model identification are improved in quality.

[0016] 3. Through the technical means of "spectrum angle matching to generate material boundary mask + pixel-level image fusion", the problem of "material boundary blur leading to defect misjudgment" in multi-material sheet metal part detection is solved. The spectrum angle matching algorithm accurately distinguishes different materials and generates a boundary mask, and pixel-level fusion highlights the material boundary while preserving the details of the visual image, so that the detection model can identify defects for different materials, effectively avoiding misjudgment due to differences in multi-material reflection, and the recognition ability of subtle defects (such as small burrs and fine cracks) is significantly enhanced, and the defect detection accuracy is greatly improved.

[0017] 4. Through the unique technical means of "LSTM time series prediction model combined with temperature data", the early warning of defects is realized. This model combines temperature data with historical defect data to accurately capture the correlation between temperature changes and defect development, and can predict the defect trend of subsequent parts in advance. When the predicted defect exceeds the tolerance, automatically output the mold adjustment instruction, without stopping the machine, the mold parameter correction can be completed, changing the traditional passive mode of "discovering defects after the fact", effectively reducing the continuous scrap caused by mold wear, and greatly reducing production losses. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the precise sheet metal part visual detection system. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0020] Please refer to Figure 1The application provides a technical solution: a precision sheet metal part visual detection system. The embodiment relates to the field of precision sheet metal detection, and is especially suitable for batch detection of evaporator shell sheet metal parts of low-temperature cold chain equipment core components. In the current cold chain industry, evaporator shells are designed in multiple models such as aluminum-steel composite, stainless steel single material, aluminum-copper composite with reinforcing ribs, and the detection environment is often in a low-temperature range of-5°C to-30°C. The existing detection methods have three major pain points: first, the lens fogs at low temperatures, causing image blurring, and the missed detection rate is as high as 28%; second, manual switching of detection models is required when mixed production of multiple models, and the time consumed by single switching is more than 5 minutes; and third, defects caused by mold wear can only be found afterwards, often causing 15-20 consecutive scrap. To solve these problems, the system integrates six aspects of hardware architecture, temperature compensation, multiple model matching, spectral-vision fusion, time sequence early warning and model optimization, and realizes full-process automatic and accurate detection.

[0021] (I) Build a hardware architecture: the core is to build a hardware system suitable for low-temperature multi-model detection. If the hardware is not selected correctly, such as a camera that cannot withstand low temperatures and insufficient computing power, even the most precise algorithms cannot be implemented. Ordinary industrial cameras will have lens fogging and slow shutter response below-15°C; without a hyperspectral module, the boundaries of multiple materials cannot be distinguished, making it difficult to identify defects.

[0022] The hardware selection refers to the low-temperature environment device adaptation requirements in the "Industrial Machine Vision System Design Specification" and the wavelength range recommendations in the "Application Guide of Hyperspectral Imaging Technology in Industrial Detection". At the same time, combined with the detection accuracy requirements of cold chain sheet metal parts, the size error is ±0.01mm, and the minimum defect recognition is 0.05mm, the performance parameters of each hardware are determined. Among them, the wavelength range selection of the hyperspectral module needs to cover the target material characteristic peak ±10nm interval to ensure that the characteristic peak capture rate is ≥99%, and this parameter is determined according to the material spectral characteristic test experiment.

[0023] The hardware architecture includes six core components: low-temperature adaptive industrial camera: adaptive temperature range covering -30°C to -5°C, resolution 5 million pixels, frame rate 20 fps, can keep image stable output at low temperature, avoid lens fogging or shutter jamming; hyperspectral imaging module: uses hyperspectral camera, wavelength range 400nm to 1000nm, spectral resolution 5nm, just covers the characteristic wavelength interval of aluminum alloy 532nm characteristic peak, steel 650nm characteristic peak, stainless steel 580nm characteristic peak, can accurately distinguish different materials; temperature sensor: measurement accuracy ±0.2°C, sampling frequency 1Hz, real-time acquisition of the environmental temperature of the detection station, providing data support for subsequent temperature compensation; edge computing node: GPU memory 16GB meets the requirement of ≥12GB, CPU is 8-core ARM Cortex-A78, can simultaneously process hyperspectral data, time series prediction model and incremental training task, ensuring that the detection process does not jam; mold parameter interaction module: uses RS485 communication interface to connect with puncher and laser welding machine, supports real-time sending of parameter adjustment instructions such as punching force and laser power, response delay ≤500ms; feature storage server: storage capacity 1TB SSD, pre-stores multiple model feature fingerprint library of 3 types of cold chain evaporator shells, model A is aluminum-steel composite flat plate type, its fingerprint contains aluminum 532nm / steel 650nm spectral interval, flat plate curvature 0, i.e. plane, hole arrangement spacing 10mm; model B is stainless steel arc, its fingerprint contains stainless steel 580nm spectral interval, arc curvature 800mm, no hole; model C is aluminum-copper composite belt reinforcement type, its fingerprint contains aluminum 532nm / copper 510nm spectral interval, reinforcement spacing 20mm, hole arrangement spacing 8mm.

[0024] Application example: In the evaporator production line of a certain cold chain equipment factory, we set up hardware according to S1, connect the feature storage server with the production line MES system, and synchronize the production plan of the day in real time, such as producing 100 pieces of model A in the morning and 80 pieces of model B in the afternoon. After the hardware is powered on, each component passes the self-checking, the low-temperature camera works continuously for 4 hours without fogging at -20°C, and the hyperspectral module can clearly see the double characteristic peaks of 532nm and 650nm in the spectral data of model A, fully meeting the subsequent detection requirements.

[0025] Existing public documents, a low-temperature sheet metal detection device, only contains low-temperature camera and ordinary vision module, without hyperspectral imaging module and multiple model feature fingerprint library, cannot distinguish multiple materials and multiple models, hardware is not adapted to low-temperature environment, cannot work stably below -10°C. The hardware architecture of this system solves the problem of "low-temperature incompatibility and multiple model incompatibility" of existing hardware through the combination of "low-temperature adaptation + multi-spectrum + high computing power", greatly improves the scene adaptation rate, and prolongs the continuous stable running time of hardware from 8 hours to 24 hours.

[0026] (ii) S2 performs temperature compensation preprocessing: mainly to solve the problem of image blur at low temperature. When the ambient temperature drops below -15℃, a thin layer of frost will form on the camera lens, resulting in a white image and blurred details. The hole position of the evaporator shell cannot be seen clearly, and 0.05mm burrs cannot be identified.

[0027] The temperature compensation technique is based on the principles of "histogram equalization for image enhancement" and "Gaussian filter for noise suppression" from "Digital Image Processing". Combined with the color deviation law of images in low temperature environment, the blue channel attenuation is most obvious, and the channel gain is adjusted accordingly. We have done experiments, and the blue channel gray value of the image at low temperature is 15%-20% lower than that at normal temperature. Adjusting the blue channel gain can effectively restore color balance.

[0028] Among them, the dynamic parameter adjustment adopts the following formula: (1) Blue channel gain adjustment formula: ; In the formula, is the adjusted blue channel gain, is the default gain at normal temperature (value 1.0), is the gain adjustment coefficient (value 0.03 / ℃, i.e. the gain increases by 0.15 for every 5℃ drop in temperature), is the difference between the reference temperature (-10℃) and the actual detection temperature (T , , is the actual temperature); (2) Histogram equalization threshold adjustment formula: ; In the formula, is the adjusted threshold, is the default threshold at normal temperature (value 0.8), m is the threshold adjustment coefficient (value 0.02 / ℃, i.e. the threshold decreases by 0.1 for every 5℃ drop in temperature), the meaning is the same as above; (3) Gaussian filter formula: ; In the formula, is the pixel coordinate in the filter kernel, is the standard deviation (value 0.9), which ensures that the hole position, rib and other detail features are retained while denoising.

[0029] The specific operation process is divided into three steps, each step is related to temperature, which is not a fixed parameter: the first step: temperature collection: the temperature sensor collects the environmental temperature in real time, for example, the temperature collected in the morning of a certain day is-20℃, and the preset reference temperature is-10℃, which is determined according to a large number of experiments. Above-10℃, the image is not obviously blurred, and below-10℃, it needs to be compensated. The second step: parameter adjustment: according to the difference between the collected temperature and the reference temperature, -20℃-(-10℃)=-10℃, adjust according to the rule that "the blue channel gain is increased by 15% and the histogram equalization threshold is reduced by 0.1 for every 5℃ decrease in temperature". Here, -10℃ difference corresponds to a decrease of 2 5℃, so the blue channel gain is increased from the default 1.0 to 1.3, that is, 1.0+2x15%, and the histogram equalization threshold is reduced from the default 0.8 to 0.6, that is, 0.8-2x0.1. The third step: image filtering: perform Gaussian filtering on the adjusted image, with a filter kernel size of 3x3 and a standard deviation , the value is tried out, too small to filter out frost mist noise, too large to blur details, 0.9 just balances between denoising and detail preservation, finally outputting a clear image without fog.

[0030] Application example: still use model A for detection, in-20℃ environment, without temperature compensation, the image of model A shows that the boundary between aluminum layer and steel layer is blurred into a ball, and the edge of the hole has obvious "fringe"; after S2 processing, the silver-white aluminum layer and the dark gray steel layer are clearly distinguished in the image, and the hole edge is smooth. The diameter of the hole is measured by image analysis software, and the error is reduced from ±0.03mm to ±0.01mm, which fully meets the requirements of subsequent shape topology fingerprint extraction.

[0031] In the existing public document, a low-temperature sheet metal detection device, the low-temperature image enhancement uses fixed histogram equalization parameters, regardless of the temperature is-15℃ or-30℃, the same threshold is used, resulting in over-enhanced image and amplified noise at-30℃; another public document, a low-temperature image processing method, only adjusts the brightness, not the channel gain, the image color problem is not solved. S2 of the system adjusts the parameters dynamically according to the temperature difference, which adds a layer of temperature correlation logic to the fixed parameter scheme, and the image clarity is improved by 40%. The defect miss detection rate at-30℃ under low temperature is reduced from 28% to 3%, meeting the requirements of precision detection.

[0032] (Three) S3 executes multi-model automatic selection: identifies the part model and loads the corresponding detection model. In multi-model mixed production, the model is switched manually on the computer by the staff, model A in the morning and model B in the afternoon. The staff needs to find the corresponding folder and load the model. The whole process takes at least 5 minutes, and it is easy to select the wrong model, such as using model B on model A, resulting in false judgment of all defects.

[0033] The basis for multi-model identification is the dual matching of "material spectral fingerprint + shape topological fingerprint". Different materials have unique spectral characteristic peaks, such as 532nm for aluminum and 580nm for stainless steel. Different shapes also have different topological parameters, such as curvature and rib spacing. Combining the two can distinguish the models with 100% accuracy. This approach is based on the "multi-feature fusion classification" method in "Pattern Recognition", which has a much higher accuracy than single-feature identification.

[0034] The weighted summation formula is used to calculate the similarity between the two fingerprints: In the formula, The overall similarity score is calculated (values ​​range from 0 to 1, with a threshold of 0.985). The shape topology fingerprint weight (value 0.4). The weight of the material spectral fingerprint is set to 0.6, because the material spectrum has higher distinguishability. Shape topological fingerprint similarity (calculated using Euclidean distance based on curvature, rib spacing, and hole arrangement spacing). The similarity of the material's spectral fingerprint is calculated using the deviation rate of the characteristic peak positions.

[0035] The calculation formula is as follows: ; In the formula, The curvature of the part to be tested. Preset curvature for the models in the library; The spacing between the ribs of the part to be inspected is 0 (0 if there are no ribs). Preset the rib spacing for the models in the library; The spacing between holes in the part to be inspected (0 if there are no holes). Preset the hole spacing for the models in the library; The calculation formula is as follows: ; In the formula, The number of characteristic peaks (e.g., aluminum-steel composites) ), For the part to be inspected One characteristic peak wavelength, For model number in the library Each characteristic peak has a preset wavelength.

[0036] The specific technical means operation is divided into "fingerprint extraction, fingerprint comparison, model loading" three steps, and each step has clear standards: the first step: extracting double fingerprints: shape topology fingerprint: from the clear image output by S2, use edge detection algorithm, which will be explained later, find the contour of model A, calculate its curvature, which is flat plate curvature 0, inter-muscle distance, which is no muscle, so the inter-muscle distance is 0, and the hole arrangement distance is 10 mm. These three parameters are the shape topology fingerprint; material spectrum fingerprint: hyperspectral imaging module collects the spectral data of model A, denoising, S34 uses wavelet transform to denoise, then filters in the preset material spectrum characteristic wavelength interval, aluminum 520-540 nm, steel 640-660 nm, finds two characteristic peaks at 532 nm and 650 nm. The position and intensity of these two peaks are the material spectrum fingerprint; the second step: fingerprint matching: input the double fingerprints into the multi-model characteristic fingerprint library of the feature storage server, calculate the similarity, compare the double fingerprints of model A with the preset fingerprints of model A in the library, the similarity reaches 98.5%, this threshold is debugged, below 98% may be part deformation, above 98.5% is matching success; the third step: loading model: after matching success, the edge computing node automatically loads the exclusive detection model of model A from the server. This model is trained in advance for the defect types of model A, such as aluminum layer burr and steel layer crack, and the loading time is less than 10 seconds.

[0037] Application example: a factory switches production models from model A to model B at noon: after S2 outputs the clear image of model B, S3 automatically extracts its shape topology fingerprint, i.e. curvature 800 mm, no muscle, no hole, and material spectrum fingerprint, i.e. 580 nm characteristic peak, and compares it with the fingerprint library with a similarity of 99.2%. Then load the exclusive model of model B, the whole process is completed in 12 seconds. Before that, manual switching takes 5 minutes and is prone to errors.

[0038] Existing public documents, multi-spectral sheet metal detection system, rely on two-dimensional code to identify model, parts on the paste two-dimensional code, high temperature welding two-dimensional code is easy to fall off, recognition rate less than 90%; another public document, multi-model sheet metal detection device, only rely on shape recognition, encounter shape similar but different material model, such as aluminum plate and steel plate will be mistaken. S3 of the system uses "double fingerprint matching" and corresponding similarity calculation formula, without pasting two-dimensional code, can also distinguish similar models, matching accuracy is 98.5%, switching time is shortened from 5 minutes to 10 seconds, solving the adaptation problem of multi-model mixed production.

[0039] (4) S4 performs spectrum-vision fusion defect recognition: accurately find defects on the part. When using single vision detection before, the disadvantage is the difference in reflectivity of multiple materials. The reflectivity of aluminum is 85%, and the reflectivity of steel is 55%. In the image taken, the normal lines of the aluminum layer and the cracks of the steel layer look similar, with a false positive rate as high as 22%. S4 uses spectrum-vision fusion method to distinguish materials first, and then find defects, with a false positive rate greatly reduced.

[0040] The technology of spectrum-vision fusion is based on "spectrum angle matching algorithm" to distinguish different materials and "image pixel level superposition technology" to combine spectrum and vision information. Referring to the method of "spectrum angle matching for material identification" in "hyperspectral remote sensing image classification and application", and combining the demand of "defect positioning needs visual image details" in industrial detection, the two are fused, spectrum is responsible for material separation, and vision is responsible for finding details.

[0041] Among them, the formula of spectrum angle matching algorithm is: ; In the formula, is the angle (unit: radian) between the to-be-detected spectrum vector and the reference material spectrum vector , is the dot product of the two vectors, , L2 norm of the two vectors respectively; when ( is the threshold value, taking 0.1 radian) is determined as the same material; The formula of pixel level superposition is: ; In the formula, is the gray value of the fused image at , is the weight of the vision image (taking 0.7, retaining details), is the weight of the spectrum mask (taking 0.3, highlighting the material boundary), is the gray value of the vision image output by S2, is the material boundary mask gray value (different materials are set to 255, 128, etc. Distinction value).

[0042] The specific operation is divided into three steps: "generating material mask, image superposition, and defect identification". Each step is designed for multi-material defect detection: Step 1: Generate material boundary mask: First, determine the characteristic wavelength corresponding to each material of model A, aluminum 532nm, steel 650nm; The hyperspectral imaging module collects the spectral data of model A, uses the spectral angle matching algorithm for band segmentation, extracts the band near 532nm as the single-band image of the aluminum layer, and the band near 650nm as the single-band image of the steel layer; The single-band image is binarized, the threshold is set to 1.2 times the spectral mean, then according to the preset color coding rule, the aluminum layer is set to white and the steel layer is set to gray, and the material boundary mask is generated. The white area in the mask is the aluminum layer and the gray area is the steel layer, with clear boundaries; Step 2: Image superposition: Coordinate calibration is performed on the material boundary mask and the clear image output by S2 using two positioning holes on the part as a reference to ensure that the pixel positions of the mask and the image correspond one by one, then the pixel superposition algorithm is used to cover the mask on the clear image to form a fusion image with "material labeling + detailed clear"; Step 3: Defect identification: input the fusion image into the exclusive detection model of model A, the model has two branches, the aluminum layer branch finds burrs with a recognition threshold of 0.05mm, and the steel layer branch finds cracks with a recognition threshold of 0.1mm, and finally outputs a defect labeling image, labeling the defect type such as aluminum layer burr, position such as hole 3 o'clock direction, and size such as 0.06mm.

[0043] Application example: when detecting a certain part of model A, the fusion image generated by S4 has clear boundaries between the aluminum layer and the steel layer, with no ambiguous lines. The model identifies that the No. 3 hole in the aluminum layer has a 0.06mm burr, which exceeds the threshold of 0.05mm, and the steel layer weld has a 0.12mm crack, which exceeds the threshold of 0.1mm. The labeling image is directly displayed on the display screen in the workshop, and the staff can immediately identify the problem. Previously, when using single-vision detection, the aluminum layer burr of this part was considered as normal texture and the steel layer crack was considered as welding mark, which caused missed detection. The present technical solution greatly reduces the possibility of missed detection.

[0044] Existing public documents, multi-spectral sheet metal detection system, only use spectral recognition of material, without combining the details of the visual image, the defect positioning accuracy is ±0.1mm, which cannot meet the precision requirements; another public document, a low-temperature sheet metal detection device only uses visual detection, with a misjudgment rate of 22%. The S4 of the present system fuses spectrum and vision through spectral angle matching and pixel superposition formula, which not only distinguishes materials but also retains details, reduces the defect misjudgment rate from 22% to 0.5%, and improves the positioning accuracy to ±0.01mm, which can identify 0.05mm tiny burrs, meeting the precision detection standards of cold chain evaporators.

[0045] (V) S5 Defect Trend Prediction and Early Warning: Its function is to detect problems in advance and avoid batch scrapping. Previously, we encountered a situation where mold wear occurred. The punch of the punching machine wore down, and the hole position offset of model A initially was 0.08mm, gradually increasing to 0.09mm, 0.10mm, until it exceeded the tolerance of 0.1mm before being discovered. By then, 18 parts had already been scrapped, resulting in financial loss. S5 analyzes historical data to predict in advance that the hole position offset will exceed the tolerance, and then issues instructions to adjust the mold, minimizing losses.

[0046] The basis for predicting time-series defects is the LSTM (Long Short-Term Memory) network model. This model excels at processing time-series data and can capture data trends. Referring to "Application of LSTM Model in Industrial Forecasting" in "Time Series Analysis and Forecasting," and considering the characteristic in sheet metal inspection that "temperature affects defect formation"—for example, parts are more brittle at low temperatures and cracks are more likely to appear—temperature data is also incorporated into the prediction model to make the prediction more accurate.

[0047] The core formula of the LSTM model is as follows: (1) Gate of Oblivion: ; (2) Input gate: , ; (3) Cell state renewal: ; (4) Output gate: , ; In the formula, For time steps, , , These are the forget gate, input gate, and output gate, respectively. (for sigmoid activation function) , These represent the cell state and the hidden layer output, respectively. for Input characteristics (including temperature, historical defect values, and model code) at all times. , , , This is the weight matrix. , , , For bias terms, Element-wise multiplication; Defect trend prediction threshold formula: In the formula, For prediction Time-based defect values ​​(such as hole position offset). a defect tolerance threshold (e.g., 0.1 mm), a prediction step (taking a value of 15, i.e., warning 15 pieces in advance).

[0048] The specific operation is divided into four steps of "building a database, making a prediction, issuing an instruction, and marking a sample", forming a data closed loop: First step: constructing a time series database: the edge computing node stores the temperature data of S2 such as -20℃, -19℃, -21℃, the defect data of S4 such as the hole position offset mean of model A 0.08mm, 0.09mm, 0.10mm, the crack quantity 2 pieces, 3 pieces, 4 pieces, and the model information of S3 model A every 10 pieces of detected parts, and stores them into the database. The data retains the records of the latest 1000 pieces of parts; Second step: defect trend prediction: extract the time series data of the latest 100 pieces of model A from the database, input the temperature-defect correlation LSTM model, the model will learn the relationship between temperature and defect, such as temperature decreases by 5℃, hole position offset increases by 0.002mm, and then predict the defect trend of the next 20 pieces of parts. For example, the prediction result shows that the hole position offset of the next 15 pieces of model A will reach 0.11mm, exceeding the tolerance of 0.1mm; Third step: output adjustment instruction: after predicting the tolerance, the edge computing node sends an adjustment instruction to the punching machine through the die parameter interaction module, such as adjusting the punching force from 800kN to 820kN. After the punching force is increased, the punch can better shape and reduce the hole position offset. After the instruction is sent, the punching machine completes the parameter adjustment within 5 seconds; Fourth step: mark misjudged samples: when storing data, the staff will manually review the defect labeling diagram output by S4, mark the misjudged samples such as normal lines mistaken as burrs, and store them together with the corresponding spectral data and clear images into the model optimization sample library for the preparation of S6.

[0049] Application example: when producing model A in a batch, S5's time series database records the hole position offset of the last 30 pieces of parts: 0.08mm for the first 10 pieces, 0.09mm for the 11th-20th pieces, and 0.10mm for the 21st-30th pieces. The LSTM model predicts that the offset of the 31st-45th pieces will reach 0.11-0.12mm, exceeding the tolerance of 0.1mm, so it immediately sends the instruction "adjust the punching force to 820kN" to the punching machine. After the adjustment, the hole position offset of the 31st-45th pieces stabilizes at 0.095mm, without exceeding the tolerance. At the same time, 20 misjudged samples are marked, such as the normal lines of the 15th piece mistaken as burrs, and stored in the sample library.

[0050] The existing disclosure file, a sheet metal defect early warning device, only relies on defect quantity statistics for early warning, without considering the influence of temperature, and the prediction accuracy is less than 80%; another disclosure file, a mold wear detection device, needs to stop detecting the mold, affecting production. The S5 of the system combines temperature and defect data, realizes trend prediction through the LSTM model core formula, and the accuracy is more than 95%. It can also give an early warning of 15 pieces in advance, without stopping the machine, and the continuous scrap rate is reduced from 12% to 0.7%, and the single batch production cost is greatly reduced.

[0051] (Six) S6 executes model iteration optimization: avoid model accuracy decline after long-term use. We found that after using the model for 3 months, the recognition accuracy will decrease from 99% to about 94%. On the one hand, there are differences in part material batches, such as the composition of a batch of aluminum is slightly different, and the spectral peak is shifted by 2nm. On the other hand, the environment changes, such as the humidity in the workshop increases, and the image noise increases. If the model is not optimized, the misjudgment rate will be higher and higher. S6 is to keep the model accuracy high.

[0052] The basis for model iteration optimization is the "incremental training strategy". This strategy does not need to retrain the entire model, but only needs to fine-tune part of the parameters with new samples, which can improve accuracy without stopping the machine. Referring to "Incremental Training in Industrial Models" in "Deep Learning Model Optimization and Deployment", and combining the requirement that the detection system "cannot stop", the "frozen backbone network + fine-tuned classification head" method is adopted. The backbone network is responsible for feature extraction and does not need to be moved; the classification head is responsible for judging defects, and fine-tuning it can adapt to new situations.

[0053] Loss function formula of incremental training (weighted for misjudgment samples): ; In the formula, is the total loss, is the number of samples, is the weight of the th sample (normal sample , misjudgment sample , enhance the influence of misjudgment samples on training), is the true label of the th sample (defect / normal), is the model prediction label, is the cross-entropy loss function. Model parameter update formula: ; In the formula, is the updated parameter (only the classification head parameter is updated), is the parameter before updating, is the learning rate (value 0.001, to avoid parameter oscillation), is the gradient of the loss function to the old parameter.

[0054] The specific operation is divided into four steps of "looking at accuracy, extracting samples, training, and updating models", which is fully automated and does not require manual intervention: First step: Monitor accuracy: The edge computing node calculates the defect recognition accuracy of each type of dedicated detection model in real time. At the end of each day, the "number of consistent defects / total number of defects" of all parts of that type are calculated, for example, the model of type A detects 100 pieces in a day, 94 pieces are consistent, and the accuracy is 94%; Second step: Trigger optimization: When the accuracy of a model of a certain type is less than 95% for 50 consecutive parts, the threshold is determined according to industry standards, and less than 95% will affect the detection quality, at this time, iterative optimization is triggered, for example, the model of type A has a continuous accuracy of 94% for 50 pieces, reaching the trigger condition; Third step: Incremental training: Extract 300 samples of type A from the model optimization sample library, of which 20 are misjudgment samples and 280 are normal samples, these samples have corresponding spectral data, clear images and manual review results; Then use the incremental training strategy to train the model, freeze the backbone network of the model, do not change its parameters, only fine-tune the parameters of the classification head, such as adjusting the threshold of defect judgment, training in 10 batches, each batch of 30 samples, training time less than 3 minutes, without affecting normal detection; Fourth step: Update the model: After training, the accuracy of the new model is back to 99.2%, the edge computing node automatically replaces the old model with the new model, and synchronizes to the spectral-vision fusion defect recognition step of S4, and the subsequent detection of type A uses the new model.

[0055] Application example: The model of type A has been used for 3 months, the accuracy has decreased from 99% to 94%, and the accuracy of 50 consecutive pieces is 94%: S6 extracts 300 samples of type A from the sample library, of which 20 are misjudgment samples, such as normal aluminum layer lines being mistaken for burrs, and then starts incremental training. After training, the new model is used to detect type A parts in the day, and the accuracy is back to 99.2%, and the normal lines that were previously misjudged can now be correctly identified. The entire optimization process takes 3 minutes, which is just enough time to complete between two batches of part production, without affecting production.

[0056] Existing public documents, sheet metal detection model optimization method, need to stop for full training, one training takes 2 hours, affecting production; Another public document, adaptive sheet metal detection model, does not use misjudgment samples for training, and the optimization effect is not good, the accuracy can only be back to 96%. The S6 of the system uses "non-stop incremental training + misjudgment sample targeted optimization", through the weighted loss and parameter update formula, the training time is shortened from 2 hours to 3 minutes, the accuracy is back to 99.2%, and 1200 hours of manual maintenance work hours can be reduced per year, and the staff previously spent 100 hours per month to manually calibrate the model.

[0057] From the above, the existing public documents and shortcomings are: a low-temperature sheet metal detection device: a sheet metal visual detection technology in a low-temperature environment is disclosed, including a low-temperature camera and a single visual module, but without a multi-model fingerprint library, without a temperature dynamic compensation formula, without a spectrum-vision fusion algorithm; a multi-spectral sheet metal detection system: a multi-spectral material identification technology is disclosed, but the hardware is not adapted to low temperature, without a double-fingerprint similarity formula, without a time sequence LSTM model; a sheet metal defect early warning device: a defect early warning technology based on time sequence data is disclosed, but without a temperature correlation formula, without a model incremental training loss function, and the early warning accuracy is low; a sheet metal detection model optimization method: the optimization technology of the detection model is disclosed, but it needs to be stopped for full training, without a weighted loss formula, and without targeted misjudgment sample optimization.

[0058] The core difference of the technical solution is: integrating "low-temperature adaptive camera + hyperspectral module + multi-model fingerprint library", and specifying the selection standard of hyperspectral wavelength coverage feature peak ± 10nm, solving the problem of "low-temperature incompatibility and multi-model incompatibility", the scene adaptation rate is greatly improved, the hardware runs continuously for 24 hours without failure; through the gain and threshold adjustment formula to realize dynamic compensation, the image clarity is improved by 40%, the missed detection rate is reduced from 28% to 3%; the matching degree is calculated by the double-fingerprint similarity formula, without the need for code, the switching time is reduced from 5 minutes to 10 seconds, the matching accuracy is 98.5%; through the spectral angle matching and pixel superposition formula to fuse the spectral and visual information, the misjudgment rate is reduced from 22% to 0.5%, the positioning accuracy is ± 0.01mm; the trend is predicted by the LSTM core formula, early warning for 15 pieces, the continuous scrap rate is reduced from 12% to 0.7%; the incremental optimization is realized by the weighted loss formula, the training time is 3 minutes, the accuracy is back to 99.2%, and the maintenance man-hours are reduced by 1200 hours / year. In summary, the system solves the industry pain points of low-temperature multi-model precision sheet metal detection through the coordinated cooperation of the six steps, and has significant practical value.

Claims

1. A precision sheet metal part vision inspection system, characterized by: Comprise the following steps: S1 build hardware architecture: including low temperature adaptation industrial camera, hyperspectral imaging module, temperature sensor, edge computing node, mold parameter interaction module, and feature storage server, the feature storage server pre-stores a multi-model feature fingerprint library, and the fingerprint data of each type in the multi-model feature fingerprint library comprises a preset material spectral feature wavelength interval and a preset shape topological feature parameter; S2 execute temperature compensation preprocessing: collect the detection environment temperature in real time through the temperature sensor, dynamically adjust the image blue channel gain and histogram equalization threshold according to the difference between the collected temperature and the preset reference temperature, the lower the temperature, the greater the gain increase and the greater the threshold decrease, and the output clear image after Gaussian filtering processing; S3 execute multi-model automatic selection: based on the clear image output by S2, extract the shape topological fingerprint of the detected sheet metal part, including curvature, rib spacing, hole arrangement rule, based on the spectral data collected by the hyperspectral imaging module, filter the feature peak through the preset material spectral feature wavelength interval, extract the material spectral fingerprint of the detected sheet metal part, input the two fingerprints into the multi-model feature fingerprint library for similarity calculation, and automatically load the corresponding model after successful matching; S4 execute spectral-vision fusion defect recognition: load the exclusive detection model determined in S3, generate a material boundary mask through the spectral data collected by the hyperspectral imaging module, superimpose the material boundary mask and the clear image output by S2, and input the exclusive detection model to identify the defect type, position and size and output the defect annotation map; S5 execute timing defect trend prediction and early warning: store the temperature data obtained by S2, the defect data output by S4 and the model information determined by S3 in the timing database through the edge computing node, predict the defect trend based on the timing database, output the mold parameter adjustment instruction when the tolerance is exceeded, and mark the misjudgment sample and store it in the model optimization sample library.

2. The precision sheet metal part vision inspection system of claim 1, wherein: Also include S6 model iteration optimization: through the edge computing node, the defect recognition accuracy of each type of exclusive detection model is calculated in real time, when the recognition accuracy of a certain type of exclusive detection model is lower than the preset accuracy threshold for a continuous preset number of parts, the misjudgment sample and the corresponding data of the type are extracted from the model optimization sample library, the exclusive detection model backbone network is frozen, and only the incremental training strategy of the model classification head is fine-tuned, the extracted data is input into the model according to the preset batch for training, and after the training is completed, the new model is updated as the exclusive detection model of the type, and is synchronized to the spectral-vision fusion defect recognition step of S4.

3. The precision sheet metal part vision inspection system of claim 1, wherein: In S1, the adaptation temperature range of the low temperature adaptation industrial camera covers minus thirty degrees Celsius to minus five degrees Celsius, the wavelength range of the hyperspectral imaging module covers four hundred nanometers to one thousand nanometers, and the measurement accuracy of the temperature sensor is plus or minus zero point two degrees Celsius.

4. The precision sheet metal part vision inspection system of claim 1, wherein: In the S3, the extraction step of the shape topology fingerprint comprises: S31, edge detection is performed on the clear image output by the S2 to determine the contour boundary of the sheet metal part to be detected; S32, the curvature value, the rib spacing value and the row and column spacing value of the hole arrangement of the part are calculated based on the contour boundary; S33, the calculated values are compared with the preset shape topology characteristic parameters of the corresponding model in the multi-model characteristic fingerprint library to form the shape topology fingerprint; the extraction step of the material spectrum fingerprint comprises: S34, the spectral data collected by the hyperspectral imaging module are denoised; S35, the spectral peak values are screened in the preset material spectrum characteristic wavelength interval; S36, the screened spectral peak values are compared with the preset material spectrum characteristic wavelength of the corresponding model in the multi-model characteristic fingerprint library to form the material spectrum fingerprint.

5. The precision sheet metal part vision inspection system of claim 1, wherein: In the S4, the specific steps of generating the material boundary mask are: S41, the characteristic wavelengths corresponding to each material of the sheet metal part to be detected are determined based on the preset material spectrum characteristic wavelength interval; S42, the spectral data collected by the hyperspectral imaging module are divided into single-band images corresponding to each material; S43, the single-band images of each material are binarized, different material regions are assigned colors according to a preset color coding rule, and the material boundary mask is generated; the step of superimposing the material boundary mask and the clear image is: S44, the material boundary mask and the clear image are coordinate calibrated to ensure that the pixel positions of the two are one-to-one corresponding; S45, the pixel superimposition algorithm is used to cover the material boundary mask to the corresponding position of the clear image.

6. The precision sheet metal part vision inspection system of claim 1, wherein: In the S5, the construction rule of the time series database is to store the temperature data, defect data and model information at intervals of detecting a preset number of sheet metal parts to be detected once, the training sample of the temperature-defect correlation model is the historical temperature data and the corresponding defect data stored in the time series database, and the number of training samples is not less than a preset sample number threshold, and the mold parameter adjustment instruction includes a stamping force adjustment value, a laser power adjustment value and an adjustment execution time.

7. The precision sheet metal part vision inspection system of claim 2, wherein: In the S6, the method for calculating the defect recognition accuracy rate is: the defect labeling image output by the S4 is compared with the artificial review result, the ratio of the number of consistent defects to the total number of defects is calculated, the preset number of sheet metal parts to be detected is fifty, the preset accuracy threshold is ninety-five percent, the preset batch of incremental training is ten batches and the number of samples in each batch is thirty.

8. The precision sheet metal part vision inspection system of claim 4, wherein: In the S31, the edge detection adopts the Canny algorithm, the preset matching threshold in the S3 is ninety-eight point five percent, the denoising processing in the S34 adopts the wavelet transform denoising algorithm, and the spectral peak values in the preset material spectrum characteristic wavelength interval in the S35 are extracted by using the threshold segmentation method and the threshold is fifty percent of the maximum value of the spectral data.

Images