Image recognition-based intelligent identification and analysis method and system for packaging surface defects

Abstract

This invention provides an intelligent identification and analysis method and system for package surface defects based on image recognition, belonging to the field of image recognition technology. This invention utilizes multi-band combined illumination to achieve maximum contrast and detectability for various types of defects in at least one band, highlighting bands more sensitive to target defect types and improving the overall targeting of detection. By performing thermal expansion correction on sub-region reference points and other points respectively, it accurately simulates and compensates for local dimensional and morphological shifts caused by temperature changes, ensuring that the three-dimensional point cloud data truly reflects the structural state of the package under standard operating conditions, thereby improving the accuracy and reliability of subsequent defect detection. Dynamic compensation based on thermal expansion correction can effectively eliminate the interference of mechanical vibration, ensuring the stability and authenticity of the point cloud data, and improving the reliability and repeatability of defect detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method and system for intelligent identification and analysis of surface defects in packaging based on image recognition. Background Technology

[0002] Package inspection, a crucial step in semiconductor manufacturing, is essential for ensuring chip performance and reliability. Traditional package defect detection methods primarily rely on manual visual inspection and simple image processing techniques, which are inefficient and susceptible to subjective factors, failing to meet the demands of modern high-volume, high-precision manufacturing. With the development of machine vision and artificial intelligence technologies, image recognition-based automated defect detection is gradually becoming mainstream. By combining high-resolution two-dimensional images with deep learning networks, such as convolutional neural networks, it is possible to effectively extract package surface features and accurately identify various defects such as cracks, bubbles, and scratches.

[0003] During image recognition, different defect types, such as cracks, bubbles, and foreign objects, respond differently to reflection, transmission, or scattering from different wavelengths of light. Some defects are clearly visible in a certain wavelength band but difficult to distinguish in other bands. A single wavelength illumination method cannot comprehensively reflect the characteristics of all types of defects, easily leading to missed or false detections. During the inspection process, the package undergoes different temperature changes. The thermal expansion coefficients of different materials vary, causing non-uniform thermal expansion or contraction deformation in different areas of the package surface. If this local difference is ignored, systematic errors can easily occur, distorting the point cloud data and affecting the accurate location and identification of defects. At the same time, mechanical vibration is unavoidable during the inspection process. These vibrations cause instantaneous positional shifts in the point cloud data on the package surface, affecting the accuracy of 3D reconstruction and defect location. If the displacement caused by vibration is not compensated, the point cloud data will contain unstructured noise and errors, affecting the discrimination effect of subsequent image recognition algorithms.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent identification and analysis method and system for packaging surface defects based on image recognition, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The image recognition-based intelligent identification and analysis method for packaging surface defects includes the following steps: S1. Set up three sets of excitation light sources at the packaging and testing station, and obtain the maximum weighted total contrast based on the genetic algorithm; S2. Obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image, and generate an enhanced image. The enhanced image is used to reflect the effect of the original image after multi-scale Retinex processing on the corresponding pixel points. S3. Divide the package into N sub-regions and generate corresponding sub-region reference points. Use multi-point temperature sensors to collect package temperature data. Perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate package thermal expansion correction coordinates. The package thermal expansion correction coordinates are used to reflect the package point coordinates under the corrected standard operating conditions. S4. Install a triaxial vibration sensor at the testing station to collect the real-time acceleration signal of the package body, and obtain the vibration displacement of the package body through the real-time acceleration signal. Superimpose vibration displacement compensation on the thermal expansion correction coordinates to generate correction point cloud coordinates. S5. Extract features from the enhanced image and the corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence score through the defect classifier. S6. Based on the comprehensive defect confidence level, perform correlation analysis to generate defect level scores and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

[0007] Further, in S1, the total contrast is the weighted image contrast of the package to be tested under the illumination of three sets of excitation light sources. The total contrast is used to reflect the difference in grayscale between the defective area of ​​the package and the surrounding area. The three sets of excitation light sources include ultraviolet light, visible light, and near-infrared light. During detection, images of the package to be tested under the illumination of the three sets of excitation light sources are acquired at the initial light source intensity, and the image contrast and weighted total contrast are obtained. The light source intensity and light source weight are adjusted based on a genetic algorithm to maximize the weighted total contrast.

[0008] Furthermore, in S2, the main band image is obtained based on maximizing the weighted total contrast. The main band image is the image with the highest contrast under the three sets of excitation light sources. The main band image is used to reflect the single band image that is most sensitive to defect information, has the highest contrast, and best highlights visual performance and information features among the image sequences collected under ultraviolet light, visible light, and near-infrared light excitation light sources.

[0009] Furthermore, in S3, each sub-region is set with a corresponding thermal expansion coefficient. The sub-region reference point is used to reflect the thermal expansion anchor point of each sub-region. First, the sub-region reference point is thermally expanded to ensure the authenticity of the reference point position. Then, the thermal expansion deformation of the point relative to the reference point is calculated to generate the thermal expansion correction coordinates of the package.

[0010] Furthermore, in S4, the real-time acquired acceleration signal is denoised and filtered to remove high-frequency noise, and the acceleration signal is processed based on the integral method to generate vibration displacement. The corrected point cloud coordinates are used to reflect the coordinate positions of the package image without vibration and without the influence of thermal expansion.

[0011] Furthermore, in S5, a deep convolutional neural network is used to extract features from the enhanced image. The corrected point cloud coordinates are input into the graph convolutional network to obtain point cloud feature vectors. The weights are adaptively adjusted based on the attention mechanism. After passing through a defect classifier, a comprehensive defect confidence score is output. The comprehensive defect confidence score is used to reflect the maximum probability of the defect corresponding to the package.

[0012] Furthermore, a correlation analysis is performed based on the comprehensive defect confidence level to obtain the Pearson correlation coefficient matrix between defects in the package. The maximum correlation and average correlation are obtained based on the correlation coefficient matrix analysis, and a comprehensive correlation feature vector is generated based on the maximum correlation and average correlation.

[0013] Furthermore, a defect set is constructed, the confidence scores of each defect are normalized, weights are assigned according to the importance of the defect type, a weighted comprehensive confidence score is calculated, the weighted comprehensive confidence score and the comprehensive correlation feature vector are analyzed based on the Mamdani fuzzy inference model, a defect level score is generated, and the defect level is divided into three levels: excellent, good, and poor.

[0014] Furthermore, a defect level of "poor" indicates a serious defect and poor packaging quality; a defect level of "good" indicates the presence of a defect and the need for manual quality inspection; and a defect level of "excellent" indicates that the package is in good condition.

[0015] This invention also provides an image recognition-based intelligent identification and analysis system for packaged surface defects, used to execute an image recognition-based intelligent identification and analysis method for packaged surface defects, including: The light source excitation module is used to set up three sets of excitation light sources at the packaging and testing station, and obtain the maximum weighted total contrast based on a genetic algorithm; The enhancement module is used to obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image, and generate an enhanced image. The thermal expansion correction module is used to divide the package into N sub-regions, generate corresponding sub-region reference points, collect package temperature data using multi-point temperature sensors, and perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate thermal expansion correction coordinates for the package. The vibration correction module is used to install a triaxial vibration sensor at the inspection station to collect the real-time acceleration signal of the package and obtain the vibration displacement of the package through the real-time acceleration signal. Vibration displacement compensation is superimposed on the thermal expansion correction coordinates to generate correction point cloud coordinates. The confidence analysis module is used to extract features from the enhanced image and the corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence after passing through the defect classifier; The output module is used to perform correlation analysis based on the comprehensive defect confidence level, generate defect level scores, and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention utilizes multi-band combined illumination to maximize the contrast and detectability of various defect types in at least one band, highlighting bands more sensitive to target defect types and improving the overall targeting of detection. By performing thermal expansion correction on sub-region reference points and other points separately, it accurately simulates and compensates for local dimensional and morphological shifts caused by temperature changes, ensuring that the 3D point cloud data truly reflects the structural state of the package under standard operating conditions, thereby improving the accuracy and reliability of subsequent defect detection. By acquiring acceleration signals and vibration displacement in real time, and performing dynamic compensation based on thermal expansion correction, it can effectively eliminate the interference of mechanical vibration, ensuring the stability and authenticity of the point cloud data, improving the reliability and repeatability of defect detection. After obtaining accurate feature data, it is input into the defect classifier, which can then output more accurate package state determination results. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 This is a block diagram of the overall system of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] Example: Please refer to Figure 1 The present invention provides a technical solution: The intelligent identification and analysis method for surface defects in packaging based on image recognition mainly addresses the following issues: During image recognition, different defect types, such as cracks, bubbles, and foreign objects, exhibit different reflection, transmission, or scattering responses to different wavelengths of light. Some defects are clearly visible in a certain wavelength band but difficult to distinguish in other bands. A single wavelength illumination method cannot comprehensively reflect the characteristics of all types of defects, easily leading to missed or false detections. During the inspection process, the package undergoes different temperature changes. The thermal expansion coefficients of different materials vary, causing non-uniform thermal expansion or contraction deformation in different areas of the package surface. Ignoring these local differences can easily introduce systematic errors, distorting the point cloud data and affecting the accurate location and identification of defects. Simultaneously, mechanical vibration is unavoidable during inspection. This vibration causes instantaneous positional shifts in the point cloud data on the package surface, affecting the accuracy of 3D reconstruction and defect location. If the displacement caused by vibration is not compensated, the point cloud data will contain unstructured noise and errors, affecting the discrimination effect of subsequent image recognition algorithms. Specific steps include: Step 1: Set up three sets of excitation light sources on the packaging and testing station. The three sets of excitation light sources are ultraviolet light, visible light and near-infrared light. The maximum weighted total contrast is obtained based on the genetic algorithm. Multispectral light sources cover multiple wavelengths. Different defect types exhibit varying responses to reflection, transmission, and scattering in specific wavelength bands. Adjusting the light source enhances defect contrast and detectability. The total contrast is a weighted image contrast of the package under test illuminated by three sets of excitation light sources. The total contrast reflects the difference in grayscale between the defective area and the surrounding area of ​​the package. During detection, images of the package under test are acquired under three sets of excitation light sources at the initial light intensity, and the image contrast is obtained. The initial light source band and intensity are selected based on sample testing, initially choosing bands and intensities more sensitive to typical defects. The formula used for image contrast is: ; in, To correspond to the image contrast under the excitation light source, Let be the grayscale value of the i-th pixel in the image corresponding to the excitation light source. , respectively, are the mean and standard deviation of the image pixels under the corresponding excitation light source, and M is the total number of image pixels.

[0021] The weighted total contrast is obtained by processing the image contrast. The formula used is: ; in, The image contrast of the packaged object under ultraviolet light, visible light, and near-infrared light excitation sources are respectively. These represent the band weights for excitation sources of ultraviolet, visible, and near-infrared light, respectively. The band weights are set based on historical data, and each band weight represents the contribution of the image in that band to the appearance of defects. A genetic algorithm is used to obtain the maximum weighted total contrast. Specifically, a real-number encoding is used to represent the light source intensity vector, with a chromosome length of 3, corresponding to the light intensities of three sets of excitation light sources. An initial population is generated, and each individual randomly selects a value within the allowable range of the device's light intensity. A light source intensity is set for each individual, and images under the three sets of excitation light sources are acquired. The image contrast is calculated, and a weighted total contrast is obtained. Individuals with higher fitness are selected for the next step using methods such as roulette wheel selection and ranking selection. A new population is generated through crossover and mutation, and the process is iterated until the fitness improvement is less than a set threshold. The weighted total contrast at this point is then output. And record the images under each excitation light source at this time.

[0022] To obtain the maximum weighted total contrast, the excitation light source needs to be adjusted for dynamic optimization. The purpose is to improve the overall multispectral image's representation of defect features, making the acquired image most sensitive to defects and with the most prominent features. Through multi-band combined illumination, various types of defects can obtain maximum contrast and detectability in at least one band, highlighting the bands that are more sensitive to the target defect type, thereby improving the overall detection's targeting and flexibility.

[0023] Step 2: Obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image to generate an enhanced image, which is used to improve the contrast between the defect area and the background. When the weighted total contrast is maximized, the image with the highest contrast under the three excitation light sources is selected as the main band image. The main band image refers to a single band image selected from the image sequence acquired under ultraviolet, visible, and near-infrared excitation light sources. This single band image is the image that is most sensitive to defect information, has the highest contrast, and best highlights visual performance and information characteristics.

[0024] Retinex enhancement was applied to the main band image, with a scale of 3, to obtain the enhanced image.

[0025] Step 3: Divide the package into N sub-regions and generate corresponding sub-region reference points. Use multi-point temperature sensors to collect package temperature data. Perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate package thermal expansion correction coordinates. Package thermal expansion correction coordinates are used to reflect the package point coordinates under the corresponding standard working conditions after correction. Because different regions in the packaging process use different materials and have different coefficients of thermal expansion, and because the reference point is usually considered a fixed reference point during thermal expansion or contraction, the coordinates of the reference point will actually expand and deform with the increase in temperature. If the reference point is simply regarded as fixed, the actual deformation will be incorrectly segmented, causing the overall position of the point cloud to shift and the correction result to have systematic errors. Therefore, a corresponding coefficient of thermal expansion is set for each sub-region. The reference point of the expansion process in the sub-region is selected based on historical data, that is, the expansion anchor point. The coordinates of the reference point will also shift with the increase in temperature. First, the reference point of the sub-region is thermally expanded to ensure the authenticity of the reference point position. Then, the thermal expansion deformation of other points relative to the reference point is calculated, thereby further obtaining the coordinates of other sub-region points and generating more accurate thermal expansion correction coordinates of the package.

[0026] Set the original coordinates of the reference point of the j-th sub-region as follows: Temperature changes in the corresponding sub-region Obtain the coordinates of the reference point after thermal expansion. The reference temperature for temperature changes is 25 degrees Celsius, and the formula used is: ,in, Given the coefficient of thermal expansion for the corresponding sub-region, this step is used to calculate the position of the reference point in sub-region j after thermal expansion under temperature changes; for the coordinates of point l in sub-region j... Obtain its relative to the reference point The vector is based on the following formula: This step is used to calculate the vector of point l relative to the reference point within sub-region j, and to calculate the vector of point l relative to the original reference point. dilation vector This provides a basis for subsequent thermal expansion transformation; thermal expansion transformation is performed on points in the upper sub-region adjacent to the j-th sub-region to generate the transformed vector. The formula used is: This is used to simulate the local linear expansion of points in other sub-regions of a point cloud due to temperature changes. It corrects the distance and position of points relative to a reference point after thermal expansion, ensuring a realistic reflection of the point cloud structure and accurate correction of local geometry. Combined with the reference point coordinates... and the transformed vector The coordinates of the correction points in all sub-regions are stitched together, and the thermal expansion correction coordinates of the entire package are generated based on a spatial interpolation smoothing algorithm.

[0027] By performing thermal expansion correction on the reference points and other points in the sub-regions, the local size and shape shifts caused by temperature changes can be accurately simulated and compensated, ensuring that the three-dimensional point cloud data truly reflects the structural state of the package under standard operating conditions, thereby improving the accuracy and reliability of subsequent defect detection.

[0028] Step 4: Install a triaxial vibration sensor at the testing station to collect the real-time acceleration signal of the package and obtain the vibration displacement of the package through the real-time acceleration signal. Superimpose vibration displacement compensation on the thermal expansion correction coordinates to generate correction point cloud coordinates. Acceleration signals are acquired in real time during the inspection of the packaged object at the inspection station. Acceleration signals in three directions are collected and represented as follows: ,in, The acceleration signals in the X, Y, and Z directions vary with time t, respectively. A bandpass filter is used to remove low-frequency drift below 5Hz and high-frequency noise above 200Hz. A digital filter is then used to process each acceleration signal separately to obtain the filtered acceleration signal. The vibration displacement is generated by processing the acceleration signals using an integral method, based on the following formula: ,in, The velocity at the initial moment of vibration, The velocity signal of the package is used as the basis for high-pass filtering and double integration to eliminate integral drift, thereby generating the vibration displacement of the package at time t. ,in, The initial displacement is used, and a second high-pass filter is applied to eliminate the cumulative integral error.

[0029] Based on the thermal expansion correction coordinates, vibration displacement is superimposed on the thermal expansion correction coordinates as an inverse compensation amount to generate correction point cloud coordinates. After comprehensive compensation for thermal expansion and vibration, the correction point cloud coordinates reflect the actual three-dimensional shape of the package under conditions of no temperature change and no mechanical vibration. It can accurately show the true size and shape of the package surface and structure. Through the correction point cloud data, false deformation or positional shift caused by environmental factors can be effectively eliminated, ensuring that the subsequent image recognition defect detection algorithm can make judgments based on real geometric information, thereby improving the accuracy and reliability of detection.

[0030] Step 5: Extract features from the enhanced image and corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence score through the defect classifier; A deep convolutional neural network is used to process the enhanced image. Image features are extracted through the ResNet-50 network structure. After forward propagation through ResNet-50, the image feature map is obtained. Global average pooling is used to convert the feature map into feature vectors. ResNet-50 is selected as the feature extraction network because its residual connection structure can effectively alleviate gradient vanishing, making it suitable for extracting complex texture features. Moreover, the model is lightweight and easy to train.

[0031] The corrected point cloud coordinates are input into a graph convolutional network. The EdgeConv operator is selected, and after multiple layers of EdgeConv stacking, global max pooling is used to obtain the point cloud feature vector. Based on an attention mechanism, the point cloud feature vector and the enhanced image feature vector are weighted and fused. During neural network training, a fully connected layer and activation function are used, with the neural network training model automatically learning to assign weights. A multilayer perceptron classifier, i.e., a defect classifier, consisting of several fully connected layers, ReLU activation, and Dropout, is employed. The input is the fused features, and the output is a comprehensive defect confidence score, which reflects the maximum probability of a defect corresponding to the encapsulation. The multilayer perceptron classifier has been trained using historical sample data.

[0032] Step 6: Perform correlation analysis based on the comprehensive defect confidence level to generate a defect level score, and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

[0033] Correlation analysis is performed based on the comprehensive defect confidence level to obtain the Pearson correlation coefficient matrix among package defects. The maximum and average correlation coefficients are then analyzed based on this matrix, and a comprehensive correlation feature vector is generated from these coefficients. By calculating the Pearson correlation coefficient matrix of the confidence levels for multiple defect categories within the package, the linear correlation strength between different defect categories is quantified. This reflects more complex defect patterns and combinations, aiding in the assessment of package quality. Extracting the maximum and average correlation coefficients from the correlation matrix yields a feature vector containing characteristic information, which serves as input for subsequent quality level evaluation. This supplements the limitations of individual defect confidence levels, making defect level assessment more comprehensive and consistent with reality.

[0034] A defect set is constructed, and the confidence scores of each defect are normalized. Weights are assigned based on the importance of the defect type, obtained from historical data and expert scores. A weighted composite confidence score is calculated to reflect the overall severity of defects in the current package. The weighted composite confidence score and the composite correlation feature vector are analyzed using the Mamdani fuzzy inference model to generate a defect level score. Defect levels are then classified into three grades: Excellent, Good, and Poor, based on the following formula: ; in, The threshold value is determined by combining historical data and expert experience, and S represents the defect level score. A defect level of "poor" indicates a serious defect and poor packaging quality; a defect level of "good" indicates the presence of a defect and the need for manual quality inspection; and a defect level of "excellent" indicates that the package is in good condition.

[0035] The standard operating condition of this invention is the baseline environmental condition at room temperature of 25 degrees Celsius.

[0036] See Figure 2 The present invention also provides an intelligent identification and analysis system for package surface defects based on image recognition, used to execute an intelligent identification and analysis method for package surface defects based on image recognition, including: The light source excitation module is used to set up three sets of excitation light sources at the packaging and testing station, and obtain the maximum weighted total contrast based on a genetic algorithm; The enhancement module is used to obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image, and generate an enhanced image. The thermal expansion correction module is used to divide the package into N sub-regions, generate corresponding sub-region reference points, collect package temperature data using multi-point temperature sensors, and perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate thermal expansion correction coordinates for the package. The vibration correction module is used to install a triaxial vibration sensor at the inspection station to collect the real-time acceleration signal of the package and obtain the vibration displacement of the package through the real-time acceleration signal. Vibration displacement compensation is superimposed on the thermal expansion correction coordinates to generate correction point cloud coordinates. The confidence analysis module is used to extract features from the enhanced image and the corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence after passing through the defect classifier; The output module is used to perform correlation analysis based on the comprehensive defect confidence level, generate defect level scores, and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

[0037] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0038] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0039] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0040] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent identification and analysis of packaging surface defects based on image recognition, characterized in that, The specific steps include: S1. Set up three sets of excitation light sources at the packaging and testing station and collect images of the packaged body respectively. Obtain the maximum weighted total contrast based on the genetic algorithm. S2. Obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image, and generate an enhanced image. The enhanced image is used to reflect the effect of the original image after multi-scale Retinex processing on the corresponding pixel points. S3. Divide the package into N sub-regions and generate corresponding sub-region reference points. Use multi-point temperature sensors to collect package temperature data. Perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate package thermal expansion correction coordinates. The package thermal expansion correction coordinates are used to reflect the package point coordinates under the corrected standard operating conditions. S4. Install a triaxial vibration sensor at the testing station to collect the real-time acceleration signal of the package body, and obtain the vibration displacement of the package body through the real-time acceleration signal. Superimpose vibration displacement compensation on the thermal expansion correction coordinates to generate correction point cloud coordinates. S5. Extract features from the enhanced image and the corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence score through the defect classifier. S6. Based on the comprehensive defect confidence level, perform correlation analysis to generate defect level scores and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

2. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: In step S1, the total contrast is the weighted image contrast of the package to be tested under the illumination of three sets of excitation light sources. The total contrast is used to reflect the difference in image grayscale between the defective area of ​​the package and the surrounding area. The three sets of excitation light sources include ultraviolet light, visible light, and near-infrared light. During detection, images of the package to be tested under the illumination of the three sets of excitation light sources are acquired under the initial light source intensity, and the image contrast and weighted total contrast are obtained. The light source intensity and light source weight are adjusted based on a genetic algorithm to maximize the weighted total contrast.

3. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: In step S2, the main band image is obtained based on maximizing the weighted total contrast. The main band image is the image with the highest contrast under the three sets of excitation light sources. The main band image is used to reflect the single band image that is most sensitive to defect information, has the highest contrast, and best highlights visual performance and information features among the image sequences collected under ultraviolet light, visible light, and near-infrared light excitation light sources.

4. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: In S3, each sub-region is set with a corresponding thermal expansion coefficient. The sub-region reference point is used to reflect the thermal expansion anchor point of each sub-region. First, the sub-region reference point is thermally expanded to ensure the authenticity of the reference point position. Then, the thermal expansion deformation of the point relative to the reference point is calculated to generate the thermal expansion correction coordinates of the package.

5. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: In step S4, the real-time acquired acceleration signal is denoised and filtered to remove high-frequency noise. The acceleration signal is then processed based on the integral method to generate vibration displacement. The corrected point cloud coordinates are used to reflect the coordinate positions of the package image without vibration and without the influence of thermal expansion.

6. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: In step S5, a deep convolutional neural network is used to extract features from the enhanced image. The corrected point cloud coordinates are input into the graph convolutional network to obtain the point cloud feature vector. The weights are adaptively adjusted based on the attention mechanism. After passing through the defect classifier, a comprehensive defect confidence score is output. The comprehensive defect confidence score is used to reflect the maximum probability of the defect corresponding to the package.

7. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 1, characterized in that: Correlation analysis is performed based on the comprehensive defect confidence level to obtain the Pearson correlation coefficient matrix between defects in the package. The maximum correlation and average correlation are obtained based on the correlation coefficient matrix analysis, and a comprehensive correlation feature vector is generated based on the maximum correlation and average correlation.

8. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 7, characterized in that: A defect set is constructed, the confidence scores of each defect are normalized, weights are assigned according to the importance of the defect type, a weighted comprehensive confidence score is calculated, the weighted comprehensive confidence score and the comprehensive correlation feature vector are analyzed based on the Mamdani fuzzy inference model, a defect level score is generated, and the defect level is divided into three levels: excellent, good, and poor.

9. The intelligent identification and analysis method for packaging surface defects based on image recognition according to claim 8, characterized in that: A defect level of "poor" indicates a serious defect and poor packaging quality; a defect level of "good" indicates the presence of a defect and the need for manual quality inspection; and a defect level of "excellent" indicates that the package is in good condition.

10. An image recognition-based intelligent identification and analysis system for packaged surface defects, used to execute the image recognition-based intelligent identification and analysis method for packaged surface defects as described in claim 1, characterized in that, include: The light source excitation module is used to set up three sets of excitation light sources at the packaging and testing station, and obtain the maximum weighted total contrast based on a genetic algorithm; The enhancement module is used to obtain the main band image based on maximizing the weighted total contrast, perform multi-scale Retinex processing on the main band image, and generate an enhanced image. The thermal expansion correction module is used to divide the package into N sub-regions, generate corresponding sub-region reference points, collect package temperature data using multi-point temperature sensors, and perform correlation analysis based on sub-region reference points, thermal expansion coefficients, and corresponding temperatures of sub-regions to generate thermal expansion correction coordinates for the package. The vibration correction module is used to install a triaxial vibration sensor at the inspection station to collect the real-time acceleration signal of the package and obtain the vibration displacement of the package through the real-time acceleration signal. Vibration displacement compensation is superimposed on the thermal expansion correction coordinates to generate correction point cloud coordinates. The confidence analysis module is used to extract features from the enhanced image and the corrected point cloud coordinates, adaptively adjust the weights based on the attention mechanism, and output the comprehensive defect confidence after passing through the defect classifier; The output module is used to perform correlation analysis based on the comprehensive defect confidence level, generate defect level scores, and classify the encapsulation defects of the package into three levels: excellent, good, and poor.

Images