An intelligent detection method for the processing of automotive decorative panels

Abstract

The present application relates to the technical field of image detection, and discloses an intelligent detection method for the processing process of automotive trim panels. The method includes: performing high-resolution image acquisition and preprocessing on the automotive trim panel to obtain standardized image data; performing region segmentation operations on the standardized image data to obtain a target region mask map of the trim panel, and extracting a feature vector set; inputting the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculating a candidate defect region mask through the defect classification model; performing multi-scale hierarchical analysis on the candidate defect region mask to obtain a defect risk heat map, and performing clamping quality analysis on the connection region between the card block and the card slot according to the defect risk heat map to obtain a clamping quality evaluation result. The present application realizes the precise positioning of the defect region, constructs a defect risk heat map to quantify the risk levels of different regions, and provides data support for the optimization of the production process.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and particularly to an intelligent detection method for the processing of automotive trim panels. Background Art

[0002] With the rapid development of the automotive industry, consumers' requirements for the quality of automotive interior parts are increasing day by day. As an important interior component, the automotive trim panel not only undertakes the aesthetic function, but also needs to ensure structural stability and use safety. The main function of the inner trim panel of the automotive trunk threshold is to cover the sheet metal of the trunk threshold and prevent passengers from being scratched by the sheet metal of the threshold when loading and unloading luggage. Its installation quality involves the correct connection between multiple components and is crucial for the overall function and service life of the trim panel. However, the existing detection of trim panels mainly relies on manual visual inspection, lacking objective, accurate, and efficient automated detection means, and it is difficult to meet the quality control requirements of mass production.

[0003] Currently, the detection of defects in the processing of automotive trim panels mainly relies on image detection technology. However, due to the numerous and diverse types of defects, the existing detection methods lack detection standards with strong universality and clear mechanisms. Traditional machine vision detection methods are mostly based on manually designed features and simple threshold judgments, and the detection accuracy for trim panels with complex structures is limited; although deep learning methods can automatically learn features, they often require a large number of labeled samples, and it is difficult to obtain and the samples are unbalanced for trim panel defect samples, making it difficult to effectively train the model. More critically, the existing detection methods are difficult to evaluate the clamping quality between trim panel components, especially the clamping state between the clamping block and the clamping groove, and the traditional disassembly test will damage the product and cannot be applied to production line detection. Summary of the Invention

[0004] This application provides an intelligent detection method for the processing of automotive trim panels. This application realizes the precise positioning of the defect area, constructs a defect risk heat map to quantify the risk levels of different areas, and provides data support for the optimization of the production process.

[0005] In a first aspect, this application provides an intelligent detection method for the processing of automotive trim panels, and the intelligent detection method for the processing of automotive trim panels includes:

[0006] Performing high-resolution image acquisition and preprocessing on the automotive trim panel to obtain standardized image data;

[0007] Performing region segmentation operation on the standardized image data to obtain a target region mask map of the trim panel, and extracting multi-modal features from the target region mask map of the trim panel to obtain a feature vector set;

[0008] Input the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculate a candidate defect region mask through the defect classification model;

[0009] Perform multi-scale hierarchical analysis on the candidate defect region mask to obtain a defect risk heat map, and perform latching quality analysis on the connection region between the card block and the card slot according to the defect risk heat map to obtain a latching quality evaluation result.

[0010] In the technical solution provided by this application, through the multi-modal feature extraction and fusion technology, multi-dimensional information on the surface of the decorative board is effectively captured, comprehensively reflecting the feature manifestations of different types of defects; the transfer learning strategy based on ResNet-v1-50 solves the problem of insufficient decorative board defect samples, significantly improving the classification accuracy, especially at the connection between the arc-shaped board and the placement board; the gradient-weighted class activation mapping technology combines multi-level feature fusion to achieve precise positioning of the defect region; the multi-scale hierarchical analysis framework analyzes step by step from the large image to the small image, gradually improving the detection resolution as the analysis scale is refined, making defect detection more accurate; innovatively, the latching state between decorative board components is indirectly evaluated based on surface morphology features, and the latching quality can be evaluated without disassembly, providing a feasible solution for on-line detection in the production line; by constructing a defect risk heat map to quantify the risk levels of different regions, data support is provided for the optimization of the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic diagram of an embodiment of the intelligent detection method for the processing of automotive decorative boards in the embodiments of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] The embodiment of the present application provides an intelligent detection method for the processing of automotive trim panels. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0014] For ease of understanding, the specific process of the embodiment of the present application will be described below. Please refer to Figure 1 , an embodiment of the intelligent detection method for the processing of automotive trim panels in the embodiment of the present application includes:

[0015] Step S101, perform high-resolution image acquisition and preprocessing on the automotive trim panel to obtain standardized image data;

[0016] It can be understood that the execution subject of the present application can be an intelligent detection system for the processing of automotive trim panels, or a terminal or a server. Specifically, it is not limited here. The embodiment of the present application takes the server as the execution subject as an example for illustration.

[0017] Specifically, a dedicated image acquisition system is built, which consists of a high-resolution industrial camera, an LED light source array with a specific wavelength, and an accurate positioning device. The industrial camera selects a high-resolution model of 1920×1080 pixels to ensure capturing the delicate texture and tiny structural features on the surface of the automotive trim panel. At the same time, to ensure uniform illumination on the surface of the trim panel, the light source system adopts a ring-shaped LED light source with a color temperature stable at 5500K±100K. This kind of light source can provide uniform and stable white light illumination, thus avoiding the influence of shadows or reflections caused by uneven light on the image quality. The accurate positioning device, through the cooperation of the conveyor belt and the automatic trigger mechanism, enables the image acquisition system to automatically capture the front view of the trim panel when the trim panel moves to the specified shooting position, realizing a fully automated image acquisition process. The original trim panel images are subjected to size standardization processing, and all the collected original images are adjusted to a unified size of 1280×720 pixels to ensure the consistency of image data and avoid feature mismatch problems caused by differences in image resolution during the model training process, obtaining standardized size images. The standardized size images are subjected to histogram equalization processing to enhance the contrast of the images. Histogram equalization is an image enhancement method that stretches the distribution of image gray values to enhance the details in the bright and dark regions of the image, making the surface texture of the trim panel more clear and prominent, obtaining contrast-enhanced images. The contrast-enhanced images are subjected to non-local means filtering. By calculating the similarity between each pixel in the image and the surrounding pixel blocks, the similar pixels are weighted and averaged, effectively removing Gaussian noise and salt-and-pepper noise in the image while maintaining the edge and detail features of the image, obtaining denoised images. The denoised images are subjected to adaptive threshold segmentation operations to generate a binary mask map of the trim panel. Adaptive threshold segmentation is a method of dynamically calculating the binary threshold for each pixel in the image. By analyzing the brightness characteristics of the local area, it can effectively cope with the situation of uneven illumination or complex background in the image and accurately separate the trim panel area from the background. Through this step, a binary mask map containing only the shape of the trim panel is obtained. Using the binary mask map, the background removal operation is performed on the denoised images to obtain pure trim panel images. During the background removal process, the non-trim panel areas in the original images are filled with black, so that only the pixel data of the trim panel body is retained in the output images. The pure trim panel images are subjected to image enhancement processing to obtain standardized image data. The image enhancement processing includes adjusting the brightness, contrast, and color saturation of the images, as well as sharpening operations based on specific filters to improve the visual effect of the images and strengthen the expression of the surface features of the trim panel.

[0018] Step S102: Perform region segmentation operations on the standardized image data to obtain a mask map of the target area of the trim panel, and extract multi-modal features from the mask map of the target area of the trim panel to obtain a feature vector set;

[0019] Specifically, the standardized image data is input into the U-Net network for feature extraction to achieve the construction of multi-scale feature maps. The U-Net network structure is designed as a symmetric structure with four layers of encoders and four layers of decoders. Among them, the encoder part uses a ResNet module with dilated convolutions to achieve multi-scale and efficient feature extraction. The decoder part gradually restores the spatial dimension of the feature map through transposed convolutions, ensuring the improvement of image segmentation accuracy while maintaining the richness of features. In the encoder, the introduction of dilated convolutions significantly expands the receptive field, enabling the network to capture detailed features and global context information in the decorative panel image. The ResNet module, through the design of residual connections, effectively avoids the problem of gradient disappearance in deep networks, enabling the model to stably extract key region features of the decorative panel in complex backgrounds. Based on the multi-scale feature maps, network training is carried out to obtain a region segmentation model. During the training process, by using a supervised learning method with a weighted boundary loss function, the model gives a higher penalty weight to the boundary region when predicting the target region of the decorative panel, improving the segmentation accuracy of the model at complex boundaries. For example, at key parts such as the connection between the arc plate and the placement plate, the installation surface of the pushing component, the periphery of the locking groove, the meshing area of the clamping block and the clamping groove, and the area where the foam adhesive adheres, the model can accurately generate a region mask map, providing a spatial positioning basis for subsequent feature extraction. After the region segmentation model is trained, the standardized image data is input into the model, and the model outputs a mask map of the target region of the decorative panel. In this mask map, different target regions are identified by different mask categories. Based on the mask map of the target region of the decorative panel, morphological features and texture features are calculated to construct a primary feature set. The extraction methods of morphological features include calculating the area, perimeter, circularity, aspect ratio, and the size of the minimum bounding rectangle of each target region. These features help to identify potential size deviations or shape anomalies in areas such as the meshing area of the clamping block and the clamping groove. At the same time, by using the local binary pattern operator to analyze the texture of the target region, statistical features such as the energy, contrast, correlation, and entropy values of the pixel gray distribution within the region are obtained. These texture features can effectively reflect whether there are defects such as scratches, material cracks, or poor welding on the surface of the decorative panel. On the basis of primary feature extraction, edge features and color features are extracted based on the mask map of the target region of the decorative panel to construct an intermediate feature set. In terms of edge feature extraction, edge detection is carried out through an improved Canny operator, and edge density, directionality, and continuity features are calculated. These features help to identify the fine structures of the installation surface of the pushing component and the periphery of the locking groove. In terms of color feature extraction, by converting the target region to the HSV color space, the mean, variance, skewness, and kurtosis of hue, saturation, and brightness within the region are calculated respectively. Color features can distinguish decorative panels made of different materials and identify phenomena such as uneven color or contamination in areas where the foam adhesive adheres. Depth features are extracted from the mask map of the target region of the decorative panel to obtain a target feature vector.Introduce the pre-trained EfficientNet-B0 network, input the image of the decorative panel mask area into the network, extract high-dimensional depth features through the convolutional neural network, and obtain a 1280-dimensional feature vector. Reduce the dimension of the high-dimensional feature vector to 128 dimensions through the principal component analysis method, so as to eliminate redundant information and retain the feature dimensions that play a key role in the identification of decorative panel defects. Perform feature selection on the primary feature set, intermediate feature set, and target feature vector to obtain a feature vector set. In the feature selection process, calculate the contribution degree of each feature to defect identification through the random forest algorithm, and retain the top 80% of the features with the highest contribution degree by analyzing the importance scores of the features, so as to eliminate low-correlation or noise features and optimize the accuracy and efficiency of the feature set.

[0020] Step S103: Input the feature vector set into the transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculate the candidate defect area mask through the defect classification model;

[0021] Specifically, a defect classification network for decorative panels based on ResNet-v1-50 is constructed. The design of this network includes a feature extraction convolutional layer, a global average pooling layer, two fully connected layers, and a Softmax classification layer. In the network structure, the feature extraction convolutional layer retains the basic architecture of ResNet-v1-50, including multiple residual modules. These modules effectively avoid the vanishing gradient problem in deep networks by introducing residual connections and extract multi-scale features in the decorative panel images through convolutional operations. The introduction of the global average pooling layer effectively replaces the traditional fully connected layer, converts the feature map into a low-dimensional feature vector, reduces the number of model parameters, and effectively avoids overfitting. In the part of the fully connected layer, it is designed as a two-layer structure. After a dimensionality reduction operation from 1024 dimensions to 256 dimensions, the feature is mapped into a binary classification output result (good product / defect) through the Softmax classification layer to ensure that the model quickly outputs the quality determination result of the decorative panel in the inference stage. The standardized image data is extracted in regions, and the original image dataset is divided into the first image dataset, the second image dataset, and the third image dataset to achieve targeted analysis of images at different scales. Among them, the first image dataset contains images of complete decorative panels, which are used to evaluate the quality status of the overall decorative panel. Such images maintain a large field of view, which helps the model capture the defect features existing in the decorative panel at the global scale. The second image dataset focuses on the local images at the connection between the arc-shaped panel and the placement panel. This area is a key part where the decorative panel is prone to deformation or installation deviation. Therefore, the images in this dataset have a small size, enabling the model to perform fine-grained detection at the detail scale. The third image dataset focuses on the pushing component area, which is an important object for the quality assessment of mechanical clamping. By providing local images with higher resolution, the model can identify subtle cracks, wear, or poor assembly on the surface of the pushing component. The feature vector set is corresponded to the first image dataset, the second image dataset, and the third image dataset respectively to obtain a complete dataset for model training. During the data preparation process, by pairing the image data with the feature vectors, multi-modal input information is provided for the model, effectively improving the accuracy and robustness of the model in the defect recognition task. During the training process, the first-stage training is performed. The parameters of the convolutional blocks in the feature extraction convolutional layer are locked, and only the parameters of the two fully connected layers are optimized. By freezing the front-end feature extraction part of the model, the model can quickly adapt to the decorative panel data distribution without spending too much time on re-learning the underlying features. In this way, the model utilizes the general features already learned on the ImageNet dataset in the pre-trained ResNet-v1-50 network, and only adjusts the final classification part to make it match the specific pattern of the decorative panel images, obtaining a classification model that is initially adapted to the decorative panel data distribution.In this stage, a relatively large learning rate (e.g., 0.001) is selected to accelerate the model convergence speed. Meanwhile, monitor the performance on the validation set during the training process to ensure that the classification accuracy of the model is improved without overfitting. Perform the second-stage training on the classification model that has been initially adapted to the data distribution of the decorative board, unlock all network layer parameters, and perform full-network fine-tuning on the entire model. Through the training of the entire network, the model not only adjusts the parameters of the classification layer but also updates the parameters of the feature extraction convolutional layer through the backpropagation mechanism, enabling the model to learn more specific visual features from the decorative board images, especially the recognition ability for the surface texture, edge details, and local morphological features of the decorative board. In this stage, a relatively small learning rate (e.g., 0.0001) is adopted to ensure that the adjustment of the model parameters is more refined and stable. Meanwhile, combine the learning rate decay strategy and regularization techniques (such as weight decay and Dropout) to prevent the model from overfitting under the conditions of small samples or imbalanced samples. During the training process, ensure that the model automatically stops training when it reaches the optimal performance by setting the training termination conditions (e.g., the validation set accuracy does not improve for five consecutive epochs or reaches the maximum number of training epochs), and obtain the optimized classification model. Based on the optimized classification model, perform inference on the test parts of the first figure dataset, the second figure dataset, and the third figure dataset, and obtain the defect classification results through model prediction. In the inference stage, the model quickly determines whether there are defects on the decorative board based on the feature vector and image data of the input image and outputs the specific classification results. To achieve the precise localization of the candidate defect regions, calculate the gradient-weighted class activation mapping based on the optimized classification model to generate a heatmap, which shows the image regions that the model focuses on when making the defect classification decision. By calculating the gradient of the score of the output class of the model with respect to the last convolutional feature map, and then weighted-fusing the gradient information with the feature map to generate a heatmap, the regions that the model considers most likely to have defects are highlighted on the image. To improve the accuracy of the candidate defect region mask, perform image processing operations on the heatmap, such as setting an adaptive threshold for binarization and removing the noise regions through morphological operations to generate the candidate defect region mask.

[0022] In this embodiment, a target score function is defined according to the defect category in the defect classification model. This target score function represents the output score of a specific defect category predicted by the model. For example, in the Softmax classification layer, this score is represented as the activation value of a specific category. The target score function is expressed as , where c represents the target defect category. By defining the target score function, it is clear which feature regions in the input image the model specifically focuses on when identifying a certain category of defects. Calculate the gradient value of this target score function with respect to the feature map of the last convolutional layer in the model to obtain the target gradient data. During the backpropagation process of the model, calculate the partial derivative of the target score function with respect to the convolutional layer feature map , and the mathematical expression is , where represents the th channel of the feature map. By calculating the influence degree of each pixel position in the feature map on the target classification score, the sensitivity of the model to the classification result at a specific position is quantified. A global average pooling operation is performed on the target gradient data in the spatial dimension to calculate the importance weights of the feature map channels. By averaging the feature map gradients in the width and height dimensions, each channel only retains one weight value, indicating the importance of the channel to the target defect category. Multiply the importance weights of the feature map channels by the corresponding feature map and sum in the channel dimension to generate an initial heat map. Based on the intermediate layer feature maps of the defect classification model, multi-scale heat maps are generated, and the initial heat map and multi-scale heat maps are weighted and fused according to preset weight coefficients to obtain an optimized heat map. In addition to the feature maps of the last convolutional layer, the feature maps of the intermediate layers in the model (such as the res3d_branch2c, res4f_branch2c, etc. layers in ResNet) are usually selected, and the Grad-CAM heat maps of these intermediate layer feature maps are calculated respectively, and different weight coefficients, such as [0.2, 0.3, 0.5], are assigned to each heat map. The weighted fusion strategy can integrate the advantages of different scale features, so that the optimized heat map can not only capture the overall features of the macroscopic decorative board, but also identify the microscopic surface details, thus improving the localization ability of the model at multiple scales, especially in complex structures such as the connection area between the arc plate and the placement plate or the meshing area between the clamping block and the clamping groove, showing stronger recognition accuracy. Perform morphological opening and closing operations on the optimized heat map to refine the candidate defect area mask. Morphological opening (using a small circular structuring element, such as a structuring element with a radius of 3) is used to eliminate the noise in the heat map, such as removing small and isolated high-brightness pixel points, to avoid the interference of these noises on the defect area detection result. And morphological closing (using a circular structuring element with a radius of 5) fills the small holes in the heat map, making the mask of the defect area more coherent and smooth, especially in places with complex boundaries such as the installation surface of the pushing component or the foam adhesive attachment area, which can significantly improve the morphological integrity of the candidate area. Calculate geometric features such as the area, perimeter, and circularity of each connected region in the morphological operation, and by setting thresholds (such as removing regions with an area less than 30 pixels or a circularity greater than 0.9), effectively filter out misdetected regions, so that the output candidate defect area mask only contains the regions that the model considers most likely to have defects.

[0023] Step S104: Perform multi-scale hierarchical analysis on the candidate defect area mask to obtain a defect risk heat map, and perform clamping quality analysis on the connection area between the clamping block and the clamping groove according to the defect risk heat map to obtain a clamping quality evaluation result.

[0024] Specifically, multi-scale hierarchical analysis is performed based on the candidate defect region mask to obtain detection results with different precisions and granularities. The multi-scale hierarchical analysis adopts a three-level progressive method. By decomposing the candidate defect region mask into the first-scale hierarchical region, the second-scale hierarchical region, and the third-scale hierarchical region, accurate identification and positioning of the defect region can be achieved in the process from the whole to the local and from coarse granularity to refinement. The first-scale hierarchical region mainly retains the image information of the overall decorative panel, has a large field of view, and provides a coarse-grained global detection perspective, so as to quickly identify significant defect features in the overall structure of the decorative panel. When performing coarse-grained defect detection on the first-scale hierarchical region, the previously constructed defect classification model is used to quickly scan the overall decorative panel image. By analyzing the highlighted regions in the candidate defect region mask and combining the classification results output by the model, a preliminary defect judgment result is generated. The defect categories existing in the image are determined through the Softmax classification output of the model, and the classification results are mapped back to the specific positions in the image to generate a preliminary heat map. According to the preliminary defect judgment result, the analysis scope is narrowed down to the second-scale hierarchical region. The analysis at this stage focuses on the local image at the connection between the arc-shaped plate and the placement plate. By inputting higher-resolution image data into the model, the model performs more accurate defect positioning operations on specific key regions at a finer scale. In this process, the Grad-CAM technology is applied to generate a refined heat map of the local region, and combined with the morphological analysis method. By calculating geometric features such as the area, perimeter, and shape complexity of the region, misdetected regions are eliminated to ensure the accuracy and reliability of the positioning results. According to the defect positioning result at the connection between the arc-shaped plate and the placement plate, defect positioning is performed on the third-scale hierarchical region, and the specific position data of the target defect is accurately calculated. The model uses a high-resolution small image dataset for inference. By analyzing smaller details in the image (such as tiny cracks or assembly errors on the surface of the pushing component), the specific position coordinates of the target defect are obtained. The automotive decorative panel is divided into multiple grids. By mapping the preliminary defect judgment result, the defect positioning result at the connection between the arc-shaped plate and the placement plate, and the target defect position data to these grids, the cumulative value of the historical defect heat map in each grid is statistically calculated to obtain the defect frequency distribution data of different regions of the decorative panel. The grid division adopts a fixed-size grid method. For example, the decorative panel image is divided into 10×10 grids, and each grid represents a specific spatial region. By statistically calculating the cumulative situation of the defect heat values in each grid, the probability distribution of defects occurring in different regions of the decorative panel is quantified. The defect frequency distribution data is standardized and the risk levels are divided to generate a defect risk heat map.By normalizing the cumulative heat value of each grid to the range of 0 to 1, and according to the preset risk level division criteria, for example, marking the heat value between 0 and 0.2 as extremely low risk, 0.2 to 0.4 as low risk, 0.4 to 0.6 as medium risk, 0.6 to 0.8 as high risk, and 0.8 to 1.0 as extremely high risk, a quantitative risk assessment result is provided, and the quality status of each area of the decorative panel is visually displayed through visual coding of different colors (for example, blue represents low risk and red represents high risk). Determine the high-risk areas based on the defect risk heat map, and conduct a clamping quality analysis on the connection area between the clamping block and the clamping groove to obtain a clamping quality assessment result. In this process, focus on the grids shown as high risk (usually the red areas) in the heat map, and evaluate whether there are problems with poor clamping by analyzing the connection status between the clamping block and the clamping groove in these areas. Specific analysis methods include calculating the texture features, edge features, and light reflection patterns on the surface of the connection area, and identifying problems such as clamping deviation, connection looseness, or assembly errors by comparing with the features of good product samples. On this basis, using a support vector machine or other classification algorithms, the clamping quality is divided into four grades: excellent, good, average, and poor, and a quantitative clamping quality assessment score is output.

[0025] In this embodiment, high-risk areas with risk values greater than a preset target value are screened out according to the defect risk heat map. A risk threshold is set, for example, 0.8. When the risk value of a certain area exceeds this threshold, the area will be marked as a high-risk area. Specifically, it includes areas such as the connection between the pushing block and the rotating groove, the edge of the connection between the arc-shaped plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area, which are prone to defects in the decorative plate structure due to mechanical movement, stress concentration, or material properties. The connection area between the clamping block and the clamping groove in the high-risk area is extracted to obtain the surface image of the connection area. The image extraction process combines the candidate defect area mask generated in the early stage and the coordinate data of the high-risk area. By locating the specific positions of the clamping block and the clamping groove in the decorative plate image, the images within these areas are cropped to ensure that the extracted images only contain the effective information on the surface of the engaging components and avoid the interference of the surrounding background. Multiple feature extraction operations are performed on the surface image of the connection area to construct a surface morphology feature set. During the feature extraction process, the texture features of the surface are calculated. For example, the gray-scale pattern distribution of surface pixels is extracted through the Local Binary Pattern (LBP) operator to obtain statistical features such as energy, contrast, correlation, and entropy values. These features can effectively reflect the material consistency and texture regularity of the surfaces of the clamping block and the clamping groove. And the surface edge features are extracted. The improved Canny operator is used to calculate the edge density, directionality, and continuity. By analyzing these features, the regularity degree of the edge lines in the engaging area is identified, and then the assembly accuracy of the clamping block and the clamping groove and the stability of the engaging state are inferred. In terms of color feature extraction, the image is converted to the HSV color space, and the mean, variance, skewness, and kurtosis of hue, saturation, and brightness are calculated respectively. These color features play an important role in identifying the light reflection characteristics of the surface of the connection area and judging whether the engaging state causes uneven surface color due to mechanical deformation. To analyze the engaging quality more comprehensively, deep learning models are used to extract the depth features of the area. These high-dimensional features can effectively capture the minute morphological changes on the surface, especially for potential quality problems in the connection area that are difficult to detect by traditional image feature extraction methods. A mapping relationship model between the engaging state and the surface morphology is constructed to associate the surface features of the connection area with the actual engaging quality state to achieve quantitative analysis of the engaging quality. When constructing the mapping relationship model, the distribution rules of the surface morphology features under different engaging states are extracted by analyzing a large number of sample data with known engaging states. For example, in the fully engaged state, the connection area between the clamping block and the clamping groove shows a regular texture distribution, the edge lines are clear and continuous, the light reflection pattern is uniform, and there are no obvious deformation marks. In the case of insufficient engagement or poor assembly, due to the influence of mechanical stress or structural looseness, there will be minute deformations on the surface of the connection area, resulting in changes in the light reflection pattern, such as the formation of irregular highlight or shadow areas. The texture features of the surface will also deviate from the normal range, such as broken edge lines, disordered texture distribution, or the appearance of fine cracks.By labeling and modeling these surface features and the engagement state, a multi-dimensional mapping relationship model is constructed, and the input surface feature set is mapped to a specific engagement quality state through model prediction. Based on the surface morphology feature set and the mapping relationship model, engagement quality analysis is carried out to obtain the engagement quality evaluation result. The surface features of the connection area in the high-risk area are input into the mapping relationship model, and the prediction result of the engagement state is obtained through model inference. Classification algorithms such as support vector machines, random forests, or deep neural networks are used to divide the engagement quality into multiple levels, such as excellent, good, average, poor, etc. During the evaluation process, a classification result of the engagement quality is output, and a quantitative score is provided, for example, representing the quality of the engagement in the form of a percentage system. In the analysis result, an excellent engagement state corresponds to a score greater than 90 points, a good state is between 80 and 90 points, an average state is 70 to 80 points, and those below 70 points are determined to be in a poor engagement state.

[0026] In the embodiment of the present application, through the multi-modal feature extraction and fusion technology, multi-dimensional information on the surface of the decorative panel is effectively captured, comprehensively reflecting the characteristic manifestations of different types of defects; the transfer learning strategy based on ResNet-v1-50 solves the problem of insufficient defective samples of the decorative panel, significantly improving the classification accuracy, especially at the connection between the arc-shaped panel and the placement panel; the gradient-weighted class activation mapping technology combines multi-level feature fusion to achieve precise positioning of the defective area; the multi-scale hierarchical analysis framework analyzes step by step from the large image to the small image, gradually improving the detection resolution as the analysis scale is refined, making the defect detection more accurate; innovatively, the engagement state between the decorative panel components is indirectly evaluated based on the surface morphology features, and the engagement quality can be evaluated without disassembly, providing a feasible solution for on-line detection in the production line; by constructing a defect risk heat map to quantify the risk levels of different regions, data support is provided for the optimization of the production process.

[0027] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0028] Use an industrial camera to collect images of the automotive decorative panel, obtain the original decorative panel image, and perform size normalization processing on the original decorative panel image to obtain a standardized size image;

[0029] Perform histogram equalization processing on the standardized size image to obtain a contrast-enhanced image, and perform non-local means filtering on the contrast-enhanced image to obtain a denoised image;

[0030] Perform adaptive threshold segmentation on the denoised image to obtain a binary mask image of the decorative panel, and use the binary mask image of the decorative panel to perform background removal on the denoised image to obtain a pure decorative panel image;

[0031] Perform image enhancement processing on the pure decorative panel image to obtain standardized image data.

[0032] Specifically, an image acquisition system is constructed to ensure that the acquired original images have sufficient resolution and quality. The image acquisition system includes a high-resolution industrial camera, an LED light source array with a specific wavelength, and an accurate positioning device. An industrial camera with 1920×1080 pixels is selected to ensure the richness of image details. In terms of light source selection, a ring-shaped LED light source with a color temperature fixed at 5500K±100K is adopted. This light source can provide a uniform and stable lighting environment, avoiding fluctuations in image quality caused by changes in the light source. Through an automatic triggering method, when the automotive trim panel reaches a preset position via the conveyor belt, the system automatically captures the front view of the trim panel, thus realizing automated image acquisition during the efficient operation of the production line. The original images are subjected to size standardization processing to obtain standardized-size images with a unified specification. All images are adjusted to a size of 1280×720 pixels. By size normalization, the image scale differences caused by different sizes of trim panels and changes in the camera distance are eliminated, improving the stability of the image processing algorithm. During the process of achieving size standardization, a bilinear interpolation algorithm is adopted. By performing weighted calculations on the pixels of the original image, an image of the target size is generated while minimizing the impact on the details and edge sharpness in the image. The standardized-size images are subjected to histogram equalization processing to obtain contrast-enhanced images. Histogram equalization is an image enhancement technique based on the pixel gray value distribution. It linearizes the cumulative distribution function of the image gray values, making the brightness distribution of the image more uniform. Mathematically, histogram equalization is achieved through the following formula:

[0033]

[0034] where, represents the enhanced pixel value, is the number of gray levels of the image, usually 256, and are the width and height of the image respectively, is the number of pixel points with a gray value of and calculates the cumulative number of pixels with pixel values not greater than . The contrast-enhanced images are subjected to non-local means filtering operations to obtain denoised images. Non-local means filtering calculates the similarity between each pixel in the image and all pixels, and performs weighted averaging on similar pixels, thereby effectively removing Gaussian noise and salt-and-pepper noise while retaining the image edges and details. The calculation formula for non-local means filtering is:

[0035]

[0036] where, represents pixel The new value after filtering, is the pixel search window, is the pixel gray value, is the pixel and the weight coefficient between them. The calculation of the weight coefficient is based on the similarity of the pixel neighborhood, and the specific definition is:

[0037]

[0038] Among them, and are the neighborhood blocks of the pixels and respectively, represents the Euclidean distance, is the filtering intensity control parameter. For the image after non-local means filtering, the noise is effectively suppressed, and the sharpness of the edge of the decorative board can be retained. Adaptive threshold segmentation is performed on the denoised image to generate a binary mask map of the decorative board. The adaptive threshold segmentation method dynamically calculates the segmentation threshold for each pixel by analyzing the local area brightness change of the image, which is suitable for the scene with uneven illumination. The image is divided into multiple small blocks, and the pixel gray histogram of each small block is calculated to determine the optimal segmentation threshold of the area. In the generated binary mask map, the white pixels represent the decorative board area, and the black pixels represent the background area. Through this mask map, the decorative board is effectively separated from the complex background. Using the binary mask map of the decorative board, the background removal operation is performed on the denoised image to obtain a pure decorative board image. During the background removal process, the pixel values of the background part in the mask map are set to zero, and only the pixel data of the decorative board part is retained, so that only the effective information of the decorative board remains in the processed image. Image enhancement processing is performed on the pure decorative board image to obtain standardized image data. In the image enhancement operation, the contrast-limited adaptive histogram equalization method is used to enhance the local contrast of the image and avoid the over-enhancement phenomenon in the traditional histogram equalization. And the edge features and brightness balance of the image are enhanced through sharpening filtering and gamma correction techniques. For example, the surface details of the decorative board are enhanced through sharpening filtering, making small scratches or defects more obvious, while gamma correction adjusts the non-linear mapping of pixel values so that the image can maintain good visibility under different brightness conditions.

[0039] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0040] The standardized image data is input into the U-Net network for feature extraction to obtain a multi-scale feature map. The U-Net network contains four layers of encoders and four layers of decoders. Each layer of encoder uses a ResNet module with hole convolution to extract features, and each layer of decoder restores the spatial dimension of the feature map through transposed convolution.

[0041] Perform network training based on multi-scale feature maps to obtain a region segmentation model;

[0042] The standardized image data is segmented using the region segmentation model to obtain a mask map of the target region of the decorative plate, wherein the target region of the decorative plate includes the connection between the curved plate and the placement plate, the installation surface of the push component, the periphery of the lock slot, the meshing area between the card block and the card slot, and the foam adhesive attachment area;

[0043] The morphological features and texture features are calculated based on the mask image of the target area of ​​the decorative plate to obtain a primary feature set, and the edge features and color features are extracted based on the mask image of the target area of ​​the decorative plate to obtain a mid-level feature set;

[0044] Deep features are extracted from the mask image of the target area of ​​the decorative plate to obtain a target feature vector, and feature selection is performed on the primary feature set, the intermediate feature set and the target feature vector to obtain a feature vector set.

[0045] Specifically, an improved U-Net network architecture is constructed, which consists of a four-layer encoder and a four-layer decoder. Each encoder layer uses a ResNet module with dilated convolution to enhance feature extraction capabilities, while the decoder part gradually restores the spatial dimensions of the feature map through transposed convolution, thereby retaining the spatial details of the image while maintaining high semantic information. The introduction of dilated convolution, by inserting holes in the convolution kernel, allows the receptive field to be exponentially expanded without increasing the amount of computation, which is especially important in segmentation models because it can capture the multi-scale features of complex structures such as curved panels, push components, and card slots in automotive decorative panels. In the feature extraction process of the U-Net network, the standardized image data is processed by a four-layer encoder, and each encoder layer consists of a ResNet module with dilated convolution, which alleviates the gradient vanishing problem in deep networks through residual connections. The calculation formula for dilated convolution is:

[0046]

[0047] in, Indicates that the output feature map is at coordinate The pixel value at is the weight of the convolution kernel, is the pixel value of the input feature map, is half the size of the convolution kernel, is the dilation rate, m represents the first index variable of the convolutional kernel, and n represents the second index variable of the convolutional kernel. The existence of the dilation rate enables the convolutional kernel to extract features in a larger receptive field. For example, when r = 2, the convolutional kernel will skip one pixel for the convolution operation, enabling the model to obtain more extensive image information without losing details. In the ResNet module, by introducing dilated convolutions, the model can extract large-scale contour information on the surface of the decorative panel and capture fine features at small scales such as around the lock slot and the meshing area of the clamping block and the clamping groove. After the encoder finishes processing, the multi-scale feature maps are passed to the decoder part. The decoder restores the spatial resolution of the feature maps through transposed convolutions, making the output mask map the same size as the original image. While restoring the spatial dimension, each layer of the decoder fuses the corresponding encoder feature maps into the decoding process through skip connections to achieve multi-scale fusion of features. Through supervised learning training of the U-Net network, a mask map of the target area of the decorative panel is output in the form of semantic segmentation. This mask map accurately labels key parts such as the connection between the arc-shaped panel and the placement panel, the installation surface of the pushing component, around the lock slot, the meshing area of the clamping block and the clamping groove, and the area where the foam adhesive adheres. Morphological features and texture features are calculated based on the mask map of the target area of the decorative panel to construct a primary feature set. In the calculation of morphological features, by analyzing the connected regions in the mask map, features such as the area, perimeter, aspect ratio, and circularity of the regions are extracted. For example, the formula for circularity is:

[0048]

[0049] where represents circularity, is the area of the region, is the perimeter of the region. When is closer to 1, it indicates that the shape of the region is closer to a circle, which is particularly effective in detecting the area where the foam adhesive adheres because the foam adhesive usually distributes in a circular or regular geometric shape on the surface of the decorative panel. For the extraction of texture features, the local binary pattern operator is used. This method generates binary patterns by comparing the gray values of a pixel with its surrounding pixels, and these patterns can effectively reflect the texture information of the region. For example, the wear or irregular texture changes on the installation surface of the pushing component are identified by calculating the entropy of the texture. After the primary feature set is completed, edge features and color features are extracted based on the mask map of the target area of the decorative panel to obtain an intermediate feature set. In the process of extracting edge features, an improved Canny operator is applied to perform edge detection on the target area, and the structural features of the region are evaluated by calculating indicators such as the edge density, directionality, and continuity of the edge length. For example, the entropy of the edge direction distribution is calculated by the following formula:

[0050]

[0051] where represents the directional entropy, is the probability distribution of the th direction in the edge direction histogram, is the total number of directions. When the directional entropy is low, it indicates that the edge lines are relatively regular. The color features are extracted by converting the image to the HSV color space and calculating the statistical information of the hue, saturation, and brightness of the pixels within the region, including the mean, variance, skewness, and kurtosis. The multi-dimensional color features can identify the color consistency of the surface material of the decorative panel and judge problems such as whether there is adhesive residue or color contamination at the connection between the arc-shaped panel and the placement panel through the color change pattern. Depth features are extracted from the mask image of the target area of the decorative panel to obtain the target feature vector. During the depth feature extraction process, a pre-trained EfficientNet-B0 model is used. The masked area image is input into the deep learning network, and a 1280-dimensional high-dimensional feature vector is extracted through the convolutional neural network. These feature vectors contain the spatial and texture information of the image and fuse the abstract semantic features learned by the depth model on a large-scale dataset. For example, through the activation pattern in the depth features, small defects in complex scenes are identified, such as fine cracks around the lock groove or assembly errors in the engagement area. Feature selection is performed on the primary feature set, intermediate feature set, and target feature vector to obtain an optimized feature vector set. During the feature selection process, the random forest algorithm is used to calculate the contribution of each feature to the identification of decorative panel defects. By analyzing the importance scores of the features, redundant features with low contribution are removed, and only the most discriminative features are retained. For example, the Gini coefficient or information gain method is used to quantitatively evaluate the importance of each feature in the classification task, so that the feature vector set is both comprehensive and efficient.

[0052] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0053] Construct a decorative panel defect classification network based on ResNet-v1-50, which includes a feature extraction convolutional layer, a global average pooling layer, two fully connected layers, and a Softmax classification layer;

[0054] Perform regional extraction on the standardized image data to obtain a first image dataset, a second image dataset, and a third image dataset. Among them, the first image dataset contains the complete decorative panel image, the second image dataset focuses on the image of the connection area between the arc-shaped panel and the placement panel, and the third image dataset focuses on the image of the pushing component area;

[0055] Correspond the feature vector set with the first image dataset, the second image dataset, and the third image dataset respectively to obtain the model training data;

[0056] Perform the first-stage training on the decorative panel defect classification network, lock the convolutional block parameters in the feature extraction convolutional layer, and only optimize the parameters of the two fully connected layers to obtain a classification model that is initially adapted to the data distribution of the decorative panel;

[0057] Perform the second-stage training on the classification model that is initially adapted to the data distribution of the decorative panel, unlock all network layer parameters for full network fine-tuning, and obtain an optimized classification model;

[0058] Based on the optimized classification model, perform inference on the test parts of the first image dataset, the second image dataset, and the third image dataset respectively to obtain a defect classification model;

[0059] Calculate the gradient-weighted class activation mapping according to the defect classification model to obtain a candidate defect region mask.

[0060] Specifically, design an efficient classification network architecture. This network is based on ResNet-v1-50 for feature extraction. By introducing a feature extraction convolutional layer, a global average pooling layer, two fully connected layers, and a Softmax classification layer, a deep learning model capable of accurately identifying automotive decorative panel defects is constructed. In the network structure, the feature extraction convolutional layer part of ResNet-v1-50 consists of 50 convolutional layers with residual connections. Each residual module realizes the addition operation of the front and back feature maps through skip connections, thus effectively solving the gradient vanishing problem in deep networks. During the feature extraction process, multi-scale features in the decorative panel image are extracted through convolutional operations, and the model's perception ability of high-semantic information in the image is gradually improved through feature map stacking and downsampling operations. The global average pooling layer is used to compress the feature map from the spatial dimension (such as 7×7×2048) into a one-dimensional vector (such as 2048 dimensions). This operation reduces the number of parameters of the model and improves the model's robustness to overfitting. In the design of the fully connected layer, the model adopts a two-layer structure, reducing the 2048-dimensional feature vector to 1024 dimensions through linear transformation, and then further reducing it to 256 dimensions, and outputting the probability distribution of binary classification (good product / defect) through the Softmax classification layer. The calculation formula of the Softmax classification layer is:

[0061]

[0062] where, represents the input image belongs to the target defect category the probability of, is the category output by the fully connected layer the corresponding activation value of, is the total number of categories (2 in this task, representing good products and defects), and e is the index parameter in the summation formula. The real activation value output by the model is converted into a probability distribution between 0 and 1 through the Softmax operation. After the classification network is constructed, the standardized image data is extracted by region, and multi-scale image analysis is achieved by dividing the original data set into the first image data set. The second image data set and the third image data set. Among them, the first image data set contains images of complete decorative panels for coarse-grained evaluation of overall quality, the second image data set focuses on the connection between the curved plate and the placement plate, which is a key area where stress concentration and assembly errors are prone to occur, and the third image data set focuses on the push component area to capture subtle defects such as wear or poor engagement on the component surface with higher resolution. In the regional image data set, by corresponding the feature vector set to each data set one by one, the model is trained specifically according to the data characteristics of images of different scales, thereby maintaining a high defect detection accuracy at both global and local scales. After completing the data preparation, the first stage of training is performed on the decorative panel defect classification network. The parameters of the convolution blocks in the feature extraction convolution layer are locked, and only the parameters of the two fully connected layers are optimized. This operation freezes the weights of the first four convolution modules in ResNet-v1-50 and only adjusts the parameters of the newly added fully connected layer, so that the model can quickly adapt to the distribution characteristics of the decorative panel image data while maintaining the original feature extraction capabilities. In the first stage of training, the model uses the cross entropy loss function, which is calculated as follows:

[0063]

[0064] in, represents the loss value, is the one-hot encoding of the actual label, is the model prediction category The probability, where Q is the number of samples. This loss function guides the update direction of the model parameters by measuring the distance between the predicted probability distribution of the model and the actual labels, enabling the fully connected layer to gradually learn the feature patterns related to defects in the decorative panel images. In specific training, the stochastic mini-batch gradient descent algorithm is adopted. By setting a relatively large initial learning rate (such as 0.001), rapid convergence is achieved in the early training stage of the model. And through the learning rate decay strategy, the model updates the parameters at a more stable speed when approaching the optimal solution. After completing the first-stage training and obtaining a classification model that is initially adapted to the data distribution of the decorative panel, the second-stage training is entered. At this time, all network layer parameters are unlocked, and the entire model is fine-tuned to obtain an optimized classification model. In this stage, the model no longer only adjusts the weights of the fully connected layer, but allows all convolutional layer parameters to participate in the update through a smaller learning rate (such as 0.0001), enabling the feature extraction part of the model to focus more on the detailed features in the decorative panel images, such as the fine cracks on the edge of the arc-shaped plate or the assembly deviation in the meshing area of the clamping block and the card slot. To prevent overfitting, regularization techniques (such as Dropout and weight decay) are introduced during the training process of the model. By randomly discarding some neurons in the fully connected layer (for example, the Dropout ratio is 0.5), the model pays more attention to global features rather than relying on a single feature. The training termination condition of the model is set to automatically stop training when the accuracy of the validation set does not improve for five consecutive epochs to avoid overfitting of the model on the training data. After obtaining the optimized classification model, based on this model, the test parts of the first image dataset, the second image dataset, and the third image dataset are respectively inferred. Whether there are defects in the decorative panel images is judged through the classification output of the model, and the specific defect category prediction results are output. In the inference stage, by analyzing the classification score distribution of the model at each pixel point, more refined defect localization information is obtained. In addition, to achieve the visualization and precise localization of the defect area, the gradient-weighted class activation mapping (Grad-CAM) is calculated based on the optimized classification model to generate a mask for the candidate defect area. During the Grad-CAM calculation process, the target score function is defined, which represents the score of the model predicting that the image belongs to the defect class. Then, the gradient of this score with respect to the feature map of the last convolutional layer of the model is calculated:

[0065]

[0066] where is the weight of the feature map channel , is the spatial size of the feature map, represents the gradient of the target score with respect to the feature map at the coordinate , is the image coordinate at The convolutional feature map. By multiplying the weights with the feature map pixel by pixel and summing them up, a two-dimensional heat map is generated. The negative values are set to zero through the ReLU activation function to highlight the defective areas that the model focuses on the most, and a candidate defective area mask is obtained.

[0067] In a specific embodiment, the process of performing the step of calculating the gradient-weighted class activation mapping according to the defect classification model to obtain the candidate defective area mask may specifically include the following steps:

[0068] Define an objective scoring function according to the defect category in the defect classification model, and calculate the gradient value of the objective scoring function with respect to the convolution kernel corresponding to the convolutional layer feature map in the defect classification model to obtain objective gradient data;

[0069] Perform global average pooling operation on the objective gradient data in the spatial dimension to obtain the importance weights of the feature map channels;

[0070] Multiply the importance weights of the feature map channels with the corresponding feature map and sum them up in the channel dimension to obtain an initial heat map;

[0071] Generate multi-scale heat maps based on the intermediate layer feature maps of the defect classification model, and perform weighted fusion on the initial heat map and the multi-scale heat maps according to preset weight coefficients to obtain an optimized heat map;

[0072] Perform morphological opening and closing operations on the optimized heat map to obtain a candidate defective area mask.

[0073] Specifically, an objective scoring function is defined for a specific defect category in the defect classification model, and this function is used to quantify the confidence of the model when predicting that an image belongs to a certain specific defect category. The objective scoring function is defined as the output value corresponding to the target defect category in the Softmax classification layer . For example, for a binary classification model (good product / defect), if the probability that the model predicts the image as a defective product is , then the objective scoring function is expressed as:

[0074]

[0075] where is the activation value of the target defect category output by the model in the fully connected layer, is the total number of classification categories, and b is the index parameter in the summation formula. By selecting a specific (e.g., the "defect" category), only focus on the responses of this category when calculating the gradient. After defining the target score function, calculate the gradient value of this score function with respect to the convolutional kernels corresponding to the feature maps of the convolutional layers in the model to obtain the target gradient data. During the gradient calculation process, select the feature map of the last convolutional layer of the model as the object for gradient calculation, and the mathematical expression of the gradient is:

[0076]

[0077] where, represents the target score function with respect to the gradient of the pixel value at the position in the feature map . By calculating this gradient, the contribution of each pixel in the feature map to the model's prediction of the defect category is quantified. The larger the gradient value, the more crucial the pixel plays in the classification decision. This process is implemented in combination with the backpropagation algorithm, and the error is passed from the output layer to the target convolutional layer through the chain rule to calculate the gradient values of all pixels. After obtaining the target gradient data, perform global average pooling operation on these gradient data in the spatial dimension to calculate the importance weights of the feature map channels. In a convolutional neural network, the feature map has multiple channels (e.g., with a size of , where and are the height and width of the feature map respectively, and is the number of channels), and global average pooling reduces the gradient value of each channel to a scalar weight by taking the average of the feature map in the spatial dimension, and its calculation formula is:

[0078]

[0079] In this formula, represents the importance weight of the feature map channel , is the gradient value, and are the height and width of the feature map respectively, and i, j are the index numbers corresponding to the summation formula. Through the global average pooling operation, the weight of each channel is proportional to its importance in predicting the target category, and this method can automatically learn the channel features that the model focuses on during the decision-making process. Multiply the importance weights of the feature map channels with the corresponding feature map pixel by pixel and sum in the channel dimension to generate the initial heatmap. The calculation formula of the initial heatmap is:

[0080]

[0081] where, Indicates the value of the heat map at the position ., is the weight of the channel ., is the pixel value of the feature map at the position ., the ReLU activation function is used to filter out values less than 0, and K is the number of channels. Ensure that the heat map only retains the regions that have a positive contribution to the target category. In the generated initial heat map, the highlighted regions represent the parts of the image that contribute more to the target defect category in the model prediction. By this method, the attention concentration regions of the model are visually displayed on the image. For example, in the decorative panel image, the potential defect parts at the connection between the arc plate and the placement plate or in the push component region are shown. To improve the accuracy of the heat map, multi-scale heat maps are generated based on the intermediate layer feature maps of the defect classification model, and the initial heat map and the multi-scale heat maps are weighted and fused according to the preset weight coefficients to obtain an optimized heat map. Select the feature maps of the intermediate layers of the model (such as the res3d_branch2c and res4f_branch2c layers of ResNet), and calculate the Grad-CAM heat maps of these feature maps respectively. These heat maps represent the attention regions of the model on different scale features. For example, when detecting the meshing area of the clip and the slot of the car decorative panel, the shallower feature maps focus on the local edge features, while the deeper feature maps pay more attention to the overall texture and morphological patterns. By setting the weight coefficient (such as [0.2, 0.3, 0.5]), the heat maps at different levels are weighted and fused, and its calculation formula is:

[0082]

[0083] where is the pixel value of the optimized heat map at the position ., is the weight coefficient of the heat map of the th layer, is the value of the th heat map at the corresponding position, is the number of feature maps participating in the fusion. Through the method of multi-scale weighted fusion, the information of the model at different feature scales is effectively integrated, so that the generated optimized heat map retains both global information and high sensitivity to the detailed regions. Perform morphological opening and closing operations on the optimized heat map to obtain the candidate defect region mask. In the image processing process, the opening operation is used to eliminate small noises. By first eroding and then dilating, the isolated high-bright points in the heat map are removed, while the closing operation fills the small holes in the heat map by first dilating and then eroding, making the morphology of the defect region more complete and coherent.

[0084] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0085] Perform multi-scale hierarchical analysis based on the candidate defect region mask to obtain the first-scale hierarchical region, the second-scale hierarchical region, and the third-scale hierarchical region;

[0086] Perform coarse-grained defect detection on the first-scale hierarchical region to obtain a preliminary defect judgment result;

[0087] Perform refined analysis on the second-scale hierarchical region based on the preliminary defect judgment result to obtain a defect localization result at the connection between the arc plate and the placement plate;

[0088] According to the defect localization result at the connection between the arc plate and the placement plate, perform defect localization on the third-scale hierarchical region to obtain target defect position data;

[0089] Divide the automotive decorative panel into multiple grids, and count the cumulative values of the historical defect heat maps of the preliminary defect judgment result, the defect localization result at the connection between the arc plate and the placement plate, and the target defect position data in each grid to obtain defect frequency distribution data;

[0090] Perform standardization processing on the defect frequency distribution data and divide the risk levels to obtain a defect risk heat map;

[0091] Determine the high-risk areas according to the defect risk heat map, and perform clamping quality analysis on the connection areas between the clamping blocks and the card slots in the high-risk areas to obtain a clamping quality evaluation result.

[0092] Specifically, a multi-scale hierarchical analysis is performed on the candidate defect region mask. By decomposing the image into regions of different scales, a progressive analysis method from the whole to the local is realized. The decorative panel image is segmented into the first-scale hierarchical region, the second-scale hierarchical region, and the third-scale hierarchical region. These three levels correspond to different detection accuracies and target regions. For example, the first-scale hierarchical region covers the entire decorative panel image and is used for coarse-grained global analysis to quickly screen out significant defect regions on the decorative panel. The second-scale hierarchical region focuses on the connection between the arc plate and the placement plate, which is a part where mechanical stress is concentrated and material deformation or assembly errors are likely to occur. The third-scale hierarchical region further narrows the analysis scope and focuses on the connection region between the pushing component and the clamping block and the card slot. By inputting a higher-resolution image into the model, the system can identify subtle defects on the surface of the connection region, such as slight surface scratches, tiny cracks at the connection, or material wear marks. Coarse-grained defect detection is performed on the first-scale hierarchical region to obtain a preliminary defect judgment result. The entire decorative panel image is quickly scanned through a defect classification model, and combined with the highlighted region of the candidate defect region mask, a statistical analysis method based on pixel activation values is used to generate a preliminary defect heat map. In the mathematical model, the calculation formula for the preliminary defect heat map is expressed as:

[0093]

[0094] where, represents the pixel value of the preliminary heat map at position , is the number of defect masks participating in the analysis, is the th defect mask's binary value (1 indicates the presence of a defect, 0 indicates no defect) at position , It is the probability value predicted by the model for the corresponding defect. Through this method, areas with defects are identified globally. Especially in the complex texture background of the overall decorative panel, the high-risk areas of defects can be quickly located. Based on the preliminary defect judgment results, a more refined analysis is performed on the second-scale hierarchical area to obtain the specific defect location results at the connection between the arc plate and the placement plate. During the refined analysis process, the model adopts a multi-scale feature fusion method, combines the preliminary heat map with multi-scale feature maps, and filters out noise areas by setting a relatively high detection threshold, only retaining the pixel points with high confidence in the defect category by the model. For example, by introducing morphological analysis techniques, geometric features of the defect area (such as area, perimeter, shape factor, etc.) are calculated to accurately depict the defect contour at the connection between the arc plate and the placement plate. The calculation formulas used for analyzing geometric features are similar to those for analyzing the connected areas in the above-mentioned mask map and will not be elaborated here. For example, when detecting the connection between the arc plate and the placement plate of the decorative panel, the model can accurately locate the material cracking or coating peeling caused by stress concentration during the assembly process by analyzing these shape features. Based on the defect location results at the connection between the arc plate and the placement plate, the analysis scope is further narrowed, and the focus is shifted to the third-scale hierarchical area for refined defect location analysis of the connection area between the pushing component and the clamping block and the card slot to obtain the target defect position data. At this stage, the image is analyzed at the pixel level by combining high-resolution image data with a deep learning model. For example, through the activation patterns in the feature maps of the convolutional neural network, the model identifies the micro-cracks, metal fatigue marks, or material aging phenomena existing in the connection area. At the same time, by introducing edge detection operators (such as the Canny operator) to calculate the continuity and linearity of the edges in the connection area, it can also assist in judging whether the mechanical engagement state between the clamping block and the card slot is good, such as whether there are problems with clamping block deformation or card slot wear. The multi-level progressive analysis method ensures that the model can maintain a high detection accuracy at both the global and local scales, especially showing excellent defect recognition ability in complex decorative panel structures. After obtaining the target defect position data, the automotive decorative panel is divided into multiple grids, and by statistically accumulating the historical defect heat map values within each grid, defect frequency distribution data is obtained. For example, by dividing the decorative panel image into grids, recording the defect occurrence frequencies in each grid at different time periods, and calculating the cumulative value of the historical heat map :

[0095]

[0096] where, is the grid position at which the cumulative defect frequency is is the time period, is the The value of the heat map at a certain period in a position By this step, the probability distribution of defects occurring in different regions of the decorative panel is effectively quantified. The defect frequency distribution data is standardized, and the risk levels are divided to generate a defect risk heat map. During the standardization process, by normalizing the cumulative value to the range of 0 to 1, and according to the preset risk level criteria (for example, 0 to 0.2 is low risk, 0.2 to 0.4 is medium risk, 0.4 to 0.6 is high risk, 0.6 to 0.8 is extremely high risk, 0.8 to 1.0 is the highest risk), different color codings are assigned to each grid. For example, blue represents low risk and red represents high risk. After determining the high-risk areas, the latching quality analysis is performed on the connection areas between the card blocks and the card slots in these areas. By analyzing the texture features, edge features, and light reflection characteristics of the connection area surface, the quality of the latching is accurately evaluated. For example, the surface features extracted by the support vector machine algorithm are input into the latching state classification model, and the latching quality is divided into four grades: excellent, good, average, and poor. Combining with the high-risk areas in the risk heat map, the analysis results are compared with the historical defect data to determine whether there are abnormal process problems in the current production process.

[0097] In a specific embodiment, the process of performing the steps to determine the high-risk areas according to the defect risk heat map and perform the latching quality analysis on the connection areas between the card blocks and the card slots in the high-risk areas to obtain the latching quality evaluation results may specifically include the following steps:

[0098] According to the defect risk heat map, the high-risk areas with risk values greater than the preset target value are screened. The high-risk areas include the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area;

[0099] Extract the connection areas between the card blocks and the card slots in the high-risk areas to obtain the surface images of the connection areas, and perform multi-feature extraction on the surface images of the connection areas to obtain the surface morphology feature set;

[0100] Construct a mapping relationship model between the latching state and the surface morphology. The mapping relationship model describes the characteristics of regular texture distribution and edge lines on the surface in the fully engaged state, and the characteristics of irregular light reflection patterns caused by surface deformation in the insufficient latching state;

[0101] Based on the surface morphology feature set and the mapping relationship model, perform latching quality analysis to obtain the latching quality evaluation results.

[0102] Specifically, high-risk areas with risk values greater than a preset target value are screened out from the defect risk heat map. The defect risk heat map is generated based on historical defect frequency distribution data by dividing the decorative panel image into multiple grids and counting the frequency distribution of defects in each grid. In the risk screening stage, by setting a risk threshold , for example , areas with risk values greater than the threshold are marked as high-risk areas, and its mathematical expression is:

[0103]

[0104] where represents the state of the high-risk mask map at position , 1 indicates a high-risk area, and 0 indicates a safe area. The risk value is the cumulative defect frequency data after standardization processing. For example, in areas where the structure is complex, mechanical stress is concentrated, and materials are prone to wear, such as the connection between the pushing block and the rotating groove, the edge of the connection between the arc-shaped plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area, high-risk areas are more likely to appear. Through the binary mask map based on the risk value, the high-risk areas that need to be focused on are accurately screened out in the entire decorative panel image. After determining the high-risk areas, the image of the connection area between the clamping block and the clamping groove in these areas is extracted to obtain the surface image of the connection area. During the image extraction process, by performing a pixel-level multiplication operation on the high-risk mask map and the original image, only the pixel values within the high-risk areas are retained, and the other parts are set to zero, so as to separate the connection area between the clamping block and the clamping groove from the complex decorative panel background. This method can ensure that only the high-risk areas are calculated during subsequent analysis, effectively reducing the consumption of computing resources and avoiding noise interference from irrelevant areas. After obtaining the surface images of the connection areas, multi-feature extraction is performed on these images to construct a surface morphology feature set. During the feature extraction process, attention is paid to the geometric morphology, texture features, edge features, and light reflection characteristics of the surface. For example, by calculating the shape features of the surface, the geometric features of the connection area are effectively quantified, which helps to identify whether the clamping block is deformed and whether there is mechanical wear in the connection area. In terms of texture feature extraction, the gray distribution features of the surface pixels are calculated through the local binary pattern method, such as energy, contrast, correlation, and entropy values. These features can reflect the material consistency of the surface and identify potential defects in the connection area through the texture change pattern, such as material microcracks caused by mechanical stress or surface wear caused by long-term friction. Edge detection is performed on the connection area image through the improved Cannv operator, and the density, directionality, and continuity of the edges are calculated, which helps to evaluate the mechanical meshing state between the clamping block and the clamping groove. For example, in the fully meshed state, the edge lines of the connection area should appear as regular parallel lines or closed boundaries, while in the case of insufficient clamping or poor assembly, the edge features show discontinuous, irregular, or broken line features. After completing the construction of the surface morphology feature set, a mapping relationship model between the clamping state and the surface morphology is established. This model associates the surface features with the actual clamping quality state to achieve quantitative quality assessment. In the mapping relationship model, by collecting a large number of sample data with known clamping states and using classification algorithms such as support vector machines or random forests, a mathematical model is established between the surface morphology features and the clamping state. For example, in the fully meshed state, the edge density and texture energy are usually relatively high, while the skewness of the light reflection pattern is close to zero, indicating a uniform light distribution without obvious high-light or shadow areas. The mathematical expression of the model is represented by the classification decision function as:

[0105]

[0106] Among them, is the clamping state prediction function, , , are the feature weights of the model, is the bias value, is the input surface morphology feature vector. By training the model, the prediction function can output the corresponding quality grades under different clamping states, such as excellent, good, average, poor, etc. During the model inference process, by inputting the surface morphology features in the high-risk area into the mapping relationship model, the score of the clamping quality is calculated, and the clamping quality is divided into different grades according to the range of the score. For example, when it indicates that the clamping state is excellent, it is good when is average, and less than 0.4 is determined to be a poor clamping state. After obtaining the clamping quality evaluation result, combined with the high-risk area data in the risk heat map, by analyzing the frequency distribution of historical defects, it is judged whether the quality state in the current production process is stable. For example, in a specific grid in the connection area, if it is in a high-risk state for a long time, it is necessary to adjust the production equipment, process parameters or raw materials to reduce the potential quality risk. And by comparing the current clamping quality evaluation result with historical data, potential problems existing in the production line are identified, such as whether there is a consistent clamping deviation in a specific device, or whether there is a periodic assembly error in a specific process.

[0107] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments and will not be described herein again.

[0108] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an intelligent detection device for the processing process of automotive decorative panels (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0109] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of various embodiments of the present invention.

Claims

1. An intelligent detection method for automobile decorative panel processing, characterized in that: The method comprises: Carry out high-resolution image acquisition and preprocessing of automobile decorative panels to obtain standardized image data; Performing a region segmentation operation on the standardized image data to obtain a decorative plate target region mask map, and calculating morphological features and texture features based on the decorative plate target region mask map to obtain a primary feature set, and extracting edge features and color features according to the decorative plate target region mask map to obtain an intermediate feature set; extracting depth features from the decorative plate target region mask map to obtain a target feature vector, and performing feature selection on the primary feature set, the intermediate feature set, and the target feature vector to obtain a feature vector set; The feature vector set is input into the transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and the candidate defect area mask is calculated by the defect classification model; specifically comprising: constructing a decorative panel defect classification network based on ResNet-v1-50, the decorative panel defect classification network comprising a feature extraction convolution layer, a global average pooling layer, two fully connected layers and a Softmax classification layer; performing region extraction on the standardized image data to obtain a first image data set, a second image data set and a third image data set, wherein the first image data set contains a complete decorative panel image, the second image data set focuses on the image of the area where the curved panel and the placement panel are connected, and the third image data set focuses on the image of the push component area; the ... to obtain a defect classification model, and the candidate defect area mask is calculated by the defect classification model; the feature vector set is input into the transfer learning network based on ResNet-v1-50 The feature vector set corresponds to the first image data set, the second image data set and the third image data set respectively, and the model training data is obtained; the first stage training is performed on the decorative panel defect classification network, the convolution block parameters in the feature extraction convolution layer are locked, and only the parameters of the two fully connected layers are optimized to obtain a classification model that is initially adapted to the decorative panel data distribution; the second stage training is performed on the classification model that is initially adapted to the decorative panel data distribution, all network layer parameters are unlocked to perform full network fine-tuning, and an optimized classification model is obtained; based on the optimized classification model, the test parts of the first image data set, the second image data set and the third image data set are respectively inferred to obtain a defect classification model; according to the defect classification model, a gradient weighted class activation map is calculated to obtain a candidate defect area mask; A multi-scale hierarchical analysis is performed on the candidate defect area mask to obtain a defect risk heat map, and a card-fitting quality analysis is performed on the connection area between the card block and the card slot according to the defect risk heat map to obtain a card-fitting quality assessment result.

2. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The high-resolution image acquisition and preprocessing of the automobile decorative panel to obtain standardized image data includes: Using an industrial camera to collect images of automobile decorative panels to obtain an original decorative panel image, and performing size standardization processing on the original decorative panel image to obtain a standardized size image; Performing histogram equalization processing on the standardized size image to obtain a contrast enhanced image, and performing non-local mean filtering on the contrast enhanced image to obtain a denoised image; Performing adaptive threshold segmentation on the denoised image to obtain a binary mask image of the decorative plate, and performing a background removal operation on the denoised image using the binary mask image of the decorative plate to obtain a pure decorative plate image; Image enhancement processing is performed on the pure decorative plate image to obtain standardized image data.

3. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The performing of a region segmentation operation on the standardized image data to obtain a mask image of a target region of a decorative plate includes: The standardized image data is input into a U-Net network for feature extraction to obtain a multi-scale feature map, wherein the U-Net network comprises a four-layer encoder and a four-layer decoder, each layer of the encoder uses a ResNet module with a hole convolution to extract features, and each layer of the decoder restores the spatial dimension of the feature map by transposed convolution; Performing network training based on the multi-scale feature map to obtain a region segmentation model; The standardized image data is segmented using the region segmentation model to obtain a mask map of the target region of the decorative panel, wherein the target region of the decorative panel includes the connection between the arc plate and the placement plate, the installation surface of the push component, the periphery of the lock slot, the engagement area between the card block and the card slot, and the foam adhesive attachment area.

4. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The step of calculating the gradient weighted class activation map according to the defect classification model to obtain a candidate defect region mask includes: Defining a target score function according to the defect category in the defect classification model, and calculating the gradient value of the target score function to the convolutional layer feature map in the defect classification model to obtain feature map gradient data; Performing a global average pooling operation on the feature map gradient data in the spatial dimension to obtain feature map channel importance weights; Multiply the feature map channel importance weight by the corresponding feature map and sum them in the channel dimension to obtain an initial heat map; Based on the intermediate layer feature map of the defect classification model, a multi-scale thermal map is generated, and the initial thermal map and the multi-scale thermal map are weightedly fused according to a preset weight coefficient to obtain an optimized thermal map; Morphological opening and closing operations are performed on the optimized heat map to obtain a candidate defect region mask.

5. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The multi-scale hierarchical analysis is performed on the candidate defect area mask to obtain a defect risk heat map, and the connection area of ​​the card block and the card slot is analyzed for the card quality according to the defect risk heat map to obtain a card quality assessment result, including: Performing multi-scale hierarchical analysis based on the candidate defect region mask to obtain a first scale hierarchical region, a second scale hierarchical region, and a third scale hierarchical region; Performing coarse-grained defect detection on the first scale level area to obtain a preliminary defect judgment result; Based on the preliminary defect judgment result, a refined analysis is performed on the second scale level area to obtain a defect location result at the connection between the curved plate and the placement plate; According to the defect location result of the connection between the arc plate and the placement plate, the defect location is performed on the third scale level area to obtain the target defect location data; Divide the automobile decorative panel into a plurality of grids, and count the preliminary defect judgment results, the defect location results at the connection between the curved panel and the placement panel, and the cumulative value of the historical defect heat map of the target defect location data in each grid to obtain defect frequency distribution data; Standardizing the defect frequency distribution data and dividing the risk levels to obtain a defect risk heat map; A high-risk area is determined according to the defect risk heat map, and a clamping quality analysis is performed on a connection area between the clamping block and the clamping slot in the high-risk area to obtain a clamping quality evaluation result.

6. The intelligent detection method for automobile decorative panel processing according to claim 5 is characterized in that: The step of determining a high-risk area according to the defect risk heat map, and performing a card-fitting quality analysis on a connection area between a card block and a card slot in the high-risk area to obtain a card-fitting quality assessment result includes: According to the defect risk heat map, high-risk areas with risk values ​​greater than a preset target value are screened, wherein the high-risk areas include the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area; Extracting the connection area between the card block and the card slot in the high-risk area to obtain a surface image of the connection area, and performing multi-feature extraction on the surface image of the connection area to obtain a surface morphology feature set; Constructing a mapping relationship model between engagement state and surface morphology, wherein the mapping relationship model describes the characteristics of regular texture distribution and edge lines on the surface in a fully engaged state, and the characteristics of irregular light reflection patterns caused by deformation of the surface in an insufficiently engaged state; The engagement quality analysis is performed based on the surface morphology feature set and the mapping relationship model to obtain an engagement quality evaluation result.

Images