Intelligent detection method for machining process of automobile decorative plate

Abstract

The invention relates to the technical field of image detection, and discloses an intelligent detection method for the machining process of an automobile decorative plate. The method comprises the following steps: performing high-resolution image acquisition and preprocessing on the automobile decorative plate to obtain standardized image data; performing region segmentation operation on the standardized image data to obtain a decorative plate target region mask image, and extracting to obtain a feature vector set; inputting the feature vector set into a ResNet-v1-50-based transfer learning network for training to obtain a defect classification model, and calculating candidate defect area masks through the defect classification model; and performing multi-scale analytic hierarchy process on the candidate defect area mask to obtain a defect risk heat map, and performing clamping quality analysis on the connection area of the clamping block and the clamping groove according to the defect risk heat map to obtain a clamping quality evaluation result. According to the method, accurate positioning of the defect area is realized, the defect risk heat map is constructed to quantify the risk levels of different areas, and data support is provided for production process optimization.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an intelligent detection method for a processing process of an automobile decorative panel. Background Art

[0002] With the rapid development of the automobile industry, consumers have increasingly higher requirements for the quality of automobile interior decoration parts. As an important interior component, automobile decorative panels not only bear the function of aesthetics, but also need to ensure structural stability and safety. The main function of the car trunk threshold decorative panel is to cover the trunk threshold sheet metal to prevent passengers from being scratched by the threshold sheet metal when loading and unloading luggage. Its installation quality involves the correct connection between multiple components, which is crucial to the overall function and service life of the decorative panel. However, the existing decorative panel inspection mainly relies on manual visual inspection, lacks objective, accurate and efficient automated inspection methods, and is difficult to meet the quality control needs of large-scale production.

[0003] At present, the detection of defects in automobile decorative panels mainly relies on image detection technology. However, due to the large number of defect types and different forms, the existing detection methods lack universal and clear detection standards. Traditional machine vision detection methods are mostly based on artificially designed features and simple threshold judgments, and the detection accuracy of decorative panels with complex structures is limited. Although deep learning methods can automatically learn features, they often require a large number of labeled samples, and decorative panel defect samples are difficult to obtain and unbalanced, making it difficult to effectively train the model. More importantly, it is difficult for existing detection methods to evaluate the quality of the engagement between decorative panel components, especially the engagement state between the card block and the card slot, and traditional disassembly testing will damage the product and cannot be applied to production line detection. Summary of the invention

[0004] The present application provides an intelligent detection method for the processing of automobile decorative panels. The present application realizes the precise positioning of defect areas, constructs a defect risk heat map to quantify the risk levels of different areas, and provides data support for production process optimization.

[0005] In a first aspect, the present application provides an intelligent detection method for a processing process of an automobile decorative panel, the intelligent detection method for a processing process of an automobile decorative panel comprising: Carry out high-resolution image acquisition and preprocessing of automobile decorative panels to obtain standardized image data; Performing a region segmentation operation on the standardized image data to obtain a decorative plate target region mask map, and extracting multimodal features from the decorative plate target region mask map to obtain a feature vector set; Inputting the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculating a candidate defect area mask through the defect classification model; A multi-scale hierarchical analysis is performed on the candidate defect area mask to obtain a defect risk heat map, and a card-fitting quality analysis is performed on the connection area between the card block and the card slot according to the defect risk heat map to obtain a card-fitting quality assessment result.

[0006] In the technical solution provided in this application, multi-modal feature extraction and fusion technology is used to effectively capture multi-dimensional information on the surface of decorative panels, and comprehensively reflect the characteristic manifestations of different types of defects; the transfer learning strategy based on ResNet-v1-50 solves the problem of insufficient defect samples of decorative panels, and significantly improves the classification accuracy, especially at the connection between the curved panel and the placement panel; the gradient weighted class activation mapping technology is combined with multi-level feature fusion to achieve precise positioning of defective areas; the multi-scale hierarchical analysis framework conducts progressive analysis from large images to small images, and gradually improves the detection resolution as the analysis scale is refined, making defect detection more accurate; the plug-in state between decorative panel components is innovatively evaluated based on surface morphological features, and the plug-in quality assessment can be performed without disassembly, providing a feasible solution for online detection of production lines; the risk level of different areas is quantified by constructing a defect risk heat map, providing data support for production process optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0008] Figure 1 It is a schematic diagram of an embodiment of the intelligent detection method for the automobile decorative panel processing process in the embodiment of the present application. DETAILED DESCRIPTION

[0009] The embodiment of the present application provides an intelligent detection method for the processing of automobile decorative panels. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0010] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiment of the present application, an embodiment of the intelligent detection method for the processing of automobile decorative panels includes: Step S101, performing high-resolution image acquisition and preprocessing on the automobile decorative panel to obtain standardized image data; It is understandable that the execution subject of the present application may be an intelligent detection system for automobile decorative panel processing, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0011] Specifically, a dedicated image acquisition system is built, which consists of a high-resolution industrial camera, an LED light source array with a specific wavelength, and a precise positioning device. The industrial camera uses a high-resolution model with 1920×1080 pixels to ensure that the delicate texture and tiny structural features on the surface of the automotive decorative panel are captured. At the same time, in order to ensure uniform illumination of the decorative panel surface, the light source system uses a ring LED light source with a color temperature stable at 5500K±100K. This light source can provide uniform and stable white light irradiation, thereby avoiding shadows or reflections caused by uneven light affecting the image quality. The precise positioning device cooperates with the conveyor belt and the automatic trigger mechanism so that when the decorative panel moves to the specified shooting position, the image acquisition system automatically captures the front view of the decorative panel, realizing a fully automated image acquisition process. The original decorative panel image is size-standardized, and all the collected original images are adjusted to a uniform size of 1280×720 pixels to ensure the consistency of the image data and avoid feature mismatch problems caused by image resolution differences during model training, so as to obtain a standardized size image. The standardized size image is histogram-equalized to enhance the contrast of the image. Histogram equalization is an image enhancement method that stretches the grayscale distribution of an image to enhance the details of the bright and dark areas of the image, making the surface texture of the decorative board clearer and more prominent, and obtaining a contrast-enhanced image. The contrast-enhanced image is subjected to non-local mean filtering. By calculating the similarity between each pixel in the image and the surrounding pixel blocks, similar pixels are weighted averaged to effectively remove Gaussian noise and salt and pepper noise in the image, while maintaining the edge and detail features of the image to obtain a denoised image. An adaptive threshold segmentation operation is performed on the denoised image to generate a binary mask map of the decorative board. Adaptive threshold segmentation is a method of dynamically calculating the binary threshold of each pixel in an image. By analyzing the brightness characteristics of the local area, it effectively copes with the situation of uneven lighting or complex background in the image and accurately separates the decorative board area from the background. Through this step, a binary mask map containing only the shape of the decorative board is obtained. Using the binary mask map, a background removal operation is performed on the denoised image to obtain a pure decorative board image. In the background removal process, the non-decorative board area in the original image is filled with black, so that only the pixel data of the decorative board body is retained in the output image. Image enhancement processing is performed on the pure decorative board image to obtain standardized image data. Image enhancement processing includes image brightness, contrast, color saturation adjustment, and sharpening operation based on specific filters to improve the visual effect of the image and strengthen the expression of the surface features of the decorative board.

[0012] Step S102, performing a region segmentation operation on the standardized image data to obtain a mask map of the target region of the decorative plate, and extracting multimodal features from the mask map of the target region of the decorative plate to obtain a feature vector set; Specifically, the standardized image data is input into the U-Net network for feature extraction to realize the construction of multi-scale feature maps. The U-Net network structure is designed as a symmetrical structure of four-layer encoder and four-layer decoder. The encoder part adopts the ResNet module with dilated convolution to realize the multi-scale and high efficiency of feature extraction, and the decoder part gradually restores the spatial dimension of the feature map by transposed convolution, ensuring that the image segmentation accuracy is improved while maintaining the feature richness. In the encoder, the introduction of dilated convolution significantly expands the receptive field, enabling the network to capture the detailed features and global context information in the decorative panel image, while the ResNet module effectively avoids the problem of gradient vanishing in the deep network through the design of residual connection, so that the model can still stably extract the key area features of the decorative panel in complex backgrounds. The network is trained based on the multi-scale feature map to obtain the regional segmentation model. During the training process, by using a supervised learning method with a weighted boundary loss function, the model gives a higher penalty weight to the boundary area when predicting the target area of ​​the decorative panel, thereby improving the segmentation accuracy of the model at complex boundaries. For example, in the key parts such as the connection between the curved plate and the placement plate, the installation surface of the push component, the periphery of the lock slot, the meshing area between the card block and the card slot, and the foam adhesive attachment area, the model can accurately generate a regional mask map, providing a spatial positioning basis for subsequent feature extraction. After the regional segmentation model is trained, the standardized image data is input into the model, and the model outputs a mask map of the target area of ​​the decorative plate. In this mask map, different target areas are identified by different mask categories. Based on the mask map of the target area of ​​the decorative plate, morphological features and texture features are calculated to construct a primary feature set. The morphological feature extraction method includes calculating the area, perimeter, circularity, aspect ratio and size of the minimum circumscribed rectangle of each target area. These features help to identify potential size deviations or shape anomalies in the meshing area between the card block and the card slot. At the same time, by using the local binary pattern operator to analyze the texture of the target area, the statistical features such as energy, contrast, correlation and entropy value of the pixel grayscale distribution in the area are obtained. These texture features can effectively reflect whether there are defects such as scratches, material cracks or poor welding on the surface of the decorative plate. On the basis of primary feature extraction, edge features and color features are extracted based on the mask map of the target area of ​​the decorative panel to construct an intermediate feature set. In terms of edge feature extraction, edge detection is performed through the improved Canny operator to calculate edge density, directionality and continuity features, which are helpful to promote the identification of fine structures around the component mounting surface and the lock slot. In terms of color feature extraction, by converting the target area to the HSV color space, the mean, variance, skewness and kurtosis of the hue, saturation and brightness in the area are calculated respectively. The color features can distinguish decorative panels of different materials and identify color unevenness or contamination in the foam adhesive attachment area, for example. Deep features are extracted from the mask map of the target area of ​​the decorative panel to obtain the target feature vector.The pre-trained EfficientNet-B0 network was introduced, and the image of the mask area of ​​the decorative panel was input into the network. The high-dimensional deep features were extracted through the convolutional neural network to obtain a 1280-dimensional feature vector. The high-dimensional feature vector was reduced to 128 dimensions through the principal component analysis method, thereby eliminating redundant information and retaining the feature dimensions that play a key role in the identification of decorative panel defects. Feature selection was performed on the primary feature set, the intermediate feature set and the target feature vector to obtain a feature vector set. In the feature selection process, the contribution of each feature to defect identification was calculated by the random forest algorithm. By analyzing the importance score of the feature, the top 80% of the contribution features were retained, thereby eliminating low-correlation or noise features and optimizing the accuracy and efficiency of the feature set.

[0013] Step S103, inputting the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculating a candidate defect area mask through the defect classification model; Specifically, a decorative panel defect classification network based on ResNet-v1-50 is constructed. The design of the network includes a feature extraction convolution layer, a global average pooling layer, two fully connected layers, and a Softmax classification layer. In the network structure, the feature extraction convolution layer retains the basic architecture of ResNet-v1-50, including multi-layer residual modules. These modules effectively avoid the gradient vanishing problem in the deep network by introducing residual connections, and extract multi-scale features in the decorative panel image through convolution operations. The introduction of the global average pooling layer effectively replaces the traditional fully connected layer, converts the feature map into a low-dimensional feature vector, reduces the number of model parameters, and effectively avoids overfitting. In the fully connected layer part, it is designed as a two-layer structure. After the dimensionality reduction operation from 1024 dimensions to 256 dimensions, the features are mapped to the output results of the binary classification (good / defective) through the Softmax classification layer, ensuring that the model quickly outputs the quality judgment results of the decorative panel in the inference stage. The standardized image data is extracted by region, and the original image data set is divided into the first image data set, the second image data set, and the third image data set to achieve targeted analysis of images of different scales. Among them, the first image dataset contains images of complete decorative panels, which are used to evaluate the quality status of the overall decorative panels. Such images maintain a large field of view, which helps the model capture the defect features of the decorative panels on a global scale. The second image dataset focuses on the local images of the connection between the curved plate and the placement plate. This area is a key part of the decorative panel that is prone to deformation or installation deviation. Therefore, the image size of this dataset is small, allowing the model to perform refined detection on a detail scale. The third image dataset focuses on the push component area, which is an important object for mechanical engagement quality assessment. By providing higher-resolution local images, the model can identify problems such as fine cracks, wear or poor assembly on the surface of the push component. The feature vector set is matched to the first image dataset, the second image dataset, and the third image dataset, respectively, to obtain a complete dataset for model training. In the data preparation process, by pairing image data with feature vectors, multimodal input information is provided to the model, which effectively improves the accuracy and robustness of the model in defect recognition tasks. During the training process, the first stage of training is performed, the parameters of the convolution blocks in the feature extraction convolution layer are locked, and only the parameters of the two fully connected layers are optimized. By freezing the front-end feature extraction part of the model, the model can quickly adapt to the data distribution of the decorative panels without spending too much time on relearning the underlying features. In this way, the model uses the common features that have been learned on the ImageNet dataset in the pre-trained ResNet-v1-50 network, and only adjusts the final classification part to match the specific pattern of the decorative panel image, resulting in a classification model that is initially adapted to the data distribution of the decorative panels.In this stage, a larger learning rate (such as 0.001) is selected to speed up the convergence of the model. At the same time, the performance on the validation set is monitored during the training process to ensure that the model improves the classification accuracy without overfitting. The second stage of training is performed on the classification model that initially adapts to the data distribution of the decorative panels, unlocks all network layer parameters, and fine-tunes the entire model. Through the training of the entire network, the model not only adjusts the parameters of the classification layer, but also updates the parameters of the feature extraction convolution layer through the back propagation mechanism, so that the model learns more specific visual features from the decorative panel image, especially the recognition ability of the surface texture, edge details and local morphological features of the decorative panel. In this stage, a smaller learning rate (such as 0.0001) is used to ensure that the adjustment of the model parameters is more refined and stable. At the same time, the learning rate decay strategy and regularization techniques (such as weight decay and Dropout) are combined to prevent the model from overfitting under small or unbalanced sample conditions. In the training process, by setting the training termination conditions (such as the validation set accuracy has not improved for five consecutive cycles or the maximum number of training cycles is reached), it is ensured that the model automatically stops training when the optimal performance is achieved, and an optimized classification model is obtained. Based on the optimized classification model, the test parts of the first image data set, the second image data set, and the third image data set are inferred respectively, and the defect classification results are obtained through model prediction. In the inference stage, the model quickly determines whether the decorative panel has defects based on the feature vector and image data of the input image, and outputs the specific classification results. In order to achieve accurate positioning of the candidate defect area, the gradient weighted class activation map is calculated based on the optimized classification model to generate a heat map to show the image area that the model focuses on when making defect classification decisions. The gradient of the last layer of convolutional feature map is calculated by calculating the score of the model output category, and the gradient information is weightedly fused with the feature map to generate a heat map, thereby highlighting the area on the image that the model believes is most likely to have defects. In order to improve the accuracy of the candidate defect area mask, image processing operations are performed on the heat map, such as setting an adaptive threshold for binarization, and removing noise areas through morphological operations to generate a candidate defect area mask.

[0014] In this embodiment, a target score function is defined according to the defect category in the defect classification model. The target score function represents the output score of a specific defect category predicted by the model. For example, in the Softmax classification layer, the score is represented as the activation value of the specific category. The target score function is expressed as , where c represents the target defect category. By defining the target score function, it is clear which feature areas in the input image the model focuses on when identifying a certain category of defects. The gradient value of the target score function to the feature map of the last convolutional layer in the model is calculated to obtain the target gradient data. During the back propagation process of the model, the target score function is applied to the feature map of the convolutional layer. Calculate the partial derivative, the mathematical expression is ,in Represents the feature map channels. By calculating the influence of each pixel at each position in the feature map on the target classification score, the sensitivity of the model to the classification result at a specific position is quantified. A global average pooling operation is performed on the target gradient data in the spatial dimension to calculate the importance weight of the feature map channel. By averaging the feature map gradient in the width and height dimensions, each channel retains only one weight value, indicating the importance of the channel to the target defect category. The feature map channel importance weight is multiplied by the corresponding feature map and summed in the channel dimension to generate an initial heat map. Based on the intermediate layer feature map of the defect classification model, a multi-scale heat map is generated, and the initial heat map and the multi-scale heat map are weighted fused according to the preset weight coefficient to obtain the optimized heat map. In addition to the feature map of the last convolutional layer, the feature maps of the intermediate layers in the model (such as res3d_branch2c, res4f_branch2c, etc. in ResNet) are usually selected, and the Grad-CAM heat maps of these intermediate layer feature maps are calculated respectively, and different weight coefficients are assigned to each heat map, such as [0.2, 0.3, 0.5]. The weighted fusion strategy can combine the advantages of features at different scales, so that the optimized heat map can capture the overall features of the macroscopic decorative panel and identify the microscopic surface details, thereby improving the positioning ability of the model at multiple scales, especially in complex structures such as the connection between the curved plate and the placement plate or the meshing area between the card block and the card slot, showing stronger recognition accuracy. Perform morphological opening and closing operations on the optimized heat map to refine the candidate defect area mask. Morphological opening operations (using small circular structural elements, such as structural elements with a radius of 3) are used to eliminate noise points in the heat map, such as removing small and isolated high-brightness pixels to avoid these noise points from interfering with the defect area detection results. Morphological closing operations (using circular structural elements with a radius of 5) make the mask of the defect area more coherent and smooth by filling small holes in the heat map, especially in places with complex boundaries such as the push component installation surface or the foam adhesive attachment area, which can significantly improve the morphological integrity of the candidate area. In the morphological operation, the geometric features such as area, perimeter, and circularity of each connected region are calculated. By setting a threshold (for example, eliminating areas with an area less than 30 pixels or a circularity greater than 0.9), the false detection areas are effectively filtered out so that the output candidate defect area mask only contains the areas that the model believes are most likely to have defects.

[0015] Step S104: perform multi-scale hierarchical analysis on the candidate defect area mask to obtain a defect risk heat map, and perform a card-fitting quality analysis on the connection area between the card block and the card slot according to the defect risk heat map to obtain a card-fitting quality assessment result.

[0016] Specifically, a multi-scale hierarchical analysis is performed based on the candidate defect region mask to obtain detection results of different precision and granularity. The multi-scale hierarchical analysis adopts a three-level progressive method, which decomposes the candidate defect region mask into the first scale level region, the second scale level region and the third scale level region, so as to achieve accurate identification and positioning of the defect region in the process from the whole to the local, from the coarse granularity to the fine granularity. The first scale level region mainly retains the image information of the decorative board as a whole, has a large field of view, and provides a coarse-grained global detection perspective, so as to quickly identify significant defect features on the overall structure of the decorative board. When performing coarse-grained defect detection on the first scale level region, the defect classification model constructed above is used to quickly scan the overall decorative board image, and a preliminary defect judgment result is generated by analyzing the highlight area in the candidate defect region mask and combining the classification results output by the model. The defect category existing in the image is determined by the Softmax classification output of the model, and the classification result is mapped back to the specific location in the image to generate a preliminary heat map. According to the preliminary defect judgment results, the analysis scope is narrowed to the second scale level area. The analysis at this stage focuses on the local image of the connection between the curved plate and the placement plate. By inputting higher resolution image data into the model, the model can perform more accurate defect location operations on specific key areas at a finer scale. In this process, Grad-CAM technology is used to generate a refined heat map of the local area, and combined with the morphological analysis method, the false detection area is eliminated by calculating the geometric features such as the area, perimeter, and shape complexity of the area to ensure the accuracy and reliability of the positioning results. According to the defect location results at the connection between the curved plate and the placement plate, the third scale level area is located, and the specific location data of the target defect is accurately calculated. The model uses a high-resolution small image data set for reasoning, and obtains the specific location coordinates of the target defect by analyzing smaller details in the image (such as tiny cracks or assembly errors on the surface of the push component). The automotive decorative panel is divided into multiple grids. By mapping the preliminary defect judgment results, the defect location results at the connection between the curved plate and the placement plate, and the target defect location data to these grids, the cumulative value of the historical defect heat map in each grid is counted to obtain the defect frequency distribution data of different areas of the decorative panel. The grid division adopts a fixed-size grid method. For example, the decorative panel image is divided into 10×10 grids. Each grid represents a specific spatial area. By counting the cumulative defect thermal values ​​in each grid, the probability distribution of defects in different areas of the decorative panel is quantified. The defect frequency distribution data is standardized and divided into risk levels to generate a defect risk heat map.By normalizing the cumulative thermal value of each grid to a range of 0 to 1, and dividing the risk level according to the preset standard, for example, marking the thermal value between 0 and 0.2 as extremely low risk, between 0.2 and 0.4 as low risk, between 0.4 and 0.6 as medium risk, between 0.6 and 0.8 as high risk, and between 0.8 and 1.0 as extremely high risk, quantitative risk assessment results are provided, and the quality status of each area of ​​the decorative panel is intuitively displayed through different color visual coding (for example, blue indicates low risk and red indicates high risk). According to the defect risk heat map, the high-risk area is determined, and the connection area between the card block and the card slot is analyzed to obtain the card quality assessment result. In this process, the focus is on the grids shown as high risk (usually red areas) in the heat map, and the connection status of the card block and the card slot in these areas is analyzed to evaluate whether there is a problem of poor card fit. The specific analysis method includes calculating the texture features, edge features and light reflection patterns of the surface of the connection area, and identifying problems such as engagement deviation, loose connection or assembly error by comparing the features with those of good samples. On this basis, support vector machines or other classification algorithms are used to classify the engagement quality into four levels: excellent, good, fair and poor, and a quantitative engagement quality assessment score is output.

[0017] In this embodiment, high-risk areas with risk values ​​greater than the preset target value are screened out according to the defect risk heat map. A risk threshold is set, for example 0.8. When the risk value of a certain area exceeds this threshold, the area will be marked as a high-risk area. Specifically, it includes the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area, which are areas in the decorative panel structure that are prone to defects due to mechanical movement, stress concentration or material properties. The connection area between the card block and the card slot in the high-risk area is extracted to obtain a surface image of the connection area. The image extraction process combines the candidate defect area mask generated in the early stage with the coordinate data of the high-risk area. By locating the specific position of the card block and the card slot in the decorative panel image, the images in these areas are cropped out to ensure that the extracted image only contains valid information on the surface of the engaging parts and avoid interference from the surrounding background. Multi-feature extraction operations are performed on the surface image of the connection area to construct a surface morphology feature set. In the feature extraction process, the texture features of the surface are calculated. For example, the distribution of the grayscale pattern of the surface pixels is extracted by the local binary pattern (LBP) operator to obtain statistical features such as energy, contrast, correlation and entropy. These features can effectively reflect the material consistency and texture regularity of the surface of the card block and the card slot. The surface edge features are extracted, and the edge density, directionality and continuity are calculated by the improved Canny operator. By analyzing these features, the regularity of the edge lines in the card area is identified, and then the assembly accuracy of the card block and the card slot and the stability of the card state are inferred. In terms of color feature extraction, the image is converted to the HSV color space, and the mean, variance, skewness and kurtosis of hue, saturation and brightness are calculated respectively. These color features are important for identifying the illumination reflection characteristics of the surface of the connection area and judging whether the card state is uneven in surface color due to mechanical deformation. In order to analyze the card quality more comprehensively, the deep features of the region are extracted by the deep learning model. These high-dimensional features effectively capture the subtle morphological changes of the surface, especially for potential quality problems in the connection area that are difficult to detect by traditional image feature extraction methods. A mapping relationship model between the engagement state and the surface morphology is constructed to associate the surface features of the connection area with the actual engagement quality state to achieve quantitative analysis of the engagement quality. When constructing the mapping relationship model, a large amount of sample data of known engagement states is analyzed to extract the distribution law of surface morphological features under different engagement states. For example, in the fully engaged state, the connection area between the card block and the card slot presents a regular texture distribution, the edge lines are clear and continuous, the light reflection pattern is uniform and there is no obvious deformation mark. In the case of insufficient engagement or poor assembly, due to the influence of mechanical stress or structural looseness, the surface of the connection area will have slight deformation, resulting in changes in the light reflection pattern, such as the formation of irregular highlights or shadow areas. The texture features of the surface may also deviate from the normal range, such as broken edge lines, disordered texture distribution, or the appearance of fine cracks.By marking and modeling these surface features and the engagement state, a multi-dimensional mapping relationship model is constructed, and the input surface feature set is mapped to a specific engagement quality state through model prediction. The engagement quality analysis is performed based on the surface morphology feature set and the mapping relationship model to obtain the engagement quality assessment result. The surface features of the connection area in the high-risk area are input into the mapping relationship model, and the prediction result of the engagement state is obtained through model reasoning. The engagement quality is divided into multiple levels, such as excellent, good, general, poor, etc., using classification algorithms such as support vector machines, random forests or deep neural networks. During the evaluation process, a classification result of the engagement quality is output, and a quantitative score is provided, such as a percentage to indicate the degree of engagement quality. In the analysis results, the score corresponding to the excellent engagement state is greater than 90 points, the good state is between 80 and 90 points, the general state is between 70 and 80 points, and the score below 70 points is judged as a poor engagement state.

[0018] In the embodiments of the present application, multi-modal feature extraction and fusion technology is used to effectively capture multi-dimensional information on the surface of the decorative panel, and comprehensively reflect the characteristic manifestations of different types of defects; the transfer learning strategy based on ResNet-v1-50 solves the problem of insufficient defect samples of the decorative panel, and significantly improves the classification accuracy, especially at the connection between the curved panel and the placement panel; the gradient weighted class activation mapping technology is combined with multi-level feature fusion to achieve precise positioning of the defective area; the multi-scale hierarchical analysis framework conducts progressive analysis from large images to small images, and gradually improves the detection resolution as the analysis scale is refined, making defect detection more accurate; the plug-in state between decorative panel components is innovatively evaluated based on surface morphological features, and the plug-in quality assessment can be performed without disassembly, which provides a feasible solution for online detection of the production line; the risk level of different areas is quantified by constructing a defect risk heat map, providing data support for production process optimization.

[0019] In a specific embodiment, the process of executing step S101 may specifically include the following steps: An industrial camera is used to collect images of automobile decorative panels to obtain original decorative panel images, and the original decorative panel images are subjected to size standardization processing to obtain standardized size images; Performing histogram equalization on the standardized size image to obtain a contrast enhanced image, and performing non-local mean filtering on the contrast enhanced image to obtain a denoised image; Performing adaptive threshold segmentation on the denoised image to obtain a binary mask image of the decorative board, and using the binary mask image of the decorative board to perform a background removal operation on the denoised image to obtain a pure decorative board image; Image enhancement processing is performed on the pure decorative plate image to obtain standardized image data.

[0020] Specifically, an image acquisition system is constructed to ensure that the collected original images have sufficient resolution and quality. The image acquisition system includes a high-resolution industrial camera, an LED light source array with a specific wavelength, and a precise positioning device. The industrial camera uses a 1920×1080 pixel camera to ensure the richness of image details. In terms of light source selection, a ring LED light source with a color temperature fixed at 5500K±100K is used. This light source can provide a uniform and stable lighting environment and avoid image quality fluctuations caused by light source changes. Through automatic triggering, when the car decorative panel passes through the conveyor belt to the preset position, the system automatically captures the front view of the decorative panel, thereby realizing automated image acquisition in the efficient operation of the production line. The original image is size-standardized to obtain standardized size images of uniform specifications. All images are adjusted to 1280×720 pixels. The image ratio differences caused by different decorative panel sizes and camera distance changes are eliminated through size normalization, thereby improving the stability of the image processing algorithm. In the process of achieving size standardization, a bilinear interpolation algorithm is used to generate an image of the target size by weighted calculation of the original image pixels, while trying to keep the details and edge clarity in the image intact. The standardized size image is processed by histogram equalization to obtain a contrast-enhanced image. Histogram equalization is an image enhancement technology based on the distribution of pixel grayscale values. It linearizes the cumulative distribution function of the image grayscale value to make the brightness distribution of the image more uniform. In mathematical expression, histogram equalization is achieved through the following formula:

[0021] in, represents the enhanced pixel value, is the grayscale level of the image, usually 256, and are the width and height of the image, The gray value is The number of pixels, Calculated pixel value is not greater than The cumulative number of pixels. Perform non-local mean filtering on the contrast enhanced image to obtain a denoised image. Non-local mean filtering calculates the similarity between each pixel and all pixels in the image and performs weighted averaging on similar pixels, thereby effectively removing Gaussian noise and salt and pepper noise while retaining the image edges and details. The calculation formula for non-local mean filtering is:

[0022] in, Represents pixels The new value after filtering is It's pixels The search window, It's pixels The gray value of It's pixels and The weight coefficient between them is calculated based on the similarity of the pixel neighborhood and is specifically defined as:

[0023] in, and Pixels and The neighborhood block of represents the Euclidean distance, is the filter intensity control parameter. The noise of the image after non-local mean filtering is effectively suppressed, and the clarity of the edge of the decorative board can be retained. Adaptive threshold segmentation is performed on the denoised image to generate a binary mask map of the decorative board. The adaptive threshold segmentation method dynamically calculates the segmentation threshold of each pixel by analyzing the brightness change of the local area of ​​the image, which is suitable for scenes with uneven lighting. The image is divided into multiple small blocks, and the pixel grayscale histogram of each small block is calculated to determine the optimal segmentation threshold of the area. In the generated binary mask map, white pixels represent the decorative board area and black pixels represent the background area. This mask map effectively separates the decorative board from the complex background. Using the binary mask map of the decorative board, the background removal operation is performed on the denoised image to obtain a pure decorative board image. In the background removal process, the pixel values ​​of the background part in the mask map are set to zero, and only the pixel data of the decorative board part is retained, so that only the effective information of the decorative board remains in the processed image. Image enhancement processing is performed on the pure decorative board image to obtain standardized image data. In the image enhancement operation, the contrast-limited adaptive histogram equalization method is used to enhance the local contrast of the image and avoid the over-enhancement phenomenon in traditional histogram equalization. The edge features and brightness balance of the image are enhanced through sharpening filtering and gamma correction technology. For example, the surface details of the decorative plate are enhanced by sharpening filtering to make small scratches or defects more obvious, while gamma correction adjusts the nonlinear mapping of pixel values ​​so that the image can maintain good visibility under different brightness conditions.

[0024] In a specific embodiment, the process of executing step S102 may specifically include the following steps: The standardized image data is input into the U-Net network for feature extraction to obtain a multi-scale feature map. The U-Net network contains four layers of encoders and four layers of decoders. Each layer of encoder uses a ResNet module with hole convolution to extract features, and each layer of decoder restores the spatial dimension of the feature map through transposed convolution. Perform network training based on multi-scale feature maps to obtain a region segmentation model; The standardized image data is segmented using the region segmentation model to obtain a mask map of the target region of the decorative plate, wherein the target region of the decorative plate includes the connection between the curved plate and the placement plate, the installation surface of the push component, the periphery of the lock slot, the meshing area between the card block and the card slot, and the foam adhesive attachment area; The morphological features and texture features are calculated based on the mask image of the target area of ​​the decorative plate to obtain a primary feature set, and the edge features and color features are extracted based on the mask image of the target area of ​​the decorative plate to obtain a mid-level feature set; Deep features are extracted from the mask image of the target area of ​​the decorative plate to obtain a target feature vector, and feature selection is performed on the primary feature set, the intermediate feature set and the target feature vector to obtain a feature vector set.

[0025] Specifically, an improved U-Net network architecture is constructed, which consists of a four-layer encoder and a four-layer decoder. Each encoder layer uses a ResNet module with dilated convolution to enhance feature extraction capabilities, while the decoder part gradually restores the spatial dimensions of the feature map through transposed convolution, thereby retaining the spatial details of the image while maintaining high semantic information. The introduction of dilated convolution, by inserting holes in the convolution kernel, allows the receptive field to be exponentially expanded without increasing the amount of computation, which is especially important in segmentation models because it can capture the multi-scale features of complex structures such as curved panels, push components, and card slots in automotive decorative panels. In the feature extraction process of the U-Net network, the standardized image data is processed by a four-layer encoder, and each encoder layer consists of a ResNet module with dilated convolution, which alleviates the gradient vanishing problem in deep networks through residual connections. The calculation formula for dilated convolution is:

[0026] in, Indicates that the output feature map is at coordinate The pixel value at is the weight of the convolution kernel, is the pixel value of the input feature map, is half the size of the convolution kernel, is the hole rate, m represents the first index variable of the convolution kernel, and n represents the second index variable of the convolution kernel. The existence of the hole rate enables the convolution kernel to extract features in a larger receptive field. For example, when r=2, the convolution kernel will skip a pixel for convolution operation, so that the model can obtain a wider range of image information without losing details. In the ResNet module, by introducing the hole convolution, the model can extract the large-scale contour information of the surface of the decorative plate and capture small-scale fine features such as the periphery of the lock slot and the meshing area between the card block and the card slot. After the encoder processing is completed, the multi-scale feature map is passed to the decoder part. The decoder restores the spatial resolution of the feature map through transposed convolution so that the output mask map is consistent with the original image size. While restoring the spatial dimension, each layer of the decoder fuses the corresponding encoder feature map into the decoding process through jump connection to achieve multi-scale fusion of features. By training the U-Net network through supervised learning, the mask map of the target area of ​​the decorative plate is output in a semantic segmentation manner. The mask map accurately marks the connection between the arc plate and the placement plate. Key parts such as the push component installation surface, the periphery of the lock slot, the meshing area between the card block and the card slot, and the foam adhesive attachment area. The morphological features and texture features are calculated based on the mask map of the target area of ​​the decorative panel to construct a primary feature set. In the calculation of morphological features, by analyzing the connected areas in the mask map, features such as area, perimeter, aspect ratio, and circularity are extracted. For example, the calculation formula for circularity is:

[0027] in, Indicates circularity, is the area of ​​the region, is the perimeter of the region. The closer it is to 1, the closer the shape of the area is to a circle, which is particularly effective when detecting foam adhesive attachment areas, because foam adhesive is usually distributed in circular or regular geometric shapes on the surface of the decorative panel. For the extraction of texture features, the local binary pattern operator is used. This method generates binary patterns by comparing the grayscale values ​​of pixels with those of surrounding pixels. These patterns can effectively reflect the texture information of the region, such as identifying wear or irregular texture changes on the installation surface of the push component by calculating the entropy of the texture. After the primary feature set is completed, edge features and color features are extracted according to the mask map of the target area of ​​the decorative panel to obtain the intermediate feature set. In the process of edge feature extraction, the improved Canny operator is applied to perform edge detection operations on the target area, and the structural characteristics of the region are evaluated by calculating indicators such as edge density, directionality, and continuity of edge length. For example, the entropy of edge direction distribution is calculated by the following formula:

[0028] in, represents directional entropy, is the first The probability distribution of directions, is the total number of directions. When the directional entropy is low, it means that the edge lines are relatively regular. The color feature is extracted by converting the image to the HSV color space and calculating the statistical information of the hue, saturation and brightness of the pixels in the area, including the mean, variance, skewness and kurtosis. The multi-dimensional color features can identify the color consistency of the surface materials of the decorative board, and judge whether there are problems such as adhesive residue or color pollution at the connection between the curved board and the placement board through the color change pattern. The deep features are extracted from the mask map of the target area of ​​the decorative board to obtain the target feature vector. In the deep feature extraction process, the pre-trained EfficientNet-B0 model is used to input the mask area image into the deep learning network, and a 1280-dimensional high-dimensional feature vector is extracted through the convolutional neural network. These feature vectors contain the spatial and texture information of the image and integrate the abstract semantic features learned by the deep model on a large-scale data set. For example, through the activation pattern in the deep features, small defects in complex scenes, such as fine cracks around the lock slot or assembly errors in the locking area, are identified. Feature selection is performed on the primary feature set, the intermediate feature set and the target feature vector to obtain an optimized feature vector set. In the feature selection process, the random forest algorithm is used to calculate the contribution of each feature to the identification of decorative board defects. By analyzing the importance score of the feature, redundant features with low contribution are eliminated, and only the most discriminating features are retained. For example, the Gini coefficient or information gain method is used to quantitatively evaluate the importance of each feature in the classification task, so that the feature vector set is both comprehensive and efficient.

[0029] In a specific embodiment, the process of executing step S103 may specifically include the following steps: Construct a decorative panel defect classification network based on ResNet-v1-50. The decorative panel defect classification network includes a feature extraction convolution layer, a global average pooling layer, two fully connected layers, and a Softmax classification layer. The standardized image data is extracted by region to obtain a first image data set, a second image data set and a third image data set, wherein the first image data set contains a complete decorative panel image, the second image data set focuses on an image of a region where the curved panel and the placement panel are connected, and the third image data set focuses on an image of a region where the push component is pushed; Corresponding the feature vector set to the first graph data set, the second graph data set, and the third graph data set, respectively, to obtain model training data; The first stage of training was performed on the decorative panel defect classification network. The parameters of the convolutional blocks in the feature extraction convolutional layer were locked, and only the parameters of the two fully connected layers were optimized to obtain a classification model that was initially adapted to the decorative panel data distribution. Perform the second phase of training on the classification model that initially adapts to the data distribution of decorative panels, unlock all network layer parameters, perform full network fine-tuning, and obtain an optimized classification model; Based on the optimized classification model, reasoning is performed on the test parts of the first image data set, the second image data set, and the third image data set to obtain a defect classification model; The gradient-weighted class activation map is calculated according to the defect classification model to obtain the candidate defect region mask.

[0030] Specifically, an efficient classification network architecture is designed. The network uses ResNet-v1-50 as the basis for feature extraction. By introducing feature extraction convolutional layers, global average pooling layers, two fully connected layers, and Softmax classification layers, a deep learning model that can accurately identify defects in automotive decorative panels is constructed. In the network structure, the feature extraction convolutional layer of ResNet-v1-50 is composed of 50 convolutional layers with residual connections. Each residual module realizes the addition operation of the previous and next feature maps through skip connections, thereby effectively solving the gradient vanishing problem in deep networks. In the feature extraction process, multi-scale features in the decorative panel image are extracted through convolution operations, and the model's perception of high semantic information in the image is gradually improved through feature map stacking and downsampling operations. The global average pooling layer is used to compress the feature map from the spatial dimension (such as 7×7×2048) to a one-dimensional vector (such as 2048 dimensions). This operation reduces the number of model parameters and improves the model's robustness to overfitting. In the fully connected layer design, the model adopts a two-layer structure, reducing the 2048-dimensional feature vector to 1024 dimensions through linear transformation, and then further reducing it to 256 dimensions, and outputting the probability distribution of the two categories (good / defective) through the Softmax classification layer. The calculation formula of the Softmax classification layer is:

[0031] in, Represents the input image Belongs to the target defect category The probability of is the category of the fully connected layer output The corresponding activation value, is the total number of categories (2 in this task, representing good products and defects), and e is the index parameter in the summation formula. The real activation value output by the model is converted into a probability distribution between 0 and 1 through the Softmax operation. After the classification network is constructed, the standardized image data is extracted by region, and multi-scale image analysis is achieved by dividing the original data set into the first image data set. The second image data set and the third image data set. Among them, the first image data set contains images of complete decorative panels for coarse-grained evaluation of overall quality, the second image data set focuses on the connection between the curved plate and the placement plate, which is a key area where stress concentration and assembly errors are prone to occur, and the third image data set focuses on the push component area to capture subtle defects such as wear or poor engagement on the component surface with higher resolution. In the regional image data set, by corresponding the feature vector set to each data set one by one, the model is trained specifically according to the data characteristics of images of different scales, thereby maintaining a high defect detection accuracy at both global and local scales. After completing the data preparation, the first stage of training is performed on the decorative panel defect classification network. The parameters of the convolution blocks in the feature extraction convolution layer are locked, and only the parameters of the two fully connected layers are optimized. This operation freezes the weights of the first four convolution modules in ResNet-v1-50 and only adjusts the parameters of the newly added fully connected layer, so that the model can quickly adapt to the distribution characteristics of the decorative panel image data while maintaining the original feature extraction capabilities. In the first stage of training, the model uses the cross entropy loss function, which is calculated as follows:

[0032] in, represents the loss value, is the one-hot encoding of the actual label, is the model prediction category The probability of the model being defective is 0.001, Q is the number of samples. This loss function guides the update direction of the model parameters by measuring the distance between the model's predicted probability distribution and the actual label, so that the fully connected layer gradually learns the characteristic patterns related to defects in the decorative panel image. In the specific training, the random mini-batch gradient descent algorithm is adopted. By setting a large initial learning rate (such as 0.001), rapid convergence is achieved in the early training stage of the model, and the learning rate decay strategy is used to enable the model to update parameters at a more stable speed when approaching the optimal solution. After completing the first stage of training and obtaining a classification model that is initially adapted to the data distribution of the decorative panel, the second stage of training is entered. At this time, all network layer parameters are unlocked and the entire model is fine-tuned to obtain an optimized classification model. In this stage, the model no longer only adjusts the weights of the fully connected layer, but allows all convolutional layer parameters to participate in the update through a smaller learning rate (such as 0.0001), so that the model feature extraction part can focus more on the detailed features in the decorative panel image, such as the fine cracks on the edge of the curved plate or the assembly deviation in the meshing area between the card block and the card slot. In order to prevent overfitting, the model introduces regularization techniques (such as Dropout and weight decay) during the training process. By randomly discarding some neurons in the fully connected layer (for example, the Dropout ratio is 0.5), the model pays more attention to global features rather than relying on a single feature. The training termination condition of the model is set to automatically stop training when the accuracy of the validation set has not improved for five consecutive cycles to avoid overfitting of the model on the training data. After obtaining the optimized classification model, the test parts of the first image data set, the second image data set, and the third image data set are inferred based on the model. The classification output of the model is used to determine whether the decorative panel image has defects, and the specific defect category prediction results are output. In the inference stage, more detailed defect location information is obtained by analyzing the classification score distribution of the model at each pixel. In addition, in order to achieve visualization and precise positioning of the defect area, the gradient weighted class activation map (Grad-CAM) is calculated based on the optimized classification model to generate a mask for the candidate defect area. In the Grad-CAM calculation process, the target score function is defined , which indicates that the model predicts that the image belongs to the defect category, and then calculates the score for the last convolutional layer feature map of the model The gradient is:

[0033] in, is the feature map channel The weight of is the spatial size of the feature map, Represents the target score for the feature map at coordinates The gradient of In image coordinates The convolution feature map of . A two-dimensional heat map is generated by multiplying the weights and the feature map pixel by pixel and summing them up. The negative values ​​are reset to zero through the ReLU activation function to highlight the defect area that the model pays most attention to, and the candidate defect area mask is obtained.

[0034] In a specific embodiment, the step of calculating the gradient weighted class activation map according to the defect classification model to obtain the candidate defect area mask may specifically include the following steps: A target score function is defined according to the defect category in the defect classification model, and a gradient value of a convolution kernel corresponding to the convolution layer feature map in the defect classification model is calculated to obtain target gradient data. Perform a global average pooling operation on the target gradient data in the spatial dimension to obtain the feature map channel importance weights; Multiply the feature map channel importance weight with the corresponding feature map and sum it in the channel dimension to get the initial heat map; Based on the intermediate layer feature map of the defect classification model, a multi-scale thermal map is generated, and the initial thermal map and the multi-scale thermal map are weighted fused according to the preset weight coefficient to obtain an optimized thermal map; Morphological opening and closing operations are performed on the optimized heat map to obtain candidate defect area masks.

[0035] Specifically, a target score function is defined for a specific defect class in the defect classification model, which is used to quantify the confidence of the model in predicting that an image belongs to a specific defect class. Defined as the target defect category in the Softmax classification layer For example, for a binary classification model (good / defective), if the model predicts that the image is defective, the probability is , then the target score function is expressed as:

[0036] in, It is the target defect category output by the model in the fully connected layer The activation value of is the total number of classification categories, and b is the index parameter in the summation formula. (For example, the "defect" category), only focus on the response of this category when calculating the gradient. After defining the target score function, calculate the gradient value of the convolution kernel corresponding to the convolution layer feature map in the model to obtain the target gradient data. In the gradient calculation process, select the feature map of the last convolution layer of the model As the object of gradient calculation, the mathematical expression of the gradient is:

[0037] in, represents the target score function For feature maps In Location By calculating this gradient, the contribution of each pixel in the feature map to the model's prediction of the defect category is quantified. The larger the gradient value, the more critical the pixel is in the classification decision. This process is implemented in conjunction with the back-propagation algorithm, which transfers the error from the output layer to the target convolution layer through the chain rule to calculate the gradient values ​​of all pixels. After obtaining the target gradient data, a global average pooling operation is performed on these gradient data in the spatial dimension to calculate the importance weights of the feature map channels. In a convolutional neural network, a feature map has multiple channels (e.g., a size of ,in and are the height and width of the feature map, respectively. is the number of channels), global average pooling reduces the gradient value of each channel to a scalar weight by averaging the feature map in the spatial dimension , and its calculation formula is:

[0038] In this formula, Represents feature map channels The importance weight of is the gradient value, and are the height and width of the feature map, i and j are the index numbers of the corresponding summation formula. Through the global average pooling operation, the weight of each channel is proportional to its importance in predicting the target category. This method can automatically learn the channel features that the model pays attention to during the decision-making process. Multiply the importance weight of the feature map channel by the corresponding feature map pixel by pixel and sum it in the channel dimension to generate the initial heat map. Initial heat map The calculation formula is:

[0039] in, Indicates that the heat map is at position The value at It is a channel The weight of It is a feature map In Location The pixel value at , the ReLU activation function is used to filter out values ​​less than 0, and K is the number of channels. Ensure that the heat map only retains areas that contribute positively to the target category. In the generated initial heat map, the highlighted area represents the image part that contributes more to the target defect category in the model prediction. This method intuitively displays the model's focus area on the image, such as the potential defect areas at the connection between the curved plate and the placement plate or in the push component area in the decorative plate image. In order to improve the accuracy of the heat map, a multi-scale heat map is generated based on the intermediate layer feature map of the defect classification model, and the initial heat map and the multi-scale heat map are weighted fused according to the preset weight coefficient to obtain an optimized heat map. Select the feature maps of the intermediate layer of the model (such as the res3d_branch2c and res4f_branch2c layers of ResNet), and calculate the Grad-CAM heat maps of these feature maps respectively. These heat maps represent the model's focus areas on different scale features. For example, when detecting the meshing area between the card block and the card slot of the car decorative panel, the shallower feature map focuses on the local edge features, while the deeper feature map pays more attention to the overall texture and morphological pattern. (For example, [0.2, 0.3, 0.5]), weighted fusion of heat maps at different levels is calculated as follows:

[0040] in, Is to optimize the heat map in the location The pixel value at It is The weight coefficient of the layer heat map, It is The value of the heat map at the corresponding position, is the number of feature maps involved in the fusion. Through the multi-scale weighted fusion method, the information of the model at different feature scales is effectively integrated, so that the generated optimized heat map retains global information and has high sensitivity to detail areas. Perform morphological opening and closing operations on the optimized heat map to obtain the candidate defect area mask. In the image processing process, the opening operation is used to eliminate small noise points. The isolated highlights in the heat map are removed by the operation of first corrosion and then expansion, while the closing operation fills the small holes in the heat map by the operation of first expansion and then corrosion, making the morphology of the defect area more complete and coherent.

[0041] In a specific embodiment, the process of executing step S104 may specifically include the following steps: Perform multi-scale hierarchical analysis based on the candidate defect region mask to obtain a first scale level region, a second scale level region, and a third scale level region; Perform coarse-grained defect detection on the first scale level area to obtain preliminary defect judgment results; Based on the preliminary defect judgment results, a refined analysis is performed on the second scale level area to obtain the defect location result at the connection between the curved plate and the placement plate; According to the defect location result at the connection between the curved plate and the placement plate, the defect location is performed on the third scale level area to obtain the target defect location data; The automobile decorative panel is divided into multiple grids, and the preliminary defect judgment results, the defect location results at the connection between the curved panel and the placement panel, and the cumulative value of the historical defect heat map of the target defect location data in each grid are counted to obtain the defect frequency distribution data; Standardize the defect frequency distribution data and divide the risk levels to obtain the defect risk heat map; The high-risk area is determined according to the defect risk heat map, and the connection area between the card block and the card slot in the high-risk area is analyzed for the card quality to obtain the card quality assessment result.

[0042] Specifically, a multi-scale hierarchical analysis is performed on the candidate defect area mask, and a progressive analysis method from the whole to the part is realized by decomposing the image into regions of different scales. The decorative panel image is divided into the first scale level region, the second scale level region and the third scale level region, and these three levels correspond to different detection accuracies and target areas. For example, the first scale level region covers the entire decorative panel image and is used for coarse-grained global analysis to quickly screen out the significant defect areas of the decorative panel, while the second scale level region focuses on the connection between the curved plate and the placement plate, which is a part where mechanical stress is concentrated and material deformation or assembly errors are prone to occur. The third scale level region further narrows the analysis scope and focuses on the connection area between the push component and the card block and the card slot. Through the higher resolution image input model, the system can identify subtle defects on the surface of the connection area, such as slight surface scratches, tiny cracks at the connection or material wear marks. Coarse-grained defect detection is performed on the first scale level region to obtain preliminary defect judgment results. The defect classification model is used to quickly scan the entire decorative panel image, and combined with the highlight area of ​​the candidate defect area mask, a statistical analysis method based on pixel activation value is used to generate a preliminary defect heat map. In the mathematical model, the calculation formula of the preliminary defect heat map is expressed as:

[0043] in, Indicates the preliminary heat map at location The pixel value at is the number of defect masks involved in the analysis, It is Defect masks at position The binary value at (1 means there is a defect, 0 means there is no defect), is the probability value of the corresponding defect predicted by the model. In this way, the defective area is identified globally, especially in the complex texture background of the decorative panel as a whole, and the high-risk area of ​​the defect can be quickly located. Based on the preliminary defect judgment results, a more refined analysis is performed on the second scale level area to obtain the specific defect location results at the connection between the curved plate and the placement plate. In the refined analysis process, the model adopts a multi-scale feature fusion method to combine the preliminary heat map with the multi-scale feature map, and by setting a higher detection threshold, the noise area is filtered out, and only the pixels with high confidence in the defect category are retained. For example, by introducing morphological analysis technology, the geometric features of the defect area (such as area, perimeter, shape factor, etc.) are calculated to accurately depict the defect contour at the connection between the curved plate and the placement plate. The calculation formula used in analyzing geometric features is similar to the connected area in the analysis mask map mentioned above, and will not be repeated here. For example, when detecting the connection between the curved plate and the placement plate of the decorative plate, the model can accurately locate the material cracking or coating peeling caused by stress concentration during the assembly process by analyzing these shape features. Based on the defect location results at the connection between the arc plate and the placement plate, the analysis scope is further narrowed, and the focus is shifted to the third scale level area. The push component and the connection area between the card block and the card slot are finely analyzed for defect location to obtain the target defect location data. At this stage, the image is analyzed at the pixel level by combining high-resolution image data with a deep learning model. For example, through the activation pattern in the feature map of the convolutional neural network, the model identifies tiny cracks, metal fatigue marks or material aging in the connection area. At the same time, by introducing edge detection operators (such as the Canny operator) to calculate the continuity and linearity of the edge of the connection area, it can also assist in judging whether the mechanical engagement state between the card block and the card slot is good, such as whether there is a problem of card block deformation or card slot wear. The multi-level progressive analysis method ensures that the model can maintain high detection accuracy at both global and local scales, especially in complex decorative panel structures. Excellent defect recognition ability. After obtaining the target defect location data, the automotive decorative panel is divided into multiple grids, and the defect frequency distribution data is obtained by counting the cumulative value of the historical defect heat map in each grid. For example, by dividing the decorative panel image into grids, record the frequency of defects in each grid in different time periods, and calculate the cumulative value of the historical heat map :

[0044] in, is the grid position The cumulative defect frequency at is the time cycle, It is Period heat map at location The value at. Through this step, the probability distribution of defects in different areas of the decorative panel is effectively quantified. The defect frequency distribution data is standardized and divided into risk levels to generate a defect risk heat map. In the standardization process, the cumulative value is normalized to a range of 0 to 1, and according to the preset risk level standard (for example, 0 to 0.2 is low risk, 0.2 to 0.4 is medium risk, 0.4 to 0.6 is high risk, 0.6 to 0.8 is extremely high risk, and 0.8 to 1.0 is the highest risk), each grid is assigned a different color code, for example, blue represents low risk and red represents high risk. After determining the high-risk area, the card block and the card slot connection area in these areas are analyzed for the card quality. By analyzing the texture features, edge features and light reflection characteristics of the surface of the connection area, the quality of the card quality is accurately evaluated. For example, the extracted surface features are input into the engagement status classification model through the support vector machine algorithm, and the engagement quality is divided into four levels: excellent, good, fair, and poor. Combined with the high-risk areas in the risk heat map, the analysis results are compared with the historical defect data to determine whether there are abnormal process problems in the current production process.

[0045] In a specific embodiment, the execution step determines the high-risk area according to the defect risk heat map, and performs a card-fitting quality analysis on the connection area between the card block and the card slot in the high-risk area. The process of obtaining the card-fitting quality evaluation result may specifically include the following steps: According to the defect risk heat map, high-risk areas with risk values ​​greater than the preset target value are screened. The high-risk areas include the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area; Extract the connection area between the card block and the card slot in the high-risk area to obtain a surface image of the connection area, and perform multi-feature extraction on the surface image of the connection area to obtain a surface morphology feature set; A mapping relationship model between engagement state and surface morphology is constructed. The mapping relationship model describes the characteristics of regular texture distribution and edge lines on the surface in the fully engaged state, and the characteristics of irregular light reflection patterns caused by deformation on the surface in the insufficiently engaged state. The engagement quality analysis is performed based on the surface morphology feature set and the mapping relationship model to obtain the engagement quality evaluation result.

[0046] Specifically, high-risk areas with risk values ​​greater than the preset target value are screened out from the defect risk heat map. The defect risk heat map is generated based on historical defect frequency distribution data by dividing the decorative panel image into multiple grids and counting the frequency distribution of defects in each grid. In the risk screening stage, a risk threshold is set. ,For example , the risk value The area greater than the threshold is marked as a high-risk area, which is mathematically expressed as:

[0047] in, Indicates that the high risk mask is at location The state of the risk value is 1, which indicates a high-risk area and 0, which indicates a safe area. It is the accumulated defect frequency data after standardization. For example, high-risk areas are more likely to appear in the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area. These parts have complex structures, concentrated mechanical stress, and easy wear of materials. Through the binary mask map based on the risk value, the high-risk areas that need to be focused on are accurately screened out in the entire decorative board image. After determining the high-risk areas, the image of the connection area between the card block and the card slot in these areas is extracted to obtain the surface image of the connection area. In the image extraction process, by multiplying the high-risk mask map with the original image at the pixel level, only the pixel values ​​in the high-risk area are retained, and the other parts are set to zero, so as to separate the connection area between the card block and the card slot from the complex decorative board background. This method can ensure that only high-risk areas are calculated in subsequent analysis, effectively reduce the consumption of computing resources, and avoid noise interference in non-related areas. After obtaining the surface image of the connection area, multi-feature extraction is performed on these images to construct a surface morphology feature set. In the feature extraction process, attention is paid to the geometric morphology, texture features, edge features and light reflection characteristics of the surface. For example, by calculating the shape features of the surface, the geometric features of the connection area can be effectively quantified, which helps to identify whether the card block is deformed and whether there is mechanical wear in the connection area. In terms of texture feature extraction, the grayscale distribution features of the surface pixels, such as energy, contrast, correlation and entropy, are calculated by the local binary pattern method. These features can reflect the material consistency of the surface and identify potential defects in the connection area through the texture change pattern, such as material microcracks caused by mechanical stress or surface wear caused by long-term friction. The edge detection of the connection area image is performed by the improved Cannv operator, and the density, directionality and continuity of the edge are calculated, which helps to evaluate the mechanical engagement state of the card block and the card slot. For example, in the fully engaged state, the edge lines of the connection area should appear as regular parallel lines or closed boundaries, while in the case of insufficient engagement or poor assembly, the edge features show discontinuous, irregular or broken line features. After completing the construction of the surface morphology feature set, a mapping relationship model between the engagement state and the surface morphology is established. The model associates the surface features with the actual engagement quality state to achieve quantitative quality evaluation. In the mapping relationship model, a large amount of sample data of known engagement states is collected and a classification algorithm such as support vector machine or random forest is used to establish a mathematical model between the surface morphological features and the engagement state. For example, in the fully engaged state, the edge density of the connection area is and texture energy Usually high, while the skewness of the light reflection pattern Close to zero, it means that the light is evenly distributed without obvious highlights or shadows. The mathematical expression of the model is through the classification decision function It is expressed as:

[0048] in, is the engagement state prediction function, , , is the feature weight of the model, is the bias value, is the input surface morphology feature vector. By training the model, the prediction function The corresponding quality level can be output in different engagement states, such as excellent, good, fair, poor, etc. In the model reasoning process, the score of the engagement quality is calculated by inputting the surface morphological features in the high-risk area into the mapping relationship model, and the engagement quality is divided into different levels according to the score range. For example, when When the When is good, It is considered normal, while if it is lower than 0.4, it is considered as a bad engagement state. After obtaining the engagement quality assessment results, combined with the high-risk area data in the risk heat map, the frequency distribution of historical defects is analyzed to determine whether the quality status of the current production process is stable. For example, in a specific grid in the connection area, if it is in a high-risk state for a long time, it is necessary to adjust the production equipment, process parameters or raw materials to reduce potential quality risks. And by comparing the current engagement quality assessment results with historical data, potential problems in the production line can be identified, such as whether there is a consistent engagement deviation in a specific device, or whether periodic assembly errors have occurred in a specific process.

[0049] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0050] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable an intelligent detection device for the processing of automobile decorative panels (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0051] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent detection method for automobile decorative panel processing, characterized in that: The method comprises: Carry out high-resolution image acquisition and preprocessing of automobile decorative panels to obtain standardized image data; Performing a region segmentation operation on the standardized image data to obtain a decorative plate target region mask map, and extracting multimodal features from the decorative plate target region mask map to obtain a feature vector set; Inputting the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculating a candidate defect area mask through the defect classification model; A multi-scale hierarchical analysis is performed on the candidate defect area mask to obtain a defect risk heat map, and a card-fitting quality analysis is performed on the connection area between the card block and the card slot according to the defect risk heat map to obtain a card-fitting quality assessment result.

2. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The high-resolution image acquisition and preprocessing of the automobile decorative panel to obtain standardized image data includes: Using an industrial camera to collect images of automobile decorative panels to obtain an original decorative panel image, and performing size standardization processing on the original decorative panel image to obtain a standardized size image; Performing histogram equalization processing on the standardized size image to obtain a contrast enhanced image, and performing non-local mean filtering on the contrast enhanced image to obtain a denoised image; Performing adaptive threshold segmentation on the denoised image to obtain a binary mask image of the decorative plate, and performing a background removal operation on the denoised image using the binary mask image of the decorative plate to obtain a pure decorative plate image; Image enhancement processing is performed on the pure decorative plate image to obtain standardized image data.

3. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The performing of a region segmentation operation on the standardized image data to obtain a decorative plate target region mask map, and extracting multimodal features from the decorative plate target region mask map to obtain a feature vector set, includes: The standardized image data is input into a U-Net network for feature extraction to obtain a multi-scale feature map, wherein the U-Net network comprises a four-layer encoder and a four-layer decoder, each layer of the encoder uses a ResNet module with a hole convolution to extract features, and each layer of the decoder restores the spatial dimension of the feature map by transposed convolution; Performing network training based on the multi-scale feature map to obtain a region segmentation model; The standardized image data is segmented using the region segmentation model to obtain a mask image of a target region of the decorative plate, wherein the target region of the decorative plate includes a connection between the arc plate and the placement plate, a mounting surface of the push assembly, a periphery of the lock slot, an engagement area between the card block and the card slot, and an area where the foam adhesive is attached; Calculating morphological features and texture features based on the decorative plate target area mask map to obtain a primary feature set, and extracting edge features and color features based on the decorative plate target area mask map to obtain an intermediate feature set; Extracting deep features from the decorative panel target area mask map to obtain a target feature vector, and performing feature selection on the primary feature set, the intermediate feature set and the target feature vector to obtain a feature vector set.

4. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The step of inputting the feature vector set into a transfer learning network based on ResNet-v1-50 for training to obtain a defect classification model, and calculating a candidate defect area mask through the defect classification model includes: Constructing a decorative panel defect classification network based on ResNet-v1-50, wherein the decorative panel defect classification network includes a feature extraction convolution layer, a global average pooling layer, two fully connected layers and a Softmax classification layer; The standardized image data is extracted by region to obtain a first image data set, a second image data set and a third image data set, wherein the first image data set includes an image of a complete decorative plate, the second image data set focuses on an image of an area where the curved plate and the placement plate are connected, and the third image data set focuses on an image of an area where the pushing component is pushed; Matching the feature vector set with the first graph data set, the second graph data set and the third graph data set respectively to obtain model training data; Performing the first stage training on the decorative panel defect classification network, locking the convolution block parameters in the feature extraction convolution layer, and only optimizing the parameters of the two fully connected layers to obtain a classification model that is preliminarily adapted to the decorative panel data distribution; Performing the second phase of training on the classification model that is initially adapted to the data distribution of the decorative panels, unlocking all network layer parameters to perform full network fine-tuning, and obtaining an optimized classification model; Based on the optimized classification model, reasoning is performed on the test parts of the first image data set, the second image data set, and the third image data set respectively to obtain a defect classification model; A gradient weighted class activation map is calculated according to the defect classification model to obtain a candidate defect region mask.

5. The intelligent detection method for automobile decorative panel processing according to claim 4 is characterized in that: The step of calculating the gradient weighted class activation map according to the defect classification model to obtain a candidate defect region mask includes: A target score function is defined according to the defect category in the defect classification model, and a gradient value of a convolution kernel corresponding to a convolution layer feature map in the defect classification model and the target score function is calculated to obtain target gradient data; Performing a global average pooling operation on the target gradient data in the spatial dimension to obtain feature map channel importance weights; Multiply the feature map channel importance weight by the corresponding feature map and sum them in the channel dimension to obtain an initial heat map; Based on the intermediate layer feature map of the defect classification model, a multi-scale thermal map is generated, and the initial thermal map and the multi-scale thermal map are weightedly fused according to a preset weight coefficient to obtain an optimized thermal map; Morphological opening and closing operations are performed on the optimized heat map to obtain a candidate defect region mask.

6. The intelligent detection method for automobile decorative panel processing according to claim 1 is characterized in that: The multi-scale hierarchical analysis is performed on the candidate defect area mask to obtain a defect risk heat map, and the connection area of ​​the card block and the card slot is analyzed for the card quality according to the defect risk heat map to obtain a card quality assessment result, including: Performing multi-scale hierarchical analysis based on the candidate defect region mask to obtain a first scale hierarchical region, a second scale hierarchical region, and a third scale hierarchical region; Performing coarse-grained defect detection on the first scale level area to obtain a preliminary defect judgment result; Based on the preliminary defect judgment result, a refined analysis is performed on the second scale level area to obtain a defect location result at the connection between the curved plate and the placement plate; According to the defect location result of the connection between the arc plate and the placement plate, the defect location is performed on the third scale level area to obtain the target defect location data; Divide the automobile decorative panel into a plurality of grids, and count the preliminary defect judgment results, the defect location results at the connection between the curved panel and the placement panel, and the cumulative value of the historical defect heat map of the target defect location data in each grid to obtain defect frequency distribution data; Standardizing the defect frequency distribution data and dividing the risk levels to obtain a defect risk heat map; A high-risk area is determined according to the defect risk heat map, and a clamping quality analysis is performed on a connection area between the clamping block and the clamping slot in the high-risk area to obtain a clamping quality evaluation result.

7. The intelligent detection method for automobile decorative panel processing according to claim 6 is characterized in that: The step of determining a high-risk area according to the defect risk heat map, and performing a card-fitting quality analysis on a connection area between a card block and a card slot in the high-risk area to obtain a card-fitting quality assessment result includes: According to the defect risk heat map, high-risk areas with risk values ​​greater than a preset target value are screened, wherein the high-risk areas include the connection between the push block and the rotating groove, the edge of the connection between the arc plate and the placement plate, the corner of the cavity structure, and the edge of the foam adhesive attachment area; Extracting the connection area between the card block and the card slot in the high-risk area to obtain a surface image of the connection area, and performing multi-feature extraction on the surface image of the connection area to obtain a surface morphology feature set; Constructing a mapping relationship model between engagement state and surface morphology, wherein the mapping relationship model describes the characteristics of regular texture distribution and edge lines on the surface in a fully engaged state, and the characteristics of irregular light reflection patterns caused by deformation of the surface in an insufficiently engaged state; The engagement quality analysis is performed based on the surface morphology feature set and the mapping relationship model to obtain an engagement quality evaluation result.

Images