An image processing-based quality inspection system and method for automotive parts

Abstract

This application provides an image processing-based quality inspection system and method for automotive parts. It involves real-time acquisition of multi-dimensional information fusion sets of key surfaces of automotive parts; determining the feature mining granularity based on two-dimensional part images; mining key feature points based on the feature mining granularity to obtain a key feature point domain; using the key feature point domain for similarity detection to obtain the detection distortion features of the key surfaces of the automotive parts; determining the crack texture features of the key surfaces of the automotive parts using an illumination-equalized grayscale image; generating multi-scale feature vectors of the key surfaces of the automotive parts through three-dimensional registration point cloud data; and performing quality inspection on the key surfaces of the automotive parts based on the detection distortion features, crack texture features, and multi-scale feature vectors to obtain the inspection results. The technical solution provided in this application can effectively extract multi-scale features of the key surfaces of automotive parts for quality inspection, thereby improving the detection accuracy of the inspection system.

Description

Technical Field

[0001] This application relates to the field of quality inspection technology, and more specifically, to an image processing-based quality inspection system and method for automotive parts. Background Technology

[0002] Image processing-based quality inspection of automotive parts is a core technology in modern intelligent manufacturing. It integrates advanced technologies such as optical imaging, digital image processing, pattern recognition, and machine learning to achieve high-speed, high-precision automated inspection of quality characteristics such as part dimensions, appearance defects, and assembly correctness. Its core objective is to identify and determine surface defects, geometric deviations, and material anomalies in parts through non-contact, rapid, and high-precision methods, thereby replacing traditional manual visual inspection or contact-based inspection and improving production efficiency and product consistency.

[0003] However, existing technologies present a wide variety of defects in automotive parts, posing a significant challenge. Minor scratches may differ from normal machining textures only in subtle grayscale differences; quantitative detection of soft deformations (such as warping in plastic parts) is difficult; and the location, shape, size, and contrast of defects are highly random and unpredictable. Traditional rule-based image processing algorithms (such as fixed threshold segmentation and template matching) have limited adaptability to this variability, easily leading to missed or false detections. Therefore, effectively extracting multi-scale features from key surfaces of automotive parts for quality inspection, thereby improving the accuracy of inspection systems, remains a major challenge for the industry. Summary of the Invention

[0004] This application provides an image processing-based automotive parts quality inspection system and method, which can effectively extract multi-scale features of key surfaces of automotive parts for quality inspection, thereby improving the inspection accuracy of the inspection system.

[0005] In a first aspect, this application provides a method for quality inspection of automotive parts based on image processing, comprising the following steps:

[0006] Real-time collection of multi-dimensional information fusion set of key aspects of automotive parts during the automotive parts production process;

[0007] The feature mining granularity in the image feature mining process is determined based on the two-dimensional part image in the multi-dimensional information fusion set. Key feature points are mined on the two-dimensional part image based on the feature mining granularity to obtain the key feature point domain of the key surface of the automotive parts. The key feature point domain is used for similarity detection to obtain the detection differentiation features of the key surface of the automotive parts.

[0008] The crack texture features of the key surfaces of the automotive parts are determined by using the illumination equalization grayscale image of the two-dimensional part image, and multi-scale feature vectors of the key surfaces of the automotive parts are generated by using the three-dimensional registration point cloud data of the multi-dimensional information fusion set.

[0009] The key surfaces of automotive parts are inspected based on the detected anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of automotive parts.

[0010] In some embodiments, the real-time acquisition of a multi-dimensional information fusion set of key aspects of automotive parts during the automotive parts production process specifically includes:

[0011] Two-dimensional images of key surfaces of automotive parts are captured in real time using industrial cameras during the production process.

[0012] Based on the simultaneous acquisition of two-dimensional part images of key surfaces of automotive parts and three-dimensional point clouds at the same time using 3D sensors, the three-dimensional point clouds are then denoised and registered to obtain three-dimensional registered point cloud data of key surfaces of automotive parts.

[0013] A multi-dimensional information fusion set of key surfaces of automotive parts during the automotive parts production process is constructed based on the two-dimensional part images and the three-dimensional registration point cloud data.

[0014] In some embodiments, determining the feature mining granularity in the image feature mining process based on the two-dimensional part image in the multidimensional information fusion set specifically includes:

[0015] Obtain the pre-set image scaling parameters;

[0016] Extract feature parameters from the two-dimensional part images in the multi-dimensional information fusion set;

[0017] The feature mining granularity in the image feature mining process is determined based on the image scaling parameters and the feature parameters.

[0018] In some embodiments, the key feature point domain of the key surface of the automotive component is obtained by mining key feature points from the two-dimensional part image based on the feature mining granularity, specifically including:

[0019] Based on the aforementioned feature mining granularity, an image mining scale model for the two-dimensional part image is constructed;

[0020] Construct a scale feature coverage map corresponding to each scale level in the image mining scale model;

[0021] For each scale feature coverage map, determine the feature aggregation coefficient of each pixel in the scale feature coverage map;

[0022] The scale feature coverage map is updated based on the feature aggregation coefficients of all pixels, thereby obtaining the coverage map update map corresponding to each scale feature coverage map.

[0023] Based on all the overlay maps, update the map to construct the key feature point domains of the critical surfaces of automotive components.

[0024] In some embodiments, determining the crack texture features of key surfaces of automotive parts using the illumination-equalized grayscale image of the two-dimensional part image specifically includes:

[0025] The two-dimensional part image is subjected to illumination compensation and texture enhancement to obtain an illumination-balanced grayscale image of the two-dimensional part image.

[0026] The texture features of the crack region in the illumination-equalized grayscale image are determined based on the grayscale co-occurrence matrix;

[0027] The texture features are vectorized to obtain the crack texture features of the key surfaces of automotive parts.

[0028] In some embodiments, generating multi-scale feature vectors for key surfaces of automotive components from the three-dimensional registration point cloud data in the multi-dimensional information fusion set specifically includes:

[0029] Multi-scale feature extraction is performed on the three-dimensional registration point cloud data in the multi-dimensional information fusion set to obtain the surface geometric descriptor, statistical features and local shape descriptor of the key surfaces of automotive parts;

[0030] Multi-scale feature vectors of key surfaces of automotive parts are generated based on the surface geometry descriptor, the statistical features, and the local shape descriptor.

[0031] In some embodiments, the quality inspection of key surfaces of automotive parts is performed based on the detected anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of automotive parts, specifically including:

[0032] A lightweight deep neural network is used for supervised learning to establish a defect type identification and quality level determination model;

[0033] The detection distortion features, crack texture features, and multi-scale feature vectors are input into the defect type identification and quality level determination model to perform quality inspection on the key surfaces of automotive parts, thereby obtaining the inspection results of the key surfaces of automotive parts.

[0034] Secondly, this application provides an image processing-based automotive parts quality inspection system for performing an image processing-based automotive parts quality inspection method, including:

[0035] The information acquisition module is used to collect multi-dimensional information fusion sets of key aspects of automotive parts in real time during the production process of automotive parts;

[0036] The alienation detection module is used to determine the feature mining granularity in the image feature mining process based on the two-dimensional part image in the multi-dimensional information fusion set, to mine key feature points of the two-dimensional part image based on the feature mining granularity, and to obtain the key feature point domain of the key surface of the automotive parts. The key feature point domain is used to perform similarity detection, and to obtain the detected alienation features of the key surface of the automotive parts.

[0037] The feature extraction module is used to determine the crack texture features of the key surfaces of the automotive parts using the illumination equalization grayscale image of the two-dimensional part image, and to generate multi-scale feature vectors of the key surfaces of the automotive parts through the three-dimensional registration point cloud data of the multi-dimensional information fusion set.

[0038] The quality inspection module is used to perform quality inspection on the key surfaces of automotive parts based on the detection anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of automotive parts.

[0039] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described image processing-based automotive parts quality inspection method.

[0040] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing-based automotive parts quality inspection method.

[0041] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0042] This application provides an image processing-based automotive parts quality inspection system and method. The system involves real-time acquisition of a multi-dimensional information fusion set of key surfaces of automotive parts during the production process. Based on the two-dimensional part image in the multi-dimensional information fusion set, the feature mining granularity is determined during image feature mining. Key feature points are mined from the two-dimensional part image based on the feature mining granularity to obtain a key feature point domain for the key surfaces of the automotive parts. Similarity detection is performed using the key feature point domain to obtain detection distortion features for the key surfaces of the automotive parts. The system uses the illumination equalization grayscale image of the two-dimensional part image to determine the crack texture features of the key surfaces of the automotive parts. A multi-scale feature vector for the key surfaces of the automotive parts is generated using the three-dimensional registration point cloud data in the multi-dimensional information fusion set. Quality inspection of the key surfaces of the automotive parts is performed based on the detection distortion features, the crack texture features, and the multi-scale feature vector to obtain the inspection results for the key surfaces of the automotive parts.

[0043] Therefore, this application first determines the feature mining granularity based on the two-dimensional part image fused with multi-dimensional information, and then mines key feature points and constructs key feature point domains according to the feature mining granularity. This allows for the comprehensive capture of the geometric features of key surfaces of parts at different scales, while highlighting abnormal information. It can adaptively balance the extraction of micro-details and macro-contour features, and uses the generated detection anomaly features as the basis for quality judgment. The detection system can more accurately identify abnormal areas of key surfaces of parts, improving defect recognition rate and overall detection accuracy. Then, by extracting crack texture features using the illumination equalization grayscale image of the two-dimensional part image, and generating multi-scale feature vectors of key surfaces using three-dimensional registration point cloud data, a deep fusion of two-dimensional texture information and three-dimensional geometric information of key surfaces of automotive parts can be achieved. Through this multi-modal, multi-scale feature extraction method, the detection system can simultaneously capture micro and macro features, achieving comprehensive perception of anomalies of key surfaces. Finally, by using detection anomaly features, crack texture features, and multi-scale feature vectors as inputs to perform quality inspection on key surfaces of automotive parts, a comprehensive and multi-level perception of surface defects of parts can be achieved, thereby improving the detection accuracy of the detection system.

[0044] In summary, the technical solution adopted in this application can effectively extract multi-scale features of key surfaces of automotive parts for quality inspection, thereby improving the inspection accuracy of the inspection system. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this embodiment of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is an exemplary flowchart of an image processing-based quality inspection method for automotive parts, as shown in some embodiments of this application.

[0047] Figure 2 This is an exemplary flowchart illustrating the determination of key feature point domains of critical surfaces of automotive components according to some embodiments of this application;

[0048] Figure 3 This is a schematic diagram of the structure of an image processing-based automotive parts quality inspection system according to some embodiments of this application;

[0049] Figure 4 This is a schematic diagram of the structure of a computer device for implementing an image processing-based method for quality inspection of automotive parts, according to some embodiments of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0051] This application provides an image processing-based automotive parts quality inspection system and method. Its core is the real-time acquisition of a multi-dimensional information fusion set of key surfaces of automotive parts during the production process. Based on the two-dimensional part image in the multi-dimensional information fusion set, the feature mining granularity in the image feature mining process is determined. Key feature points are mined from the two-dimensional part image based on the feature mining granularity to obtain the key feature point domain of the key surfaces of the automotive parts. Similarity detection is performed using the key feature point domain to obtain the detection distortion features of the key surfaces of the automotive parts. The crack texture features of the key surfaces of the automotive parts are determined using the illumination equalization grayscale image of the two-dimensional part image. Multi-scale feature vectors of the key surfaces of the automotive parts are generated using the three-dimensional registration point cloud data in the multi-dimensional information fusion set. Quality inspection of the key surfaces of the automotive parts is performed based on the detection distortion features, the crack texture features, and the multi-scale feature vectors to obtain the inspection results of the key surfaces of the automotive parts. This scheme can effectively extract multi-scale features of the key surfaces of automotive parts for quality inspection, thereby improving the detection accuracy of the inspection system.

[0052] To better understand the above technical solutions, a detailed description of the technical solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. (Refer to...) Figure 1 The figure is an exemplary flowchart of an image processing-based quality inspection method for automotive parts according to some embodiments of this application. The figure mainly includes the following steps:

[0053] In step S101, a multi-dimensional information fusion set of key aspects of automotive parts is collected in real time during the production process of automotive parts.

[0054] In some embodiments, the real-time acquisition of multi-dimensional information fusion sets of key aspects of automotive parts during the production process can be achieved in the following ways:

[0055] Two-dimensional images of key surfaces of automotive parts are captured in real time using industrial cameras during the production process.

[0056] Based on the simultaneous acquisition of two-dimensional part images of key surfaces of automotive parts and three-dimensional point clouds at the same time using 3D sensors, the three-dimensional point clouds are then denoised and registered to obtain three-dimensional registered point cloud data of key surfaces of automotive parts.

[0057] A multi-dimensional information fusion set of key surfaces of automotive parts during the automotive parts production process is constructed based on the two-dimensional part images and the three-dimensional registration point cloud data.

[0058] In practical implementation, firstly, industrial cameras are deployed at key workstations on the automotive parts production line. These cameras use high-speed shutters and global snapshot modes to capture real-time 2D images of key surfaces of automotive parts during production. Then, 3D sensors can be used to simultaneously acquire 3D point clouds of these key surfaces. To ensure time synchronization and spatial consistency, the industrial cameras and 3D sensors can be linked to the conveyor encoder via a PLC or hardware trigger to achieve cycle-time triggering. Motion compensation is then performed using encoder displacement to ensure that the 2D images and 3D point clouds are acquired under the same workpiece conditions. Finally, the acquired 3D point clouds can be filtered to remove unwanted particles. Noise reduction, including statistical outlier removal, radius neighborhood filtering, and normal smoothing, is used to reduce the interference of sensor noise on geometric feature extraction. Then, using coarse registration combined with a fine ICP algorithm, the denoised point cloud is precisely registered with the CAD model or tooling coordinate system to obtain high-precision 3D registered point cloud data for key surfaces of automotive parts. Finally, a multi-dimensional information fusion set of key surfaces of automotive parts during the production process can be constructed based on the 2D part image and the 3D registered point cloud data. This dataset, composed of the 2D part image and the 3D registered point cloud data, forms a multi-dimensional information fusion set of key surfaces of automotive parts during the production process.

[0059] In step S102, the feature mining granularity in the image feature mining process is determined based on the two-dimensional part image in the multi-dimensional information fusion set. Key feature points are mined on the two-dimensional part image based on the feature mining granularity to obtain the key feature point domain of the key surface of the automotive parts. The key feature point domain is used for similarity detection to obtain the detection differentiation features of the key surface of the automotive parts.

[0060] In some embodiments, determining the feature mining granularity in the image feature mining process based on the two-dimensional part images in the multi-dimensional information fusion set can be achieved in the following ways:

[0061] Obtain the pre-set image scaling parameters;

[0062] Extract feature parameters from the two-dimensional part images in the multi-dimensional information fusion set;

[0063] The feature mining granularity in the image feature mining process is determined based on the image scaling parameters and the feature parameters.

[0064] In practical implementation, firstly, pre-set image scaling parameters can be obtained. These parameters are used to adjust the two-dimensional part image to a scale suitable for feature mining. The image scaling parameters can be pre-set based on historical experience and standard parameters of automotive parts production lines, which will not be elaborated here. Secondly, feature parameters of the two-dimensional part image in the multi-dimensional information fusion set can be extracted. These feature parameters include the length and width features of the two-dimensional part image, i.e., the image length and width are used as length and width features, respectively. Finally, the feature mining granularity in the image feature mining process can be determined based on the image scaling parameters and feature parameters. This feature mining granularity represents the mining scale in the image feature mining process, determining the level of detail and coverage of feature mining. In actual implementation, this feature mining granularity can be determined using the following formula:

[0065]

[0066] in, This indicates the granularity of feature mining during the image feature mining process. The parameters represent the image scaling transformation, where W represents the length feature of the two-dimensional part image and H represents the width feature of the two-dimensional part image.

[0067] Preferably, in some embodiments, reference is made to Figure 2 As shown in the figure, this is an exemplary flowchart of determining the key feature point domain of a key surface of an automotive component according to some embodiments of this application. In this embodiment, the key feature point mining of the two-dimensional part image based on the feature mining granularity to obtain the key feature point domain of the key surface of the automotive component can be achieved by the following steps:

[0068] In step S1021, an image mining scale model of the two-dimensional part image is constructed based on the feature mining granularity.

[0069] In step S1022, a scale feature coverage map corresponding to each scale level in the image mining scale model is constructed;

[0070] In step S1023, for each scale feature coverage map, the feature aggregation coefficient of each pixel in the scale feature coverage map is determined;

[0071] In step S1024, the scale feature coverage map is updated based on the feature aggregation coefficients of all pixels, thereby obtaining the coverage map update map corresponding to each scale feature coverage map.

[0072] In step S1025, the key feature point domain of the key surface of the automotive component is constructed based on all the coverage map update maps.

[0073] In specific implementation, firstly, an image mining scale model for the two-dimensional part image can be constructed based on the feature mining granularity. That is, the two-dimensional image is divided into multiple scale levels according to the size of the feature mining granularity. For example, when the feature mining granularity is N, the two-dimensional image can be divided into N scale levels, where each scale level corresponds to a different scale image, so as to capture micro-texture and macro-structure information at the same time. The set model composed of all scale levels in hierarchical order is used as the image mining scale model for the two-dimensional part image. Then, a scale feature coverage map corresponding to each scale level in the image mining scale model can be constructed. That is, a scale feature coverage map of the same size is constructed according to the scale image size of each scale level in the image mining scale model.

[0074] Furthermore, in specific implementation, for each scale feature coverage map, the feature clustering coefficient of each pixel in the scale feature coverage map can be determined. The feature clustering coefficient represents the feature density at the corresponding pixel location in the scale feature coverage map. This can be achieved by using a feature point detection algorithm to detect the number of feature points at each pixel location in the scale feature coverage map, and then using the detected number of feature points as the feature clustering coefficient of the corresponding pixel. Then, the scale feature coverage map can be updated based on the feature clustering coefficients of all pixels. This involves comparing the feature clustering coefficients of all pixels with a pre-set threshold, discarding feature points at locations where the feature clustering coefficient is greater than the threshold, and finally obtaining the updated feature clustering coefficient. The scale feature map containing feature points is used as the corresponding cover map update map. The cover map update map corresponding to each scale feature map can be obtained through the above method. Finally, the key feature point domain of the key surface of the automotive parts can be constructed based on all the cover map update maps. The key feature point domain refers to the set and spatial distribution area of ​​key points that can represent the key surface features of the automotive parts, which are selected through the feature mining process in the two-dimensional part image. The feature points in the cover map update maps of all scales can be fused at multiple scales. The spatial distribution and set of feature points can be obtained by combining methods such as maximum response projection, weighted superposition, or non-maximum suppression, thereby constructing the key feature point domain of the key surface of the automotive parts.

[0075] In some embodiments, similarity detection is performed using the key feature point domain to obtain the detection differentiation features of key surfaces of automotive parts. Specifically, this can be achieved in the following manner:

[0076] Obtain the standard feature point domain of key surfaces of automotive parts;

[0077] Similarity detection is performed based on the key feature point domain and the standard feature point domain to obtain the detection variation features of key surfaces of automotive parts.

[0078] In practice, firstly, a standard feature point domain for the key surfaces of automotive parts can be obtained. This standard feature point domain is a reference key point set obtained through statistical analysis of key feature points of historical high-quality parts or manual calibration. Then, similarity detection can be performed based on the key feature point domain and the standard feature point domain to obtain the detection anomaly features of the key surfaces of automotive parts. These anomaly features represent the degree of anomaly in the feature distribution of the key surfaces of automotive parts. In actual implementation, the key feature point domain of the key surfaces of automotive parts can be matched with the standard feature point domain. That is, a descriptor-based feature matching algorithm (such as SIFT, ORB, FPFH) is used to calculate the Euclidean distance, angle difference, or local feature similarity between points. After the matching is completed, the key point matching error, local offset, proportion of unmatched points, and feature response difference are statistically analyzed. The key point matching error, local offset, proportion of unmatched points, and feature response difference are then weighted and summed according to the set weights. The calculation result is then used as the detection anomaly features of the key surfaces of automotive parts.

[0079] It should be noted that by determining the feature mining granularity based on the 2D part image fused from multi-dimensional information, and then mining key feature points and constructing key feature point domains according to the feature mining granularity, the geometric features of the key surfaces of the parts can be comprehensively captured at different scales. Simultaneously, it highlights abnormal information and adaptively balances the extraction of microscopic details and macroscopic contour features, achieving multi-scale and multi-level feature representation. This ensures that key defects are not overlooked while suppressing the interference of background noise and irrelevant information on the detection results. Using the generated detection-differentiated features as the basis for quality judgment, the detection system can more accurately identify abnormal areas on the key surfaces of parts, improve the defect recognition rate and overall detection accuracy, and provide solid technical support for highly reliable online quality inspection.

[0080] In step S103, the crack texture features of the key surfaces of the automotive parts are determined using the illumination equalization grayscale image of the two-dimensional part image, and a multi-scale feature vector of the key surfaces of the automotive parts is generated by the three-dimensional registration point cloud data in the multi-dimensional information fusion set.

[0081] In some embodiments, determining the crack texture features of key surfaces of automotive parts using the illumination-equalized grayscale image of the two-dimensional part image can be achieved in the following manner:

[0082] The two-dimensional part image is subjected to illumination compensation and texture enhancement to obtain an illumination-balanced grayscale image of the two-dimensional part image.

[0083] The texture features of the crack region in the illumination-equalized grayscale image are determined based on the grayscale co-occurrence matrix;

[0084] The texture features are vectorized to obtain the crack texture features of the key surfaces of automotive parts.

[0085] In practical implementation, firstly, illumination compensation and texture enhancement can be performed on the 2D part image to eliminate brightness unevenness caused by changes in the workstation light source or metal reflection, while highlighting the details of cracks and minor surface defects. Adaptive histogram equalization can be used to perform illumination equalization processing on the image to obtain an image with consistent grayscale. Then, high-pass filtering is used to enhance the texture, making the linear structure of the crack and local grayscale changes more obvious, thereby generating an illumination-equalized grayscale image of the 2D part image. Then, the texture features of the crack region in the illumination-equalized grayscale image can be determined based on the grayscale co-occurrence matrix, that is, the grayscale values ​​can be used to determine the texture features of the crack region in the illumination-equalized grayscale image. The co-occurrence matrix analyzes the texture characteristics of crack regions in the illumination-equalized grayscale image, calculates the contrast, correlation, energy, entropy, and other indicators of the crack regions, and uses them as texture features of the crack regions in the illumination-equalized grayscale image. Finally, the texture features can be vectorized, that is, the extracted texture features are processed into feature vectors, and the constructed unified high-dimensional feature vector is used as the crack texture features of the key surfaces of automotive parts. The crack texture features are used to describe the spatial distribution, texture direction, and intensity characteristics of cracks on the key surfaces of automotive parts, providing accurate input for subsequent defect identification, quality assessment, and multimodal feature fusion.

[0086] In some embodiments, generating multi-scale feature vectors for key surfaces of automotive parts from the three-dimensional registration point cloud data in the multi-dimensional information fusion set can be achieved in the following manner:

[0087] Multi-scale feature extraction is performed on the three-dimensional registration point cloud data in the multi-dimensional information fusion set to obtain the surface geometric descriptor, statistical features and local shape descriptor of the key surfaces of automotive parts;

[0088] Multi-scale feature vectors of key surfaces of automotive parts are generated based on the surface geometry descriptor, the statistical features, and the local shape descriptor.

[0089] In practical implementation, firstly, multi-scale feature extraction can be performed on the 3D registration point cloud data fused from multi-dimensional information. This involves calculating surface geometric descriptors such as local normal vectors, curvature, and concavity / convexity differences at the microscale, statistical features such as point cloud density, thickness deviation, and local height distribution at the mesoscale to reflect the consistency between the local morphology and surface of the critical surface, and extracting local shape descriptors at the macroscale to capture the overall contour and neighborhood structure information. Then, multi-scale feature vectors of the critical surfaces of automotive parts can be generated based on the surface geometric descriptors, statistical features, and local shape descriptors. This involves constructing high-dimensional multi-scale feature vectors from the surface geometric descriptors, statistical features, and local shape descriptors through feature concatenation, thereby generating multi-scale feature vectors of the critical surfaces of automotive parts. These multi-scale feature vectors comprehensively represent the multi-level geometric information of the critical surfaces of parts, from micro-texture to macro-shape, providing accurate and rich feature inputs for subsequent crack detection, deformation determination, and multi-modal quality assessment.

[0090] It should be noted that by extracting crack texture features using a uniformly shaded grayscale image of a 2D part image, and simultaneously generating multi-scale feature vectors for key surfaces using 3D registered point cloud data, a deep fusion of 2D texture information and 3D geometric information of key surfaces of automotive parts can be achieved. In this process, the uniformly shaded grayscale image highlights the texture details of micro-cracks, scratches, and surface defects, while the multi-scale feature vectors comprehensively characterize the macroscopic contour, local shape, and surface curvature of the key surfaces. Through this multimodal, multi-scale feature extraction method, the detection system can simultaneously capture microscopic and macroscopic features, achieving comprehensive perception of anomalies on key surfaces, enhancing sensitivity to micro-defects and local deformations, and reducing the impact of illumination changes or surface reflections on detection accuracy, thereby significantly improving the accuracy and reliability of part quality inspection.

[0091] In step S104, the key surfaces of the automotive parts are inspected based on the detected anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of the automotive parts.

[0092] In some embodiments, the quality inspection of key surfaces of automotive parts is performed based on the detected anomaly features, the crack texture features, and the multi-scale feature vectors to obtain the inspection results of the key surfaces of automotive parts. Specifically, this can be achieved in the following manner:

[0093] A lightweight deep neural network is used for supervised learning to establish a defect type identification and quality level determination model;

[0094] The detection distortion features, crack texture features, and multi-scale feature vectors are input into the defect type identification and quality level determination model to perform quality inspection on the key surfaces of automotive parts, thereby obtaining the inspection results of the key surfaces of automotive parts.

[0095] In practical implementation, firstly, a supervised learning model can be established using a lightweight deep neural network (such as MobileNet, EfficientNet, or a lightweight convolutional neural network) to simultaneously achieve defect type identification and quality level determination, thus obtaining a defect type identification and quality level determination model. In actual implementation, multiple batches of labeled samples can be collected, using detection features such as deformation features, crack texture features, and multi-scale feature vectors as input features to construct a unified high-dimensional input vector. The network can then be designed to adapt to multi-modal feature fusion. Feature concatenation or attention mechanisms can be used to enhance key feature responses. Finally, defect type classification (such as cracks, scratches, dents, etc.) and quality level determination can be performed on the multiple output branches of the network. The defect type identification and quality grade determination model (e.g., qualified, minor defect, severe defect) uses cross-entropy loss function or weighted loss function to jointly optimize the classification and scoring tasks during the training phase. At the same time, batch normalization, Dropout and data augmentation strategies are adopted to improve the generalization ability of the model. Then, the detection of abnormal features, crack texture features and multi-scale feature vectors can be input into the defect type identification and quality grade determination model to perform quality inspection on the key surfaces of automotive parts. Thus, through forward propagation, the defect type prediction and corresponding quality grade output of the key surfaces of automotive parts are obtained, thereby generating the detection results of the key surfaces of automotive parts. The detection results include the defect type classification and quality grade score of the key surfaces of automotive parts.

[0096] It should be noted that by using detection anomaly features, crack texture features, and multi-scale feature vectors as inputs to perform quality inspection on the key surfaces of automotive parts, it is possible to achieve comprehensive and multi-level perception of surface defects. By fusing these three types of features and inputting them into a supervised learning model for judgment, it is possible not only to enhance the sensitivity to anomalies such as micro-cracks, scratches, dents, and local deformations, but also to suppress the interference of background noise and illumination changes on the detection. This enables the effective extraction and accurate analysis of multi-scale and multi-modal features of the key surfaces, thereby improving the defect recognition rate and quality assessment accuracy of parts.

[0097] Therefore, this application first determines the feature mining granularity based on the two-dimensional part image fused with multi-dimensional information, and then mines key feature points and constructs key feature point domains according to the feature mining granularity. This allows for the comprehensive capture of the geometric features of key surfaces of parts at different scales, while highlighting abnormal information. It can adaptively balance the extraction of micro-details and macro-contour features, and uses the generated detection anomaly features as the basis for quality judgment. The detection system can more accurately identify abnormal areas of key surfaces of parts, improving defect recognition rate and overall detection accuracy. Then, by extracting crack texture features using the illumination equalization grayscale image of the two-dimensional part image, and generating multi-scale feature vectors of key surfaces using three-dimensional registration point cloud data, a deep fusion of two-dimensional texture information and three-dimensional geometric information of key surfaces of automotive parts can be achieved. Through this multi-modal, multi-scale feature extraction method, the detection system can simultaneously capture micro and macro features, achieving comprehensive perception of anomalies of key surfaces. Finally, by using detection anomaly features, crack texture features, and multi-scale feature vectors as inputs to perform quality inspection on key surfaces of automotive parts, a comprehensive and multi-level perception of surface defects of parts can be achieved, thereby improving the detection accuracy of the detection system.

[0098] In summary, the technical solution adopted in this application can effectively extract multi-scale features of key surfaces of automotive parts for quality inspection, thereby improving the inspection accuracy of the inspection system.

[0099] Furthermore, in another aspect of this application, in some embodiments, this application provides an image processing-based automotive parts quality inspection system, with reference to... Figure 3 The figure is a schematic diagram of the structure of an image processing-based automotive parts quality inspection system according to some embodiments of this application. The image processing-based automotive parts quality inspection system includes:

[0100] Information acquisition module 201 is used to collect multi-dimensional information fusion sets of key aspects of automotive parts in real time during the production process of automotive parts;

[0101] The alienation detection module 202 is used to determine the feature mining granularity in the image feature mining process based on the two-dimensional part image in the multi-dimensional information fusion set, to mine key feature points in the two-dimensional part image based on the feature mining granularity, and to obtain the key feature point domain of the key surface of the automotive parts, and to use the key feature point domain to perform similarity detection, thereby obtaining the detected alienation features of the key surface of the automotive parts.

[0102] Feature extraction module 203 is used to determine the crack texture features of the key surfaces of automotive parts using the illumination equalization grayscale image of the two-dimensional part image, and to generate multi-scale feature vectors of the key surfaces of automotive parts through the three-dimensional registration point cloud data in the multi-dimensional information fusion set.

[0103] The quality inspection module 204 is used to perform quality inspection on the key surfaces of automotive parts based on the detection anomaly features, the crack texture features, and the multi-scale feature vector, thereby obtaining the inspection results of the key surfaces of automotive parts.

[0104] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described image processing-based automotive parts quality inspection method.

[0105] In some embodiments, reference Figure 4 The figure is a schematic diagram of the structure of a computer device implementing an image processing-based automotive parts quality inspection method according to some embodiments of this application. The image processing-based automotive parts quality inspection method in the above embodiments can... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0106] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the image processing-based automotive parts quality inspection method of this application.

[0107] The communication bus 302 can be used to transmit information between the aforementioned components.

[0108] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0109] The memory 303 stores program code for executing the solution of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the image processing-based automotive parts quality inspection method can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0110] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0111] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0112] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0113] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing-based automotive parts quality inspection method.

[0114] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0115] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for quality inspection of automotive parts based on image processing, characterized in that, Includes the following steps: Real-time collection of multi-dimensional information fusion set of key aspects of automotive parts during the automotive parts production process; The feature mining granularity in the image feature mining process is determined based on the two-dimensional part image in the multi-dimensional information fusion set. Key feature points are mined on the two-dimensional part image based on the feature mining granularity to obtain the key feature point domain of the key surface of the automotive parts. The key feature point domain is used for similarity detection to obtain the detection differentiation features of the key surface of the automotive parts. The crack texture features of the key surfaces of the automotive parts are determined by using the illumination equalization grayscale image of the two-dimensional part image, and multi-scale feature vectors of the key surfaces of the automotive parts are generated by using the three-dimensional registration point cloud data of the multi-dimensional information fusion set. The key surfaces of automotive parts are inspected based on the detected anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of automotive parts. Specifically, the multi-dimensional information fusion set for key aspects of automotive parts during the real-time acquisition process includes: Two-dimensional images of key surfaces of automotive parts are captured in real time using industrial cameras during the production process. Based on the simultaneous acquisition of two-dimensional part images of key surfaces of automotive parts and three-dimensional point clouds at the same time using 3D sensors, the three-dimensional point clouds are then denoised and registered to obtain three-dimensional registered point cloud data of key surfaces of automotive parts. A multi-dimensional information fusion set of key surfaces of automotive parts during the automotive parts production process is constructed based on the two-dimensional part images and the three-dimensional registration point cloud data. Specifically, determining the feature mining granularity in the image feature mining process based on the two-dimensional part image in the multi-dimensional information fusion set includes: Obtain the pre-set image scaling parameters; Extract feature parameters from the two-dimensional part images in the multi-dimensional information fusion set; The feature mining granularity in the image feature mining process is determined based on the image scaling parameters and the feature parameters. Specifically, the key feature point domain of the key surfaces of automotive parts is obtained by mining key feature points from the two-dimensional part image based on the aforementioned feature mining granularity, including: Based on the aforementioned feature mining granularity, an image mining scale model for the two-dimensional part image is constructed; Construct a scale feature coverage map corresponding to each scale level in the image mining scale model; For each scale feature coverage map, determine the feature aggregation coefficient of each pixel in the scale feature coverage map; The scale feature coverage map is updated based on the feature aggregation coefficients of all pixels, thereby obtaining the coverage map update map corresponding to each scale feature coverage map. Based on all the overlay maps, update the map to construct the key feature point domains of the critical surfaces of automotive components.

2. The image processing-based method for quality inspection of automotive parts as described in claim 1, characterized in that, Determining the crack texture features of key surfaces of automotive parts using the illumination-equalized grayscale image of the two-dimensional part image specifically includes: The two-dimensional part image is subjected to illumination compensation and texture enhancement to obtain an illumination-balanced grayscale image of the two-dimensional part image. The texture features of the crack region in the illumination-equalized grayscale image are determined based on the grayscale co-occurrence matrix; The texture features are vectorized to obtain the crack texture features of the key surfaces of automotive parts.

3. The image processing-based method for quality inspection of automotive parts as described in claim 1, characterized in that, Generating multi-scale feature vectors for key surfaces of automotive parts from the 3D registration point cloud data fused from the multi-dimensional information fusion set specifically includes: Multi-scale feature extraction is performed on the three-dimensional registration point cloud data in the multi-dimensional information fusion set to obtain the surface geometric descriptor, statistical features and local shape descriptor of the key surfaces of automotive parts; Multi-scale feature vectors of key surfaces of automotive parts are generated based on the surface geometry descriptor, the statistical features, and the local shape descriptor.

4. The image processing-based method for quality inspection of automotive parts as described in claim 1, characterized in that, The quality inspection of key surfaces of automotive parts is performed based on the aforementioned detection anomaly features, crack texture features, and multi-scale feature vectors, resulting in the inspection results of the key surfaces of automotive parts. Specifically, this includes: A lightweight deep neural network is used for supervised learning to establish a defect type identification and quality level determination model; The detection distortion features, crack texture features, and multi-scale feature vectors are input into the defect type identification and quality level determination model to perform quality inspection on the key surfaces of automotive parts, thereby obtaining the inspection results of the key surfaces of automotive parts.

5. An image processing-based automotive parts quality inspection system, used to execute the image processing-based automotive parts quality inspection method as described in any one of claims 1 to 4, characterized in that, include: The information acquisition module is used to collect multi-dimensional information fusion sets of key aspects of automotive parts in real time during the production process of automotive parts; The alienation detection module is used to determine the feature mining granularity in the image feature mining process based on the two-dimensional part image in the multi-dimensional information fusion set, to mine key feature points of the two-dimensional part image based on the feature mining granularity, and to obtain the key feature point domain of the key surface of the automotive parts. The key feature point domain is used to perform similarity detection, and to obtain the detected alienation features of the key surface of the automotive parts. The feature extraction module is used to determine the crack texture features of the key surfaces of the automotive parts using the illumination equalization grayscale image of the two-dimensional part image, and to generate multi-scale feature vectors of the key surfaces of the automotive parts through the three-dimensional registration point cloud data of the multi-dimensional information fusion set. The quality inspection module is used to perform quality inspection on the key surfaces of automotive parts based on the detection anomaly features, the crack texture features, and the multi-scale feature vectors, thereby obtaining the inspection results of the key surfaces of automotive parts.

6. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the image processing-based automotive parts quality inspection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing-based automotive parts quality inspection method as described in any one of claims 1 to 4.

Images