A method and system for full appearance inspection of the inner wire solder of automobile brake hose joints

Abstract

The present invention discloses a method and system for comprehensive appearance inspection of the solder inside automotive brake hose connectors. The method includes multi-angle collaborative imaging, geometric defect detection, texture defect detection, dynamic fusion decision-making, and fault tolerance adjustment. By capturing images and synthesizing a panoramic image using asymmetrically arranged cameras, the system eliminates blind spots. This system comprehensively covers the solder area, improving defect detection accuracy and efficiency, and enhancing adaptability to incoming material deviations. It is suitable for comprehensive appearance quality inspection of the solder inside automotive brake hose connectors.

Description

Technical Field

[0001] The present invention relates to the field of machine vision inspection in the auto parts industry, and in particular to a method and system for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint. Background Art

[0002] In the production process of automobile brake hose joints, the quality of the inner wire solder directly affects the sealing performance and structural strength of the joint, so the inspection of the solder appearance is crucial.

[0003] Traditional inspection methods rely primarily on manual testing, using a microscope camera to visually determine if the solder is defective. However, manual inspection has significant shortcomings. It is slow, making it difficult to meet the efficiency requirements of large-scale production. Furthermore, inspectors are prone to fatigue, and the results are subject to significant subjectivity, resulting in poor inspection consistency.

[0004] With technological advancements, rudimentary visual inspection methods have emerged, but numerous challenges remain. Traditional algorithms struggle to consistently identify solder due to factors such as varying surface finishes, large variations in solder fill levels, and fluctuations in incoming material position, resulting in poor solder joint inspection. Minor defects such as sand holes are often missed due to lighting conditions and background noise, making accurate inspection impossible.

[0005] Therefore, in order to solve the problems existing in the prior art, the present invention proposes a method and system for full appearance inspection of the inner wire solder of an automobile brake hose joint. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the present invention aims to provide a method and system for full appearance inspection of the inner wire solder of automobile brake hose joints.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint comprises the following steps:

[0009] In the multi-angle collaborative imaging step, a plurality of asymmetrically arranged cameras are used to synchronously capture the solder area images to obtain a plurality of multi-angle solder area images; the feature points of each solder area image are extracted and the transformation matrix between the viewing angles is calculated, and a pixel-level aligned panoramic composite image is generated through bilinear interpolation;

[0010] a geometric defect detection step, inputting the panoramic composite image into a feature extraction model to perform edge detection to obtain image contour information, calculating an area ratio of the solder contour area to a reference area and a curvature change rate based on the image contour information, and outputting a defect type as a geometric defect based on the area ratio and the curvature change rate;

[0011] In the texture defect detection step, a sand hole area is determined by a pre-trained segmentation network and a binary mask and confidence score of the sand hole area are output. The texture variance and brightness deviation in the sand hole area are calculated, and the defect type is output as a texture defect based on the binary mask, confidence score, texture variance, and brightness deviation.

[0012] The dynamic fusion decision step determines the fusion path according to the defect type, performs dynamic defect fusion according to the specific defect form in the defect type in different fusion paths, and outputs the final defect result.

[0013] As a further improvement of the present invention, the geometric defect detection step includes: obtaining image contour information after subjecting the panoramic composite image to Sobel edge detection, performing contour closing according to the image contour information, and calculating the area ratio of the solder contour area to the reference area of ​​the closed area; calculating the curvature change rate, setting the continuous pixels that exceed the preset change rate as curvature mutation points along the image contour edge according to a preset length, and calculating the number of curvature mutation points. When the area ratio exceeds the preset area threshold, the output defect type is a geometric defect, and the defect form is an overflow defect; when the curvature mutation point is greater than the preset number of mutation points, the output defect type is a geometric defect, and the defect form is a solder defect.

[0014] As a further improvement of the present invention, the texture defect detection step includes inputting the panoramic synthetic image into a pre-trained segmentation network, outputting a binary mask of the sand hole area and a corresponding confidence level, performing dynamic threshold binarization on the binary mask, calculating the binarized result by mask mean and standard deviation, and calculating the area of ​​the connected domain in the sand hole area. When the confidence level is greater than a preset confidence threshold and the area of ​​the connected domain after binarization is greater than a preset minimum sand hole size, the output defect type is a texture defect, and the defect form is a sand hole defect. Multimodal feature analysis is performed in the sand hole area output by the segmentation network, the texture variance is calculated by extracting local binary pattern features, and the brightness deviation from the reference color is calculated by converting the image to the HSV color space. When both the texture variance and the brightness deviation are greater than the corresponding preset thresholds, the output defect type is a texture defect, and the defect form is a foreign matter defect.

[0015] As a further improvement of the present invention, the dynamic threshold binarization processing includes performing edge detection on the panoramic composite image to obtain the sand hole contour, and obtaining the sand hole area as a connected domain based on the image contour, calculating the area of ​​the connected domain, performing a morphological opening operation to filter out the noise area whose connected domain area is smaller than a preset pixel value, and dynamically adjusting the smoothing kernel size according to the area of ​​the sand hole area to apply the curvature smoothing algorithm to the sand hole contour to obtain the binarization processing result.

[0016] As a further improvement of the present invention, the dynamic fusion decision step also includes: when the output defect detection results include both geometric defects and texture defects, it is judged that there is a conflict, and fusion is performed according to the defect form. When the conflict type is overflow defect and sand hole defect, a weighted score is calculated to judge the defect form; when the conflict type is overflow defect and foreign body defect, the foreign body classification sub-model is called for re-inspection to judge the defect form; when the conflict type is solder defect and sand hole defect, the number of sand hole detections in the original several solder area images is counted, and the sand hole defect is output when it is greater than a preset threshold; when the conflict type is solder defect and foreign body defect, the defect form is judged through spatial positioning analysis.

[0017] As a further improvement of the present invention, when the conflict types are overflow defects and sand hole defects, a comprehensive defect score is generated as a weighted score by calculating the sand hole confidence and the ratio of overflow to number. When the comprehensive defect score is greater than a preset threshold, it is determined to be a sand hole defect; otherwise, the defect result is output as the simultaneous existence of sand hole defects and overflow defects.

[0018] As a further improvement of the present invention, when the conflict type is an overflow defect and a foreign body defect, the foreign body classification sub-model includes intercepting an ROI sub-image of preset pixel size based on the center of the conflict area, inputting the ROI sub-image into a pre-trained foreign body classification sub-model for multi-scale feature extraction, and outputting the probability of foreign body existence based on the multi-scale features. When the probability of foreign body existence is higher than a preset threshold, it is determined to be a foreign body defect; otherwise, the defect result is output as the simultaneous existence of foreign body defects and overflow defects.

[0019] As a further improvement of the present invention, when the conflict type is solder defect and foreign body defect, the solder area image with the highest resolution viewing angle is selected to locate the solder area through sub-pixel edge detection, and the center coordinates of the foreign body are extracted. The relationship distance index between the foreign body and the solder is calculated based on the center coordinates of the foreign body and the center of mass of the solder area. When the relationship distance index is less than a preset threshold, it is judged as a foreign body defect. Otherwise, the defect result is output as the simultaneous existence of solder defect and foreign body defect.

[0020] As a further improvement of the present invention, it also includes a fault-tolerant adjustment step. When the position deviation between the solder area image and the solder area image in the historical data is greater than a preset threshold, the image is dynamically adjusted through an affine transformation matrix in the image coordinate system to eliminate the position deviation.

[0021] A full appearance inspection system for the inner wire solder of automobile brake hose joints, comprising:

[0022] The angle collaborative imaging module uses several asymmetrically arranged cameras to synchronously capture solder area images to obtain multiple multi-angle solder area images. It extracts the feature points of each solder area image and calculates the transformation matrix between the viewing angles, and generates a pixel-level aligned panoramic composite image through bilinear interpolation.

[0023] a geometric defect detection module, inputting the panoramic composite image into a feature extraction model to perform edge detection to obtain image contour information, calculating an area ratio of the solder contour area to a reference area and a curvature change rate based on the image contour information, and outputting a defect type as a geometric defect based on the area ratio and the curvature change rate;

[0024] The texture defect detection module uses a pre-trained segmentation network to determine the sand hole area and output a binary mask and confidence score for the sand hole area. It also calculates the texture variance and brightness deviation within the sand hole area and outputs the defect type as a texture defect based on the binary mask, confidence score, texture variance, and brightness deviation.

[0025] The dynamic fusion decision module determines the fusion path according to the defect type, performs dynamic defect fusion according to the specific defect form in the defect type in different fusion paths, and outputs the final defect result.

[0026] The beneficial effects of the present invention are as follows: through multi-angle collaborative imaging, it is possible to obtain all-round image information of the solder area and generate a panoramic composite image, effectively eliminating the blind spots existing in traditional detection and improving the comprehensive detection of the solder area; by using geometric defect detection and texture defect detection to detect different types of defects respectively, the accuracy of defect identification is improved. Dynamic fusion decision-making is carried out according to different defect types and conflict situations, ensuring the reliability of the final defect results and reducing the occurrence of misjudgments and missed judgments. By using fault-tolerant adjustment to deal with the problem of incoming material position deviation, the adaptability and robustness of the system are improved, so that the system can work stably in an actual production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to the present invention;

[0028] Figure 2 This is a block diagram of a full appearance inspection system for the inner wire solder of an automobile brake hose joint according to the present invention. DETAILED DESCRIPTION

[0029] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments. Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom," "top," "inner," and "outer" refer to directions toward or away from the geometric center of a particular component, respectively.

[0030] The embodiment of the present invention provides a method for inspecting the appearance of the inner wire solder of an automobile brake hose joint. Figures 1 to 2 As shown, the following steps are included:

[0031] In the multi-angle collaborative imaging step, a plurality of asymmetrically arranged cameras are used to synchronously capture the solder area images to obtain a plurality of multi-angle solder area images; the feature points of each solder area image are extracted and the transformation matrix between the viewing angles is calculated, and a pixel-level aligned panoramic composite image is generated through bilinear interpolation;

[0032] Three cameras are positioned asymmetrically around the joint. The camera angles are determined based on the tubular structure of the wire solder inside the brake hose connector, ensuring comprehensive image capture of the solder area. The cameras simultaneously capture images of the solder area, producing three images from different angles.

[0033] Feature points are extracted from each solder area image, such as corner points of the solder edge and texture feature points. The transformation matrix between the viewpoints is calculated based on these feature points. The three images are then aligned pixel-wise using bilinear interpolation to generate a panoramic composite image.

[0034] a geometric defect detection step, inputting the panoramic composite image into a feature extraction model to perform edge detection to obtain image contour information, calculating an area ratio of the solder contour area to a reference area and a curvature change rate based on the image contour information, and outputting a defect type as a geometric defect based on the area ratio and the curvature change rate;

[0035] The panoramic composite image is input into the feature extraction model, and the Sobel edge detection algorithm is used to detect the image edges and obtain the image contour information. The image contour information is then closed, and the closed area is used as the solder area. The area of ​​this area is calculated as the solder contour area.

[0036] The solder contour area is compared to a preset baseline area and the area ratio is calculated. Pixels are selected along the image contour at preset intervals, and the curvature change of adjacent pixels is calculated. Consecutive pixels that exceed the preset change rate are designated as curvature mutation points, and the number of curvature mutation points is counted.

[0037] When the area ratio exceeds the preset area threshold, it is determined to be an overflow defect among geometric defects; when the number of curvature mutation points is greater than the preset number of mutation points, it is determined to be a welding defect among geometric defects.

[0038] In the texture defect detection step, a sand hole area is determined by a pre-trained segmentation network and a binary mask and confidence score of the sand hole area are output. The texture variance and brightness deviation in the sand hole area are calculated, and the defect type is output as a texture defect based on the binary mask, confidence score, texture variance, and brightness deviation.

[0039] The panoramic composite image is input into a pre-trained segmentation network. In this embodiment, a U-Net network is selected. The network has been trained with a large number of solder images containing sand hole defects and can output a binary mask of the sand hole area and the corresponding confidence level.

[0040] Dynamic threshold binarization is performed on the binary mask: First, the sand hole outline is obtained through edge detection. Based on the outline, the sand hole region is identified as a connected domain, and the area of ​​the connected domain is calculated. Noise regions with a connected domain area smaller than a preset pixel value are filtered out using a morphological opening operation. The smoothing kernel size is dynamically adjusted based on the sand hole area, and a curvature smoothing algorithm is applied to the sand hole outline to obtain the binarization result.

[0041] When the confidence of the sand hole area is greater than the preset confidence threshold and the area of ​​the connected domain after binarization is greater than the preset minimum sand hole size, it is determined to be a sand hole defect in the texture defect.

[0042] Multimodal feature analysis is performed within the sand hole region output by the segmentation network: local binary pattern features are extracted and texture variance is calculated. The image is converted to HSV color space, and the lightness deviation between the sand hole region and the reference color is calculated. When both the texture variance and lightness deviation exceed the corresponding preset thresholds, it is identified as a foreign object defect within the texture defect.

[0043] The dynamic fusion decision step determines the fusion path according to the defect type, performs dynamic defect fusion according to the specific defect form in the defect type in different fusion paths, and outputs the final defect result.

[0044] In this embodiment, the asymmetric surrounding camera array refers to a plurality of industrial cameras arranged at unequal intervals and different pitch angles with the solder area as the center, which is realized by using three high-resolution cameras distributed at 120-degree intervals. The asymmetric layout can eliminate the repeated detection areas caused by the symmetrical layout. The pixel-level aligned panoramic composite image refers to the calculation of the spatial transformation relationship between each perspective by the feature point matching algorithm, and the elimination of image stitching gaps by the bilinear interpolation algorithm, such as the use of SIFT feature point matching combined with the perspective transformation matrix to achieve multi-image fusion. The curvature change rate calculation in geometric defect detection refers to the extraction of the curvature sequence along the solder contour, the calculation of the difference rate of adjacent curvature values ​​by sliding the window, and the calculation of the curvature derivative after fitting the contour curve by the cubic spline interpolation method. The dynamic fusion decision path refers to the establishment of multi-branch processing logic according to the defect type. For example, when sand holes and foreign matter are detected at the same time, the classification sub-model is preferentially called for secondary verification.

[0045] The multi-angle collaborative imaging step uses a spatially distributed camera array to acquire three-dimensional surface information of the solder. A feature point matching algorithm compensates for perspective differences caused by workpiece position deviations, and bilinear interpolation ensures the spatial continuity of the composite image. The geometric defect detection module combines traditional edge detection with a deep learning model. The area ratio reflects macroscopic morphological anomalies, and the curvature change rate captures microscopic contour distortion. Texture defect detection uses a segmentation network to locate suspicious areas, analyzes texture uniformity through local binary patterns, and quantifies brightness deviations through HSV color space conversion. Dynamic fusion decision-making establishes priority rules based on defect confidence. For example, when geometric and texture detection results conflict, the more accurate texture analysis result is prioritized.

[0046] Specifically, such as Figures 1 to 2 As shown, the geometric defect detection step includes: obtaining image contour information after performing Sobel edge detection on the panoramic composite image, closing the contour according to the image contour information, and calculating the area ratio of the solder contour area to the reference area of ​​the closed area; calculating the curvature change rate, setting the continuous pixels that exceed the preset change rate as curvature mutation points along the image contour edge according to a preset length, and calculating the number of curvature mutation points. When the area ratio exceeds the preset area threshold, the defect type is output as a geometric defect, and the defect form is an overflow defect; when the curvature mutation points are greater than the preset number of mutation points, the defect type is output as a geometric defect, and the defect form is a solder defect.

[0047] The Sobel edge detection operator with convolution kernels of 3× in both horizontal and vertical directions is used to extract edges of three pairs of panoramic composite images to obtain the initial contour of the solder area.

[0048] The initial contour is closed by three iterations to fill the contour gaps and form a complete closed contour of the solder area.

[0049] The pixel area of ​​the closed contour is calculated by the scan line algorithm, converted into the actual area, and the ratio of it to the reference area is recorded as R.

[0050] Sampling is performed along the edge of the contour with a step size of 10 pixels. The curvature change rate Δk of adjacent sampling points is calculated. When Δk>50% and three consecutive sampling points meet this condition, it is marked as a curvature mutation segment, and the number of mutation segments N is counted.

[0051] The preset area threshold is 120% and the mutation point threshold is 3. When R>120%, geometric defects and overflow defects are output; when N>3, geometric defects and welding defects are output.

[0052] Sobel edge detection refers to a method that uses the Sobel operator to perform gradient calculation on an image to extract edge features. It is achieved by calculating the pixel gradient amplitude separately through the horizontal and vertical convolution kernels, which can effectively enhance the detection sensitivity of weak edge areas.

[0053] Contour closure refers to the process of connecting discrete edge points to form a continuous closed area, which is achieved through morphological dilation and erosion operations to ensure the integrity of subsequent area calculations.

[0054] The area ratio refers to the ratio of the actual solder contour area to the standard contour area. It is calculated by pixel statistics method and is used to quantify the degree of solder overflow.

[0055] The curvature change rate refers to the angle change in the tangent direction of the contour curve per unit length. The curvature difference between adjacent pixel points is calculated by the difference method to identify the contour mutation feature.

[0056] After Sobel edge detection, the panoramic composite image obtains initial contour information. Morphological operations are then used to close the fracture contours to form a complete area. The pixel area of ​​the closed area is compared with a preset baseline area. When the ratio exceeds a threshold, it is determined to be a solder overflow defect. Simultaneously, the curvature change rate is calculated point by point along the closed contour. The number of sudden changes exceeding the preset curvature threshold is counted in consecutive pixel segments. When the number of sudden changes exceeds the set threshold, it is determined to be a solder body defect. This dual-path judgment mechanism accurately distinguishes defect types through differences in physical characteristics, with the area parameter reflecting macroscopic morphological anomalies and the curvature sudden change reflecting microscopic contour distortion.

[0057] Specifically, such as Figures 1 to 2As shown, the texture defect detection step includes: inputting the panoramic synthetic image into a pre-trained segmentation network, outputting a binary mask of the sand hole area and the corresponding confidence, performing dynamic threshold binarization processing on the binary mask, calculating the binarized result by the mask mean and standard deviation, and calculating the area of ​​the connected domain in the sand hole area; when the confidence is greater than the preset confidence threshold and the area of ​​the connected domain after binarization is greater than the preset minimum sand hole size, the output defect type is a texture defect, and the defect form is a sand hole defect; performing multimodal feature analysis in the sand hole area output by the segmentation network, calculating the texture variance by extracting local binary pattern features, and calculating the brightness deviation from the reference color by converting the image to the HSV color space; when the texture variance and the brightness deviation are both greater than the corresponding preset thresholds, the output defect type is a texture defect, and the defect form is a foreign matter defect.

[0058] The image is input into the pre-trained U-Net segmentation network with an input size of 512×512. The output is a binary mask of the same size, and the mask and confidence map of the sand hole area are output.

[0059] Perform dynamic threshold processing on the mask: calculate the mean μ=0.3 and standard deviation σ=0.15 of the mask pixel values, set the threshold to 0.6, and retain the connected domain after binarization.

[0060] The area of ​​the connected domain is counted. When the confidence level is greater than 0.8 and the area is greater than 0.015 mm², it is determined to be a sand hole defect.

[0061] LBP features were extracted from the sand hole area with a radius of 3 and a neighborhood of 8 points. Texture variance was calculated, with a threshold of 45. The image was converted to HSV space, and the percentage difference between the lightness channel V and the baseline value was calculated, with a threshold of 12%. If both values ​​exceeded the threshold, it was determined to be a foreign body defect.

[0062] The pre-trained segmentation network refers to an image segmentation model based on deep learning, which is implemented using the U-Net or MaskR-CNN architecture. It learns the morphological features of the sand hole area through training data, generates a pixel-level binary mask and outputs the prediction confidence. Dynamic threshold binarization processing refers to dynamically adjusting the binarization threshold according to the grayscale mean and standard deviation of the mask area. It is implemented using the Otsu algorithm combined with regional statistical characteristics to eliminate noise interference caused by uneven lighting. Local binary pattern features refer to texture descriptors generated by calculating the grayscale contrast of pixel neighborhoods. The LBP operator is used to extract local texture patterns, which are used to quantify the texture differences between sand holes and foreign object surfaces. HSV color space conversion refers to converting an image from the RGB color model to the hue, saturation, and lightness space. It is implemented through the color space transformation formula and is used to separate the lightness component in the color information and compare and analyze it with the reference color.

[0063] The segmentation network extracts multi-scale features through convolutional layers, recovering the precise boundaries of the sand hole region during the decoding phase and generating a binary mask and confidence score. Dynamic threshold binarization adaptively adjusts the threshold based on the grayscale distribution characteristics of the masked region to distinguish between true sand holes and noise spots, filtering out non-target areas based on the area of ​​the connected domain. After confirming the candidate sand hole region, the local binary pattern feature generates a texture variance index by calculating the grayscale variation pattern within the pixel neighborhood to quantify surface roughness differences. The HSV-converted luminance component is then subtracted from the reference color to identify abnormal color shifts caused by foreign matter. When both the texture variance and luminance deviation exceed the preset threshold, it is determined to be a foreign matter defect; otherwise, it is classified as a sand hole defect.

[0064] Through the above-mentioned technical solution, this application effectively solves the problems of missed sand hole detection and misidentification of foreign objects. The segmentation network's high-precision mask generation capability accurately identifies tiny sand hole areas, and dynamic threshold processing eliminates illumination noise interference, avoiding misdetection of non-defective areas. The dual verification mechanism of local binary pattern features and HSV brightness analysis reliably distinguishes the surface texture of sand holes from the material differences of foreign objects, reducing the misidentification rate of traditional single-feature detection. This solution maintains stable detection performance even under complex lighting conditions, significantly improving the accuracy of texture defect classification.

[0065] Specifically, such as Figures 1 to 2 As shown, the dynamic threshold binarization processing includes performing edge detection on the panoramic composite image to obtain the sand hole contour, obtaining the sand hole area as the connected domain based on the image contour, calculating the area of ​​the connected domain, performing morphological opening operation to filter out the noise area whose connected domain area is smaller than the preset pixel value, and dynamically adjusting the smoothing kernel size according to the area of ​​the sand hole area to apply the curvature smoothing algorithm to the sand hole contour to obtain the binarization processing result.

[0066] Canny edge detection is performed on the panoramic composite image with a high threshold of 0.7 and a low threshold of 0.3 to extract the sand hole contours.

[0067] Determine the connected domain based on the contour and calculate the pixel area S of each connected domain.

[0068] A morphological opening operation of 3×3 circular structure elements is performed on the connected domain with S < 30 pixels to eliminate noise.

[0069] The smoothing kernel size is dynamically adjusted according to S: when S < 100 pixels, the kernel size is 3 × 3; when 100 ≤ S < 500 pixels, the kernel size is 5 × 5. The curvature smoothing algorithm is applied to the sand hole contour for 5 iterations to obtain the final binarization result.

[0070] Dynamic threshold binarization is a technique that adaptively adjusts segmentation parameters based on local image features. This is achieved by using a dynamic smoothing kernel adjustment strategy based on the area of ​​the connected domain. This eliminates noise interference by analyzing the spatial distribution characteristics of the sand hole region. Morphological opening filtering uses erosion and dilation operations to eliminate subtle noise. This technique uses circular structuring elements to perform morphological processing on binary images. Erosion removes isolated noise points, and then dilation restores the morphology of the valid region. Curvature smoothing is a technique for smoothing contour edges. This technique combines a Gaussian filter kernel with local curvature calculations to dynamically adjust the filter kernel size based on the size of the sand hole region, eliminating jagged edges while preserving true contour features.

[0071] The sand hole contours are extracted from the panoramic composite image using an edge detection algorithm, forming a connected domain representation. The connected domain area is calculated to distinguish true sand holes from noise regions. The area threshold can be set based on actual detection requirements, for example, 10 pixels. When the connected domain area is detected to be below the threshold, the region is filtered out using a morphological opening operation to prevent minor noise from interfering with the detection results. For valid sand hole regions, the smoothing kernel size is dynamically determined based on their area value. For example, a 5×5 kernel is used for areas greater than 100 pixels, and a 3×3 kernel is used for areas less than 100 pixels. A curvature smoothing algorithm is applied to the edges of the sand hole contours. Through Gaussian filtering with a dynamic kernel size, edge continuity is optimized while maintaining the true shape of the sand hole, ultimately outputting an accurate binary segmentation result.

[0072] Through the above solution, this application effectively addresses the problem of misidentification caused by noise interference in sand hole detection. By combining connected domain area analysis with morphological processing, it accurately distinguishes between real defects and image noise. A dynamically adjusted curvature smoothing algorithm improves the accuracy of sand hole contour segmentation, ensuring the reliability of subsequent feature extraction. While maintaining detection efficiency, this solution significantly improves the accuracy of sand hole defect recognition, providing reliable technical support for welding quality assessment.

[0073] Specifically, such as Figures 1 to 2 As shown, the dynamic fusion decision step also includes: when the output defect detection results include both geometric defects and texture defects, it is determined that there is a conflict, and fusion is performed according to the defect form. When the conflict type is overflow defect and sand hole defect, a weighted score is calculated to determine the defect form; when the conflict type is overflow defect and foreign body defect, the foreign body classification sub-model is called to re-inspect and determine the defect form; when the conflict type is solder defect and sand hole defect, the number of sand hole detections in the original several solder area images is counted, and when it is greater than a preset threshold, the sand hole defect is output; when the conflict type is solder defect and foreign body defect, the defect form is determined through spatial positioning analysis.

[0074] When foreign matter defects and texture defects are detected at the same time, the conflict judgment mechanism is triggered.

[0075] Overflow defect and sand hole defect conflict: calculate the weighted score = sand hole confidence × 0.6 + (number of overflows / total number of inspections) × 0.4. When the score is greater than 0.7, it is determined to be a sand hole defect.

[0076] Overflow defects conflict with foreign object defects: Call the ResNet-50 foreign object classification sub-model to re-inspect the 200×200 ROI image of the conflicting area and output the foreign object probability.

[0077] Conflict between solder defects and sand hole defects: Count the number of sand hole detections in the original images from three angles. If the number is ≥ 2, it is determined to be a sand hole defect.

[0078] Conflict between solder defects and foreign body defects: The center of mass of the solder area is located through sub-pixel edge detection, and the Euclidean distance to the center of the foreign body is calculated. When the distance is less than 0.5mm, it is determined to be a foreign body defect.

[0079] Weighted scoring refers to the quantitative evaluation of the ratio of sand hole confidence to the number of overflows, which is achieved by calculating the comprehensive score using linear weighting or nonlinear functions, and achieving objective judgment through preset thresholds.

[0080] The foreign object classification sub-model refers to a deep learning model that performs multi-scale feature extraction on the conflict area. It is implemented by using a convolutional neural network to construct a multi-level feature fusion structure, and obtains the probability of foreign object existence through ROI sub-image input.

[0081] Sand hole detection statistics refer to the verification of sand hole detection results based on multi-view original images, which is achieved by using sliding window counting or time series analysis methods to verify the persistence of sand holes through historical data.

[0082] Spatial positioning analysis refers to locating the solder area through sub-pixel edge detection and calculating the geometric relationship between foreign matter and solder, which is achieved by using centroid coordinate distance calculation or regional overlap analysis to establish spatial distribution correlation indicators.

[0083] When multiple defect types are detected coexisting, the corresponding fusion path is selected according to the conflict type. For overflow and sand hole conflicts, a comprehensive score is generated by the ratio of the sand hole confidence to the number of overflows. For example, the confidence is multiplied by the number ratio and then compared with the preset threshold. If it is higher than the threshold, it is determined to be a sand hole defect. For overflow and foreign matter conflicts, the ROI image of the conflict area is intercepted and input into the pre-trained sub-model. For example, the ResNet-18 network is used to extract multi-scale features, and the probability of the presence of foreign matter is output for re-inspection. For solder and sand hole conflicts, the frequency of sand holes detected in all original images is counted. For example, it is set to output a sand hole defect when the number of detections exceeds 80% of the total number of viewing angles. For solder and foreign matter conflicts, the solder boundary is located through sub-pixel edge detection. For example, the Sobel operator is combined with the interpolation algorithm to obtain sub-pixel coordinates, and the Euclidean distance between the center of the foreign matter and the center of mass of the solder is calculated. When the distance is less than the preset threshold, it is determined to be a foreign matter defect.

[0084] Through the above technical solutions, this application effectively solves the problem of conflicting judgments when multiple defects coexist, avoiding misjudgments or missed detections due to feature overlap. In the sand hole and overflow conflict scenario, the probability of subjective misjudgment is reduced through quantitative assessment of confidence and frequency. In the foreign matter and solder conflict scenario, spatial positioning analysis is used to accurately distinguish the differences in attached foreign matter and solder morphology. In the multi-view sand hole detection scenario, statistical verification eliminates the interference of single-view false detections, significantly improving the detection accuracy of complex defect scenarios.

[0085] Specifically, such as Figures 1 to 2 As shown in the figure, when the conflict types are overflow defects and sand hole defects, a comprehensive defect score is generated as a weighted score by calculating the sand hole confidence and the ratio of overflow to number. When the comprehensive defect score is greater than the preset threshold, it is judged as a sand hole defect. Otherwise, the defect result output is that both sand hole defects and overflow defects exist.

[0086] The sand hole confidence refers to the quantified output value of the probability of the existence of the sand hole area by the segmentation network. It is implemented by the probability value output by the activation function of the last layer of the segmentation network. This parameter reflects the credibility of the sand hole detection results. The overflow ratio refers to the ratio of the number of times the overflow defect occurs in historical detections to the total number of detections. It is implemented by calculating the frequency of occurrence of defect types recorded in the statistical database. This indicator characterizes the persistence and stability characteristics of the overflow defect. The comprehensive defect score refers to the composite calculation value that integrates the sand hole confidence and the overflow ratio. It is implemented using a linear weighted summation formula. Its function is to establish a quantitative basis for defect priority determination. The preset threshold refers to a pre-set judgment threshold, which is determined through experimental data optimization and is used to divide the critical conditions of single defects and compound defects.

[0087] When the inspection system identifies both overflow defects and sand hole defects, it first extracts the confidence value of the current sand hole area from the segmentation network output. It also calculates the ratio of the number of overflow defects to the total number of inspections within the workpiece batch from historical inspection records. The sand hole confidence value and the overflow ratio are input into a weighted calculation formula. For example, the sand hole confidence value is multiplied by a weight coefficient of 0.7, and the overflow ratio is multiplied by a weight coefficient of 0.3, and then the sum is calculated to generate a comprehensive defect score. When this score exceeds the preset threshold, the current primary defect type is determined to be a sand hole defect; if it does not exceed the threshold, the workpiece is determined to have both defect types. This process can adapt to the quality control requirements of different production batches by dynamically adjusting the weight coefficient and threshold parameters.

[0088] This application effectively solves the judgment conflict problem in the scenario where multiple defects coexist. In the welding inspection of automobile brake hoses, when solder overflow and sand hole defects occur at the same time, it can accurately distinguish them based on real-time detection data and historical statistical information, avoiding the risk of missed detection caused by misjudgment of a single defect type, and increasing the accuracy of multi-defect detection from 78% of traditional methods to 94%. At the same time, it retains the complete judgment results of compound defects, providing reliable data support for subsequent quality analysis.

[0089] Specifically, such as Figures 1 to 2 As shown, when the conflict type is overflow defect and foreign body defect, the foreign body classification sub-model includes intercepting an ROI sub-image of preset pixel size based on the center of the conflict area, inputting the ROI sub-image into the pre-trained foreign body classification sub-model for multi-scale feature extraction, and outputting the probability of foreign body existence based on the multi-scale feature. When the probability of foreign body existence is higher than the preset threshold, it is determined to be a foreign body defect. Otherwise, the defect result is output as the existence of both foreign body defect and overflow defect.

[0090] A ROI sub-image is a local image region captured based on the center of the conflicting region. This is achieved using a fixed-size rectangular frame or a dynamically adjusted elliptical region to focus on the local area where potential foreign objects are located. Multi-scale feature extraction involves extracting macroscopic morphological and microscopic texture features from an image using convolution kernels or feature pyramid structures of varying scales. This is achieved using the Inception module or spatial pyramid pooling structure to capture the composite features of foreign objects at different resolutions. The probability of foreign object presence is the normalized probability value output by the classifier. The Softmax function or Sigmoid function is used to perform probability mapping on multi-scale features to quantitatively assess the likelihood of foreign object presence.

[0091] When the detection system identifies a conflict between an overflow defect and a foreign body defect, it first intercepts a local image of a fixed size based on the geometric center coordinates of the conflicting area. For example, a rectangular area of ​​64×64 pixels extending outward from the center point is used as the ROI sub-image to eliminate interference from the surrounding solder overflow area. Subsequently, the sub-image is input into the pre-trained foreign body classification sub-model, which extracts features of different scales through parallel convolution paths. For example, 3×3 and 5×5 convolution kernels are used to extract local details, and the global average pooling layer is combined to obtain macroscopic morphological features. After the multi-scale features are fused through the fully connected layer, the probability value of the presence of foreign matter is output through the Sigmoid function. When the probability value exceeds the preset threshold, for example, 0.85, it is judged as a foreign body defect; if it is lower than the threshold, it is judged as the presence of both foreign body defects and overflow defects.

[0092] This application effectively addresses the difficulty traditional detection methods face in accurately distinguishing between overflow and foreign object defects. By focusing on a local ROI and conducting multi-scale feature analysis, the system significantly reduces the interference of texture in the solder overflow area with foreign object detection, reducing the false positive rate from 32% in traditional methods to less than 8%. Furthermore, a decision-making mechanism based on probability thresholds enhances the system's ability to identify tiny foreign objects and blurred edges, enabling the detection of metal particles with a diameter of less than 0.5 mm, for example, improving detection robustness in complex defect scenarios.

[0093] Specifically, such as Figures 1 to 2 As shown, when the conflict type is solder defect and foreign body defect, the solder area image with the highest resolution viewing angle is selected to locate the solder area through sub-pixel edge detection, and the center coordinates of the foreign body are extracted. The relationship distance index between the foreign body and the solder is calculated based on the foreign body center coordinates and the centroid of the solder area. When the relationship distance index is less than the preset threshold, it is determined to be a foreign body defect. Otherwise, the defect result is output as the simultaneous existence of solder defect and foreign body defect.

[0094] The highest-resolution solder area image is the image with the highest pixel density, selected from images captured from multiple angles. This is achieved using an industrial camera with a resolution of 12 megapixels or higher. It provides a sufficiently clear image foundation for sub-pixel edge detection. Sub-pixel edge detection is an edge location technique that surpasses conventional pixel-level accuracy. It utilizes a gradient interpolation algorithm, such as the Sobel operator combined with cubic spline interpolation, to improve edge location accuracy to 0.1 pixel. It accurately captures the microscopic morphological characteristics of the solder. The relationship distance metric is a quantitative parameter formed by calculating the Euclidean distance between the center coordinates of the foreign object and the center of mass of the solder area. This metric is implemented using a two-point distance formula in a two-dimensional coordinate system. It establishes an objective basis for determining the spatial correlation of defects. The preset threshold is a critical value set based on the physical dimensions of the solder and the foreign object. For example, it can be set to 1.2 times the equivalent diameter of the solder area. It serves to distinguish between independent defects and coexisting defects.

[0095] When spatial overlap between solder defects and foreign body defects is detected, the highest-resolution viewing angle image is selected for sub-pixel edge detection, enabling accurate identification of the solder's contour boundary and calculation of its centroid coordinates. Simultaneously, the center coordinates output by the foreign body detection module and the solder's centroid coordinates are imported into the spatial relationship analysis model, and a relationship distance index is generated by calculating the Euclidean distance between the two. When this index is lower than the preset threshold, it indicates that the foreign body is within the solder's morphological range and is determined to be a single foreign body defect; when the index exceeds the threshold, it is determined that the two defects exist independently. For example, when a 0.3mm foreign body is detected on the solder surface, if the distance between it and the adjacent solder's centroid is 0.5mm and the preset threshold is 0.4mm, it is determined to be a defect in which the foreign body and solder coexist.

[0096] Through the above technical solution, this application can accurately distinguish the independent existence of solder and foreign matter on the solder surface. In detection scenarios where the spatial distance between the two is less than 1mm, the false positive rate is reduced from 32% of traditional methods to less than 5%. At the same time, by establishing a quantitative spatial relationship judgment standard, the subjective differences in manual re-inspection are avoided, making the detection results repeatable and verifiable.

[0097] Specifically, such as Figures 1 to 2 As shown, it also includes a fault-tolerant adjustment step. When the position deviation between the solder area image and the solder area image in the historical data is greater than a preset threshold, the image is dynamically adjusted through an affine transformation matrix in the image coordinate system to eliminate the position deviation.

[0098] Position deviation greater than a preset threshold means that an image registration algorithm calculates the translation, rotation, or scale difference between the current image and a historical reference image, using a Euclidean distance metric based on feature point matching. When the difference exceeds a pre-set tolerance, an adjustment mechanism is triggered. This threshold avoids unnecessary calculations caused by minor position fluctuations, ensuring system efficiency.

[0099] The affine transformation matrix is ​​a linear transformation model that includes translation, rotation, and scaling parameters. It is implemented by fitting the feature point correspondences between the current image and the reference image using the least squares method. By preserving the image's geometric characteristics, this matrix eliminates positional deviations while avoiding image content distortion, ensuring geometric consistency for subsequent defect detection.

[0100] The fault-tolerance adjustment step works by matching feature points between the real-time solder area image and the historical reference image, calculating the spatial transformation parameters between the two sets of images. If the parameter deviation exceeds a preset threshold, the current image is dynamically corrected based on the affine transformation matrix, including translation compensation, rotation correction, and scaling adjustment. The corrected image maintains spatial alignment with the historical data, ensuring the stability of the inspection algorithm across different batches of workpieces. This process is performed through online calculation, without interrupting the inspection process, ensuring the real-time performance of the system.

[0101] In some specific embodiments, the historical reference images can be samples from a library of manually calibrated images of qualified standard products, such as batches of workpiece images with stable welding process parameters. The specific parameters of the affine transformation can be dynamically updated using an optimization algorithm, such as calculating a sliding average of the transformation parameters over multiple consecutive image frames to suppress the influence of single-shot measurement noise.

[0102] Compared to existing technologies, traditional methods typically rely on mechanical positioning devices or fixed calibration templates for position calibration, which cannot cope with dynamic incoming material deviations. This solution uses adaptive correction in image space to achieve flexible compensation for position deviations without increasing hardware costs, effectively reducing the false detection rate caused by workpiece placement errors.

[0103] Through the above technical solution, this application solves the problem of image registration failure caused by fluctuations in the incoming workpiece position and avoids misjudgments caused by image offsets in the detection algorithm. Dynamic affine transformation enables adaptive alignment of the image space, allowing subsequent defect detection modules to operate stably based on geometrically consistent input data, significantly improving the system's adaptability to actual production environments.

[0104] A full appearance inspection system for the inner wire solder of automobile brake hose joints, comprising:

[0105] The angle collaborative imaging module uses several asymmetrically arranged cameras to synchronously capture solder area images to obtain multiple multi-angle solder area images; extracts the feature points of each solder area image and calculates the transformation matrix between the viewing angles, and generates a pixel-level aligned panoramic composite image through bilinear interpolation; it contains three 20-megapixel industrial cameras, a synchronous trigger controller and an image preprocessing unit to realize feature point extraction and panoramic synthesis.

[0106] The geometric defect detection module inputs the panoramic composite image into the feature extraction model for edge detection to obtain image contour information, calculates the area ratio of the solder contour area to the reference area and the curvature change rate based on the image contour information, and outputs the defect type as a geometric defect based on the area ratio and the curvature change rate; deploys the FPGA processing unit and contour analyzer of the Sobel edge detection algorithm to calculate the area ratio and the curvature mutation point.

[0107] The texture defect detection module uses a pre-trained segmentation network to determine the sand hole area and output a binary mask and confidence level for the sand hole area. It also calculates the texture variance and brightness deviation within the sand hole area and outputs the defect type as a texture defect based on the binary mask, confidence level, texture variance, and brightness deviation. It includes a U-Net segmentation network inference engine, an LBP feature extractor, and an HSV color analysis unit.

[0108] The dynamic fusion decision module determines the fusion path based on the defect type, dynamically fuses defects based on the specific defect forms within the defect type in different fusion paths, and outputs the final defect result. It also stores preset conflict handling rules and outputs the final defect result.

[0109] The above shows and describes the basic features, principles, and advantages of the present invention. It should be noted that the present invention is not limited to the above embodiments, which are only some embodiments. Without departing from the spirit and scope of the present invention, various improvements and supplements made are considered to be within the scope of protection of the present invention.

Claims

1. A method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint, characterized in that: The steps include: In the multi-angle collaborative imaging step, a plurality of asymmetrically arranged cameras are used to synchronously capture the solder area images to obtain a plurality of multi-angle solder area images; the feature points of each solder area image are extracted and the transformation matrix between the viewing angles is calculated, and a pixel-level aligned panoramic composite image is generated through bilinear interpolation; a geometric defect detection step, inputting the panoramic composite image into a feature extraction model to perform edge detection to obtain image contour information, calculating an area ratio of the solder contour area to a reference area and a curvature change rate based on the image contour information, and outputting a defect type as a geometric defect based on the area ratio and the curvature change rate; In the texture defect detection step, a sand hole area is determined by a pre-trained segmentation network and a binary mask and confidence score of the sand hole area are output. The texture variance and brightness deviation in the sand hole area are calculated, and the defect type is output as a texture defect based on the binary mask, confidence score, texture variance, and brightness deviation. Dynamic fusion decision step, determining the fusion path according to the defect type, dynamically fusing defects according to the specific defect forms in the defect type in different fusion paths, and outputting the final defect result; The dynamic fusion decision step also includes: when the output defect detection results include both geometric defects and texture defects, determining that there is a conflict, fusing according to the defect form, and when the conflict type is an overflow defect and a sand hole defect, calculating a weighted score to determine the defect form and outputting a result as a sand hole defect or the simultaneous presence of a sand hole defect and an overflow defect; when the conflict type is an overflow defect and a foreign matter defect, calling a foreign matter classification sub-model for re-inspection to determine the defect form, and outputting a result as a foreign matter defect or the simultaneous presence of a foreign matter defect and an overflow defect; when the conflict type is a solder defect and a sand hole defect, counting the number of sand hole detections in the original images of several solder areas, and outputting a sand hole defect when the number is greater than a preset threshold; when the conflict type is a solder defect and a foreign matter defect, determining the defect form through spatial positioning analysis and outputting a result as a foreign matter defect or the simultaneous presence of a solder defect and a foreign matter defect.

2. A method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: The geometric defect detection step includes performing Sobel edge detection on the panoramic composite image to obtain image contour information, performing contour closing according to the image contour information, and calculating the area ratio of the solder contour area to the reference area of ​​the closed area; The curvature change rate is calculated, and continuous pixels that exceed the preset change rate along the edge of the image contour are set as curvature mutation points according to a preset length. The number of curvature mutation points is calculated. When the area ratio exceeds the preset area threshold, the output defect type is a geometric defect and the defect form is an overflow defect; when the curvature mutation points are greater than the preset number of mutation points, the output defect type is a geometric defect and the defect form is a solder defect.

3. The method for full appearance inspection of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: The texture defect detection step includes inputting the panoramic synthetic image into a pre-trained segmentation network, outputting a binary mask of the sand hole area and a corresponding confidence level, performing dynamic threshold binarization processing on the binary mask, calculating the binarized result by mask mean and standard deviation, and calculating the area of ​​the connected domain in the sand hole area. When the confidence level is greater than a preset confidence threshold and the area of ​​the connected domain after binarization is greater than a preset minimum sand hole size, the defect type is output as a texture defect and the defect form is a sand hole defect. Multimodal feature analysis is performed in the sand hole area output by the segmentation network, texture variance is calculated by extracting local binary pattern features, and brightness deviation from the reference color is calculated by converting the image to the HSV color space. When both the texture variance and the brightness deviation are greater than the corresponding preset thresholds, the defect type is output as a texture defect and the defect form is a foreign matter defect.

4. A method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to claim 3, characterized in that: The dynamic threshold binarization processing includes performing edge detection on the panoramic composite image to obtain the sand hole contour, obtaining the sand hole area as a connected domain based on the image contour, calculating the area of ​​the connected domain, performing a morphological opening operation to filter out the noise area whose connected domain area is smaller than a preset pixel value, and dynamically adjusting the smoothing kernel size according to the area of ​​the sand hole area to apply a curvature smoothing algorithm to the sand hole contour to obtain a binarization processing result.

5. The method for full appearance inspection of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: When the conflict type is overflow defect and sand hole defect, a comprehensive defect score is generated as a weighted score by calculating the sand hole confidence and the ratio of overflow to number. When the comprehensive defect score is greater than the preset threshold, it is judged as a sand hole defect. Otherwise, the defect result output is that both sand hole defect and overflow defect exist.

6. The method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: When the conflict type is an overflow defect and a foreign body defect, the foreign body classification sub-model includes intercepting an ROI sub-image of preset pixel size based on the center of the conflict area, inputting the ROI sub-image into the pre-trained foreign body classification sub-model for multi-scale feature extraction, and outputting the probability of foreign body existence based on the multi-scale features. When the probability of foreign body existence is higher than a preset threshold, it is determined to be a foreign body defect. Otherwise, the defect result is output as the presence of both foreign body defect and overflow defect.

7. The method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: When the conflict type is solder defect and foreign body defect, the solder area image with the highest resolution viewing angle is selected to locate the solder area through sub-pixel edge detection, and the center coordinates of the foreign body are extracted. The relationship distance index between the foreign body and the solder is calculated based on the foreign body center coordinates and the center of mass of the solder area. When the relationship distance index is less than the preset threshold, it is judged as a foreign body defect. Otherwise, the defect result is output as the existence of both solder defect and foreign body defect.

8. The method for inspecting the overall appearance of the inner wire solder of an automobile brake hose joint according to claim 1, characterized in that: The method also includes a fault-tolerant adjustment step. When the position deviation between the solder area image and the solder area image in the historical data is greater than a preset threshold, the image is dynamically adjusted in the image coordinate system through an affine transformation matrix to eliminate the position deviation.

9. A system for inspecting the appearance of the inner wire solder of an automobile brake hose joint, applicable to the method for inspecting the appearance of the inner wire solder of an automobile brake hose joint according to any one of claims 1 to 8, characterized in that: include: The angle collaborative imaging module uses several asymmetrically arranged cameras to synchronously capture solder area images to obtain multiple multi-angle solder area images. It extracts the feature points of each solder area image and calculates the transformation matrix between the viewing angles, and generates a pixel-level aligned panoramic composite image through bilinear interpolation. a geometric defect detection module, inputting the panoramic composite image into a feature extraction model to perform edge detection to obtain image contour information, calculating an area ratio of the solder contour area to a reference area and a curvature change rate based on the image contour information, and outputting a defect type as a geometric defect based on the area ratio and the curvature change rate; The texture defect detection module uses a pre-trained segmentation network to determine the sand hole area and output a binary mask and confidence score for the sand hole area. It also calculates the texture variance and brightness deviation within the sand hole area and outputs the defect type as a texture defect based on the binary mask, confidence score, texture variance, and brightness deviation. The dynamic fusion decision module determines the fusion path according to the defect type, performs dynamic defect fusion according to the specific defect form in the defect type in different fusion paths, and outputs the final defect result.

Images