A wire harness crimping defect detection system and method based on machine vision

Abstract

The present invention belongs to the field of computer vision technology. The present invention discloses a wire harness crimping defect detection system and method based on machine vision, including a multi-spectral light source control module for using a ring-shaped LED light source and coaxial light for hybrid illumination, and adapting to different wire harness surface materials by dynamically adjusting the light source angle and intensity; a multi-dimensional perception acquisition module for real-time collecting wire harness image data and crimping geometric data based on the dynamically adjusted light source angle and intensity; a vision enhancement and fusion module for denoising the wire harness image data by using a non-local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of a smoothing factor, enhancing the texture contrast of the wire harness image data by superimposing a multi-scale Retinex algorithm, and fusing the wire harness image data and the crimping geometric data to construct a multi-modal feature map of the crimping part, which is helpful for the accuracy of defect detection and for timely discovering and solving potential problems.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology. More specifically, the present invention relates to a wire harness crimping defect detection system and method based on machine vision. Background Art

[0002] A patent with the publication number CN118760105A discloses a crimping detection system and method for wire harness connectors. The system includes: a parameter optimization module that collects crimping data of wire harness connectors based on a Gaussian process model and records the type of crimping terminals, the characteristics of wire types, and the corresponding crimping quality each time of crimping. In the present invention, by introducing data collection and analysis based on the Gaussian process model, the initial crimping parameters are optimized, the control accuracy of the contact resistance and crimping quality during the crimping process is improved, and a real-time monitoring and experimental feedback loop is adopted to ensure continuous update of the crimping parameters, significantly improving the crimping consistency and success rate. By finely adjusting the crimping force, speed, and time, the integrity and strength of the crimping interface are further ensured. These optimization measures significantly reduce the crimping defect rate, improve production efficiency and product reliability, and systematically collect and analyze crimping quality data, enabling the production process to quickly adapt to different production requirements and material changes.

[0003] The existing wire harness crimping defect detection systems and methods mainly have the following problems:

[0004] In the prior art, fixed light source configurations often cannot adapt to the reflection characteristics of different wire harness surface materials, resulting in overexposure or underexposure on the surfaces of some materials and affecting the image quality; in traditional lighting schemes, due to the fixation and mismatch of light sources, the difference in reflectivity of different materials may lead to too low image contrast, unable to highlight surface details and affecting subsequent defect detection; traditional lighting methods do not consider the diversity of wire harness surface materials, and the lighting effects on all surface materials under the same light source may not be ideal; many existing systems require manual adjustment of the angle and intensity of the light source, which is complex to operate and does not have self-adaptive capabilities; different surface materials have different lighting requirements, and using inappropriate light source configurations may result in uneven illumination on the wire harness surface, thus affecting the accuracy of defect detection.

[0005] In traditional image denoising methods, the details of the image, especially the edge details, are often ignored during the denoising process, resulting in blurred or distorted edges of the image; many existing denoising algorithms use fixed similarity metrics in the prior art and cannot make adaptive adjustments for different image regions. Especially in the case where there are different materials or textures in the image, the fixed metric method may not accurately reflect the similarity between regions; traditional denoising methods often lose some detail information during the denoising process. Especially when dealing with high-frequency noise during the denoising process, it is easy to remove important textures and structural information in the image at the same time; many existing denoising algorithms have unstable denoising effects when facing different types of noise, which may lead to incomplete removal of noise or the generation of new noise; some existing denoising methods may have a problem of excessive computational complexity when dealing with complex or large-scale images, and the denoising intensity of many existing algorithms is fixed and cannot be flexibly adjusted according to different image features;

[0006] In traditional image processing methods, the harness image may have brightness differences due to uneven illumination, affecting the accurate extraction of details; when conventional image enhancement methods enhance the contrast, it may lead to the loss of details or excessive blurring in the image; traditional image processing methods usually process images based on a single scale and cannot effectively process details of different sizes and different textures; in traditional methods, the intensity of contrast enhancement is usually adjusted globally, which may lead to over-enhancement or insufficient contrast in some regions of the image.

[0007] In view of this, the present invention proposes a harness crimping defect detection system and method based on machine vision to solve the above problems. Summary of the Invention

[0008] To overcome the above defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A harness crimping defect detection system based on machine vision, comprising:

[0009] A multi-spectral light source control module, which is used to adopt a hybrid illumination of a ring-shaped LED light source and coaxial light, and adapt to different harness surface materials by dynamically adjusting the light source angle and intensity;

[0010] A multi-dimensional perception acquisition module, which is based on the dynamically adjusted light source angle and intensity, and real-time acquires harness image data and crimping geometric data;

[0011] A visual enhancement fusion module, which is used to denoise the harness image data by using a non-local means denoising algorithm and controlling the similarity metric range through dynamic adjustment of the smoothing factor; superimpose a multi-scale Retinex algorithm to enhance the texture contrast of the harness image data through local contrast; fuse the harness image data and the crimping geometric data to construct a multi-modal feature map of the crimping part;

[0012] The crimping defect detection module is used to take the multi-modal feature map of the crimping part as the input of the crimping defect detection model, and predict the probability of crimping defects; according to the predicted probability of crimping defects, it is judged whether the wire harness crimping has defects;

[0013] The defect level classification module, if the wire harness crimping has defects, automatically collects the crimping defect data; inputs the crimping defect data into the trained defect level prediction model, and predicts the defect level of the wire harness crimping;

[0014] The real-time alarm module is used to classify the alarm level according to the defect level of the wire harness crimping, and trigger an audible and visual alarm to the defect detection terminal; each module is connected by wired and / or wireless means.

[0015] Preferably, the method of using a ring LED light source and coaxial light for mixed illumination and dynamically adjusting the light source to adapt to different wire harness surface materials includes:

[0016] Obtain the wire harness image data through the camera, classify the wire harness surface material, and preset the wire harness surface material categories as , where represents the copper core; represents the coating; represents the insulating layer; considering the reflectivity of the wire harness surface material, the wire harness surface material is divided into high reflectivity materials, medium reflectivity materials and low reflectivity materials; the reflectivity of the wire harness surface material is defined as: ; where is the reflectivity of the wire harness surface material; is the reflected light intensity; is the incident light intensity;

[0017] Preset the first reflectivity threshold and the second reflectivity threshold , if , then it is determined that the wire harness surface material is a high reflectivity material; if , then it is determined that the wire harness surface material is a medium reflectivity material; if , then it is determined that the wire harness surface material is a low reflectivity material; according to the imaging effects of the wire harness surface material under different light sources, different light source combinations are selected, and different light source combinations include ; where is the ring LED light source; is the coaxial light source;

[0018] After identifying the wire harness surface material and selecting different light source combinations, according to the current contrast ratio of the wire harness image data, dynamically adjust the light source angle and light source intensity through the light source angle adjustment formula and the light source intensity adjustment formula; the contrast ratio is: ; where is the maximum luminance value in the wire harness image data; is the minimum luminance value in the wire harness image data;

[0019] The light source angle adjustment formula is: ; where, is the adjusted light source angle; is the initial light source angle; is the light source angle adjustment factor; is the target range contrast; is the current range contrast;

[0020] The light source intensity adjustment formula is: ; where, is the adjusted light source intensity; is the initial light source intensity; is the reflectivity factor; is the reflectivity of the current wire harness surface material; is the light source intensity adjustment factor;

[0021] The reflectivity factor is restricted and constrained by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is: ; where, is the number of wire harness surface material categories; is the weight factor of the current range contrast; is the weight factor of the number of wire harness surface material categories.

[0022] Preferably, the wire harness image data includes a wire harness surface image, a crimp terminal image, and a crimping part image; the crimping geometry data includes the height, volume, crimping angle, thickness, and diameter of the terminal at the crimping part.

[0023] Preferably, the method for denoising the wire harness image data by using the non-local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor includes:

[0024] For each pixel point in the wire harness image data , calculate its edge intensity using an edge detection algorithm ; where, is the measure of the edge intensity; is each pixel point at the pixel value; is the abscissa of each pixel point; is the ordinate of each pixel point;

[0025] For any pair of pixel points in the image and , calculate the adaptive similarity measure through the similarity measure formula;

[0026] The similarity measure formula is: ; where represents the similarity measure between pixel points and ; is the pixel value of pixel point ; is the pixel value of pixel point ; is the gradient information at pixel point ; is the gradient information at pixel point ; is the adjustment factor for controlling the weights of gray - level difference and gradient information; is the smoothing factor for controlling the range of similarity measure;

[0027] Dynamically limit the smoothing factor for controlling the range of similarity measure through the smoothing factor limit formula. The smoothing factor limit formula is: ; where is the total number of pixels in the wire harness image data; is the sum of edge intensities; is the adjustment factor for controlling the influence of the total number of pixels in the wire harness image data on the smoothing factor; is the adjustment factor for controlling the influence of edge intensity on the smoothing factor;

[0028] Based on the edge intensity , modify the similarity measure by modifying the similarity measure formula to enhance the weight of the edge region;

[0029] The modified similarity measure formula is: ; where is the Euclidean distance between pixel points and ; is the edge intensity at pixel point ; is the adjustment factor for controlling the edge - weighting effect;

[0030] Use the non - local means denoising formula to perform weighted averaging according to the similarity weights of each pixel point and its neighboring pixels, thereby removing noise. The non - local means denoising formula is: ; where represents the value of pixel in the denoised wire harness image data; represents the position of the pixel being processed currently; represents pixel each pixel in the neighborhood ; represents the neighborhood pixel pixel value.

[0031] Preferably, the method for enhancing the texture contrast of the wire harness image data by the superimposed multi-scale Retinex algorithm includes;

[0032] Blur the wire harness image data through Gaussian filters of different scales to obtain multi-scale images, and each scale generates a blurred image; use convolution operation to achieve the blurring of the wire harness image data: wherein, represents the pixel value of the blurred image processed by the Gaussian filter of scale at the pixel point ; represents the Gaussian kernel function; represents the convolution operation;

[0033] For each scale , calculate the ratio between the pixel value at each pixel point and : ; wherein, is the ratio image at scale ;

[0034] Based on the ratio calculation, adjust the ratio image through the local contrast enhancement formula; the local contrast enhancement formula is: ; wherein, is the image after local contrast enhancement; is the parameter factor for adjusting the enhancement intensity;

[0035] Perform weighted summation on the ratio images from different scales, fuse the enhancement information of each scale, and obtain the multi-scale enhanced image; the multi-scale enhanced image is: ; wherein, is the multi-scale enhanced image; is the number of scales; is the scale under the ratio image; is the scale weight coefficient; is the th scale; is the scale index;

[0036] Restore the dynamic range of the multi-scale enhanced image using inverse logarithmic mapping. By performing exponential mapping on the multi-scale enhanced image, restore the multi-scale enhanced image to the brightness range of the wire harness image data.

[0037] Preferably, the method of fusing the wire harness image data with the crimping geometry data to construct the multi-modal feature map of the crimping part includes:

[0038] Apply the ICP algorithm to perform point cloud alignment on the wire harness image data and the crimping geometry data iteratively, continuously adjust the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometry data, and find the best transformation that minimizes the distance between the two sets of point clouds.

[0039] Preset an error threshold. During each iteration, the ICP algorithm calculates the distance error between the two sets of point clouds under the current transformation and determines whether the preset error threshold or the maximum number of iterations is reached. If so, end the iteration; after alignment, the point clouds of the wire harness image data and the crimping geometry data will be fused into a whole, thereby generating the multi-modal feature map of the crimping part.

[0040] Preferably, the method for constructing the crimping defect detection model includes:

[0041] Divide the data set into a training set, a validation set, and a test set; construct a crimping defect detection model. The sample set is a subset of the data set, and each sample set includes the historical multi-modal feature map of the crimping part and the corresponding crimping defect probability.

[0042] The crimping defect detection model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; the input layer is the historical multi-modal feature map of the crimping part, and the output layer is the crimping defect probability; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model.

[0043] Use the mean square error as the loss function to measure the error between the predicted value and the actual value of the model; use the training set to train the risk probability prediction model, update the model parameters through the backpropagation algorithm and the gradient descent method to minimize the loss function; use the validation set to evaluate the performance of the crimping defect detection model by calculating the coefficient of determination and tune the hyperparameters of the model.

[0044] Select the Adam optimization algorithm as the optimizer, adjust the hyperparameters of the model until the performance no longer improves or reaches the preset number of iterations and then stop; use the test set to evaluate the performance of the model in the prediction task, and use the trained crimping defect detection model to predict the current multi-modal feature map of the crimping part to obtain the crimping defect probability.

[0045] Preferably, the method for judging whether a defect occurs in the wire harness crimping according to the predicted crimping defect probability includes:

[0046] Set a preset crimping defect probability threshold, and compare the predicted crimping defect probability with the preset crimping defect probability threshold;

[0047] If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is determined that there is no defect in the wire harness crimping;

[0048] If the predicted crimping defect probability is greater than or equal to the preset crimping defect probability threshold, it is determined that there is a defect in the wire harness crimping.

[0049] Preferably, the training method of the defect level prediction model includes:

[0050] Divide the data set into a training set, a test set and a validation set, and construct a defect level prediction model. The defect level prediction model includes an input layer, a hidden layer and an output layer; the input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the wire harness crimping; the output layer is provided with neurons equal in number to the number of defect levels of the wire harness crimping, and each neuron corresponds to the prediction probability of a type of defect level of the wire harness crimping. The softmax function is used as the activation function; the defect level prediction model is a multi-layer perceptron MLP model;

[0051] Use multi-class cross-entropy as the loss function of the model to measure the difference between the predicted value and the actual value of the model; use the training set to train the error type recognition model, and update the model parameters through the backpropagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy index.

[0052] Select the SGD optimization algorithm as the optimizer, tune the model according to the performance feedback of the validation set, and adjust the model parameters until the performance no longer improves or reaches the preset stop condition; use the test set to evaluate the performance of the model in the prediction task, and use the trained defect level prediction model to predict the current crimping defect data to obtain the defect level of the wire harness crimping.

[0053] A wire harness crimping defect detection method based on machine vision includes:

[0054] S1. Adopt a combination of annular LED light source and coaxial light for illumination, and adapt to different wire harness surface materials by dynamically adjusting the light source angle and intensity;

[0055] S2. Based on the dynamically adjusted light source angle and intensity, collect wire harness image data and crimping geometric data in real time;

[0056] S3. Use the non-local means denoising algorithm to control the similarity measurement range by dynamically adjusting the smoothing factor, and denoise the wire harness image data; superimpose the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data with the crimping geometry data to construct a multi-modal feature map of the crimping part;

[0057] S4. Take the multi-modal feature map of the crimping part as the input of the crimping defect detection model, and predict the probability of crimping defects; judge whether there are defects in the wire harness crimping according to the predicted probability of crimping defects;

[0058] S5. If there are defects in the wire harness crimping, automatically collect the crimping defect data; input the crimping defect data into the trained defect level prediction model to predict the defect level of the wire harness crimping;

[0059] S6. Divide the alarm level according to the defect level of the wire harness crimping, and trigger an audible and visual alarm to the defect detection terminal.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] By dynamically adjusting the angle and intensity of the light source, the present invention enables the wire harness surface materials with different reflectivities to obtain the best lighting conditions, ensuring clear and accurate image data for wire harnesses made of high-reflectivity materials, medium-reflectivity materials, or low-reflectivity materials, and avoiding image blurring or detail loss caused by excessive or insufficient light; by adjusting the light source angle and intensity according to the extreme contrast of the wire harness image data, the dynamic range of the image can be optimized, enhancing image details, especially in the transition area between high-reflection and low-reflection surfaces, improving the image contrast, making the defects on the wire harness surface more obvious, and facilitating subsequent defect detection; by precisely adjusting the angle and intensity of the light source, the lighting effect in the image can be adapted to the needs of different materials, improving the presentation accuracy of image details. This is crucial for high-precision detection and analysis, especially when dealing with complex wire harness structures, and can improve the accuracy of the defect detection system; the system has an adaptive light source adjustment function, which can automatically adjust the light source parameters according to different wire harness surface materials and the reflection characteristics of the current image, reducing manual intervention; the hybrid lighting technology of the ring-shaped LED light source and the coaxial light source is adopted, which can combine the advantages of the two light sources and adapt to wire harnesses made of materials with different reflectivities;

[0062] By dynamically adjusting the smoothing factor in the similarity metric and combining it with the weight of edge intensity, the weight of the edge region is enhanced, enabling better preservation of edge information during the denoising process. This addresses the problem that traditional denoising methods often blur image details, especially edge details, thereby improving the quality of the denoised image and enhancing the edge sharpness of the image. An adaptive similarity metric formula is adopted, which adjusts its similarity metric according to the gray values and gradient information of different pixels by controlling the weights of gray difference and gradient information, thus more accurately identifying the noise regions and detail regions in the image. The smoothing factor that controls the similarity metric range is dynamically adjusted through the smoothing factor limit formula, enabling the algorithm to flexibly adjust the denoising intensity according to the different characteristics of the image. Based on the non-local means denoising formula, noise in the image is removed by weighted averaging the similarity weights of each pixel point and its neighboring pixels, while maintaining the detail part of the image, especially the preservation of high-frequency signals such as texture and edge parts is more prominent.

[0063] By introducing the multi-scale Retinex algorithm and blurring the image at each scale, the details at different levels in the image are effectively enhanced. Combined with local contrast enhancement, the texture contrast of the wire harness image is further improved, making the tiny details in the image more prominent, which is helpful for subsequent defect detection and quality assessment. By processing the wire harness image at multiple scales, it can adapt to details at different scales, thereby achieving more comprehensive and accurate enhancement. It can solve the problems of illumination differences and local feature prominence at different scales, and improve the clarity and contrast of the image at each scale. By introducing the local contrast enhancement formula, the method can finely adjust the contrast of the local area of the image, improve the detail performance, and avoid excessive adjustment of the overall brightness range. By weighted summing the image information at different scales, not only the detail information at each scale is fused, but also the weights of each scale in the final image can be adjusted according to the contribution values of different scales, making the enhancement effect more balanced. Brief Description of the Drawings

[0064] Figure 1 It is a schematic structural diagram of a wire harness crimping defect detection system based on machine vision of the present invention;

[0065] Figure 2 It is a schematic flow diagram of a wire harness crimping defect detection method based on machine vision of the present invention;

[0066] Figure 3 It is a flow chart of a method for constructing a multi-modal feature map of the crimping part provided by the present invention. Detailed Description of the Invention

[0067] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] Embodiment 1

[0069] Please refer to Figure 1 as shown, Embodiment 1 further illustrates a wire harness crimping defect detection system based on machine vision proposed by the present invention, including:

[0070] With the development of industrial automation and intelligence, wire harnesses are increasingly widely used in various electronic devices. Especially in the fields of automobiles, aviation, and electronic products, the quality of wire harnesses directly affects the safety and stability of the devices. The crimping quality of wire harnesses is crucial for the reliability of electrical connections. Therefore, it is particularly important to accurately and quickly detect wire harness crimping defects.

[0071] Currently, traditional wire harness crimping defect detection methods mostly rely on manual visual inspection or automatic detection based on traditional image processing. These methods have some significant limitations. For example, manual inspection has low efficiency, is easily interfered by human factors, and cannot meet the real-time requirements in large-scale production; while traditional automatic detection methods, especially image processing-based detection, are often affected by factors such as lighting conditions, differences in wire harness surface materials, and image noise, resulting in inaccurate detection results, even missed detections or false detections, and cannot meet the requirements of high precision and high efficiency.

[0072] In the prior art, although there are already some detection systems based on machine vision, they still face the following several problems:

[0073] Poor light source adaptability: Different wire harness surface materials (such as copper cores, coatings, insulation layers, etc.) have different light reflection characteristics, and traditional light sources cannot be flexibly adapted, resulting in poor image quality and affecting the accuracy of subsequent detections.

[0074] Noise interference: Noise generated in the image due to factors such as uneven lighting and environmental noise may have an adverse impact on the image quality. The effect of traditional image denoising methods is limited, and it is difficult to effectively remove noise in a complex background.

[0075] Low detection accuracy: Traditional defect detection algorithms usually rely on simple feature extraction and comparison, and it is difficult to accurately identify subtle crimping defects. Especially in complex crimping parts and variable wire harness surface materials, missed detections or false detections are likely to occur.

[0076] Poor real-time performance: In the environment of high-efficiency production lines, traditional wire harness defect detection methods often have slow processing speeds and cannot meet the requirements of real-time production monitoring, increasing the risks in the production process.

[0077] To effectively solve the above problems, the present invention proposes a wire harness crimping defect detection system based on machine vision, including:

[0078] A multi-spectral light source control module, which is used to adopt a hybrid lighting of a ring-shaped LED light source and coaxial light, and adapt to different wire harness surface materials by dynamically adjusting the light source angle and intensity;

[0079] A multi-dimensional perception acquisition module, which, based on the dynamically adjusted light source angle and intensity, real-time acquires wire harness image data and crimping geometric data;

[0080] A vision enhancement and fusion module, which is used to denoise the wire harness image data by using the non-local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor; superimpose the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data and the crimping geometric data to construct a multi-modal feature map of the crimping part;

[0081] A crimping defect detection module, which is used to take the multi-modal feature map of the crimping part as the input of the crimping defect detection model, and predict the crimping defect probability; judge whether there is a defect in the wire harness crimping according to the predicted crimping defect probability;

[0082] A defect level classification module, if there is a defect in the wire harness crimping, it automatically collects the crimping defect data; inputs the crimping defect data into the trained defect level prediction model to predict the defect level of the wire harness crimping;

[0083] A real-time alarm module, which is used to classify the alarm level according to the defect level of the wire harness crimping and trigger an audible and visual alarm to the defect detection terminal; each module is connected in a wired and / or wireless manner.

[0084] The method of adopting a hybrid lighting of a ring-shaped LED light source and coaxial light and dynamically adjusting the light source to adapt to different wire harness surface materials includes:

[0085] Acquire wire harness image data through a camera, classify the wire harness surface materials, and preset the wire harness surface material categories as , where represents the copper core; represents the plating layer; represents the insulating layer; considering the reflectivity of the wire harness surface materials, the wire harness surface materials are divided into high-reflectivity materials, medium-reflectivity materials, and low-reflectivity materials; the reflectivity of the wire harness surface materials is defined as: ; where is the reflectivity of the surface material of the wire harness; is the intensity of the reflected light; is the intensity of the incident light;

[0086] The first preset reflectivity threshold and the second reflectivity threshold , if , it is determined that the surface material of the wire harness is a high-reflectivity material; if , it is determined that the surface material of the wire harness is a medium-reflectivity material; if , it is determined that the surface material of the wire harness is a low-reflectivity material; according to the different imaging effects of the surface material of the wire harness under different light sources, different light source combinations are selected. Different light source combinations include ; among them, is a ring-shaped LED light source; is a coaxial light source. A coaxial light source is a light source illumination method in which the light source is collinear with the optical axis of the camera lens. That is to say, the light travels along the same axis as the viewing angle of the camera. The advantage of this illumination method is that it can reduce the interference of reflected light on the object surface, making the surface texture and details clearer, especially suitable for imaging tasks that require highlighting the details of the object surface;

[0087] After identifying the surface material of the wire harness and selecting different light source combinations, according to the current range contrast of the wire harness image data, the light source angle and light source intensity are dynamically adjusted through the light source angle adjustment formula and the light source intensity adjustment formula; the range contrast is: ; among them, is the maximum brightness value in the wire harness image data; is the minimum brightness value in the wire harness image data;

[0088] The light source angle adjustment formula is: ; among them, is the adjusted light source angle; is the initial light source angle (preset based on the wire harness surface material category ); is the light source angle adjustment factor, which mainly adjusts the angle of the light source, that is, how to adjust the angle of the light source relative to the wire harness to achieve the best imaging effect; is the target range contrast; is the current range contrast;

[0089] The light source intensity adjustment formula is: ; among them, is the adjusted light source intensity; is the initial light source intensity (preset based on the wire harness surface material category ); is the reflectivity factor, which is used to control the influence of reflectivity on the light source intensity; is the reflectivity of the current wire harness surface material; is the light source intensity adjustment factor, which mainly adjusts the intensity of the light source, that is, improves the poor contrast of the image by adjusting the brightness of the light source;

[0090] The reflectivity factor is restricted by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is: ; where, is the number of wire harness surface material categories; is the weight factor of the current poor contrast; is the weight factor of the number of wire harness surface material categories;

[0091] When the reflectivity is high, the light intensity reflected from the surface is strong, and it may be necessary to reduce the light source intensity to avoid overexposure; when the reflectivity is low, it may be necessary to increase the light source intensity to supplement the insufficient reflection;

[0092] The number of types of wire harness surface materials (such as copper, coating, insulation layer, etc.) affects the reflectivity and the quality of the final image. Under the influence of different materials, the light source intensity needs to be adjusted differently. By dynamically adjusting the weight factors of the number of material types and the poor contrast, the challenges of adjusting the light source intensity for multiple materials can be solved to ensure the adaptation of each material; the poor contrast of the image (i.e., the brightness difference of the image) also affects the adjustment of the light source intensity; a higher contrast may mean that no strong light source adjustment is required, while a lower contrast may require an increase in the light source intensity.

[0093] Under different reflectivity and image contrast conditions, the adjustment range of the light source intensity should be appropriately restricted. If the reflectivity factor is too large or too small, it may cause the image to be overexposed or too dark, affecting the quality. By setting the constraint of the reflectivity factor, excessive or insufficient adjustment can be avoided, ensuring that the light source intensity is always within a reasonable range;

[0094] For example, the current poor contrast is 10, the number of wire harness surface material categories is 3, the weight factor of the current poor contrast is 0.6; the weight factor of the number of wire harness surface material categories is 0.4; then the restricted reflectivity factor is .

[0095] The wire harness image data includes the wire harness surface image, the crimp terminal image, and the crimping part image; the crimping geometric data includes the height, volume, crimping angle, thickness and diameter of the terminal at the crimping part.

[0096] A method for denoising wire bundle image data using the non - local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor includes:

[0097] For each pixel point in the wire bundle image data , calculate its edge strength using an edge detection algorithm ; where is a measure of the edge strength; is the pixel value at each pixel point , that is, the brightness or color intensity of this point in the image; is the abscissa of each pixel point; is the ordinate of each pixel point;

[0098] For any pair of pixel points and in the image, calculate the adaptive similarity measurement through the similarity measurement formula, considering the pixel value difference and gradient information;

[0099] The similarity measurement formula is: ; where represents the similarity measurement between pixel points and , indicating the similarity of these two pixel points. The larger this value, the more similar these two pixels are, and the smaller it is, the greater their difference. During the denoising process, the similarity between pixel points determines their weights in the weighted average; is the pixel value of pixel point ; is the pixel value of pixel point ; is the gradient information at pixel point ; is the gradient information at pixel point ; is an adjustment factor for controlling the weights of gray - scale difference and gradient information; is a smoothing factor for controlling the similarity measurement range;

[0100] Dynamically limit the smoothing factor for controlling the similarity measurement range through the smoothing factor limit formula. The smoothing factor limit formula is: ; where is the total number of pixels in the wire bundle image data; is the sum of the edge strengths; is an adjustment factor for controlling the influence of the total number of pixels in the wire bundle image data on the smoothing factor; is an adjustment factor for controlling the influence of the edge strength on the smoothing factor;

[0101] For example, the total number of pixels in the wire harness image data is 100,000, and the sum of edge intensities is 20,000. The adjustment factor that controls the influence of the total number of pixels in the wire harness image data on the smoothing factor is 0.0001, and the adjustment factor that controls the influence of edge intensity on the smoothing factor is 1; then the restricted smoothing factor is .

[0102] Based on the edge intensity , modify the similarity measure by modifying the similarity measure formula to enhance the weight of the edge region, ensuring that the noise in the edge region is reduced while the details are retained;

[0103] Modify the similarity measure formula to: ; where is the Euclidean distance between pixel points and ; is the edge intensity at pixel point ; is the adjustment factor that controls the edge weighting effect;

[0104] Use the non-local means denoising formula to perform weighted averaging according to the similarity weights of each pixel point and its neighboring pixels, thereby removing noise; the non-local means denoising formula is: ; where represents the value of pixel in the wire harness image data after denoising, which is obtained by weighted averaging of the neighboring pixels around pixel , aiming to eliminate noise; represents the position of the pixel being processed currently, referring to a pixel point in the image, indicating the position in the image, and can be regarded as a point in the image coordinate system (but in the formula, the specific coordinate values are not written, and directly use to represent the position); represents each pixel in the neighborhood of pixel ; represents the pixel value of the neighboring pixel .

[0105] Overlay the multi-scale Retinex algorithm. The method for enhancing the texture contrast of the wire harness image data through local contrast includes;

[0106] Blur the wire harness image data through Gaussian filters of different scales to obtain multi-scale images, and each scale generates a blurred image; use convolution operations to achieve the blurring of the wire harness image data: Among them, represents the pixel value of the blurred image processed by the Gaussian filter with scale at the pixel point ; represents the Gaussian kernel function, which is a two-dimensional Gaussian function used for blurring the image; represents the convolution operation;

[0107] For each scale , calculate the ratio between the pixel value at each pixel point and : ; Among them, is the ratio image at scale , showing the enhancement effect of the image at this scale;

[0108] Based on the ratio calculation, adjust the ratio image through the local contrast enhancement formula to enhance the high-frequency details in the image; the local contrast enhancement formula is: ; Among them, is the image after local contrast enhancement; is the parameter factor for adjusting the enhancement intensity;

[0109] Perform a weighted sum of the ratio images from different scales, fuse the enhancement information of each scale, and obtain the multi-scale enhanced image; the multi-scale enhanced image is: ; Among them, is the multi-scale enhanced image, that is, the enhanced image obtained by weighted summing the ratio images at different scales; is the number of scales; is the ratio image at scale ; is the weight coefficient of scale ; is the th scale; is the index of the scale, with a value range from 1 to ;

[0110] Use the inverse logarithmic mapping to restore the dynamic range of the multi-scale enhanced image. By performing an exponential mapping on the multi-scale enhanced image, restore the multi-scale enhanced image to the brightness range of the wire harness image data.

[0111] Fuse the wire harness image data with the crimping geometry data. As Figure 3 shown, the method for constructing the multi-modal feature map of the crimping part includes:

[0112] Apply the ICP algorithm to align the point clouds of the wire harness image data and the crimping geometry data iteratively, continuously adjust the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometry data, and find the best transformation that minimizes the distance between the two sets of point clouds;

[0113] Preset an error threshold. During each iteration, the ICP algorithm calculates the distance error between the two sets of point clouds under the current transformation and determines whether the preset error threshold or the maximum number of iterations is reached. If so, the iteration ends; after the alignment is completed, the point clouds of the wire harness image data and the crimping geometry data will be fused into a whole, thereby generating a multi-modal feature map of the crimping part.

[0114] The method for constructing a crimping defect detection model includes:

[0115] Divide the data set into a training set, a validation set, and a test set; construct a crimping defect detection model. The sample set is a subset of the data set, and each sample set includes a historical multi-modal feature map of the crimping part and the corresponding crimping defect probability;

[0116] The crimping defect detection model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; the input layer is the historical multi-modal feature map of the crimping part, and the output layer is the crimping defect probability; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model;

[0117] Use the mean squared error as the loss function to measure the error between the predicted value and the actual value of the model; use the training set to train the risk probability prediction model, and update the model parameters through the backpropagation algorithm and the gradient descent method to minimize the loss function; use the validation set to evaluate the performance of the crimping defect detection model by calculating the coefficient of determination and tune the hyperparameters of the model;

[0118] Select the Adam optimization algorithm as the optimizer, adjust the hyperparameters of the model until the performance no longer improves or reaches the preset number of iterations and then stop; use the test set to evaluate the performance of the model in the prediction task, and use the trained crimping defect detection model to predict the current multi-modal feature map of the crimping part to obtain the crimping defect probability.

[0119] The method for judging whether a wire harness crimping has a defect according to the predicted crimping defect probability includes:

[0120] Preset a crimping defect probability threshold, and compare the predicted crimping defect probability with the preset crimping defect probability threshold;

[0121] If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is judged that the wire harness crimping has no defect;

[0122] If the predicted probability of crimping defect is greater than or equal to the preset crimping defect probability threshold, it is determined that a defect has occurred in the wire harness crimping.

[0123] The training method of the defect level prediction model includes:

[0124] Divide the data set into a training set, a test set, and a validation set, and construct a defect level prediction model. The defect level prediction model includes an input layer, a hidden layer, and an output layer; the input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the wire harness crimping; the output layer is set with neurons equal in number to the number of defect levels of the wire harness crimping, and each neuron corresponds to the prediction probability of a type of defect level of the wire harness crimping. The softmax function is used as the activation function; the defect level prediction model is a multi-layer perceptron MLP model.

[0125] Use multi-class cross-entropy as the loss function of the model to measure the difference between the predicted value and the actual value of the model; use the training set to train the error type recognition model, and update the model parameters through the backpropagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy metric.

[0126] Select the SGD optimization algorithm as the optimizer, tune the model according to the performance feedback of the validation set, and adjust the model parameters until the performance no longer improves or reaches the preset stop condition; use the test set to evaluate the performance of the model in the prediction task, and use the trained defect level prediction model to predict the current crimping defect data to obtain the defect level of the wire harness crimping.

[0127] The method for dividing the alarm level according to the defect level of the wire harness crimping includes: setting the corresponding alarm level according to the defect level, and each defect level corresponds to an alarm level. For example, a minor defect corresponds to a low-level alarm, a medium defect corresponds to a medium-level alarm, and a serious defect corresponds to a high-level alarm; once a defect is detected, the system will compare the defect level with the preset alarm level.

[0128] If the defect level exceeds or is equal to a certain specific alarm level, an alarm is triggered; when the alarm condition is met, the system will transmit the alarm information to the defect detection terminal through an audible and visual alarm device. The audible and visual alarm can be carried out in ways such as sound, flash, or screen prompt.

[0129] The preset crimping defect probability threshold is set by the staff. Different crimping defect probabilities are collected through the defect detection terminal, and the average value of multiple crimping defect probabilities is taken as the preset crimping defect probability threshold; similarly, the preset error threshold is set.

[0130] In this embodiment, by dynamically adjusting the angle and intensity of the light source, the best lighting conditions can be obtained for wire harness surface materials with different reflectivities, ensuring clear and accurate image data for high-reflectivity materials, medium-reflectivity materials, and low-reflectivity materials alike, and avoiding image blurring or detail loss caused by excessive or insufficient light; by adjusting the light source angle and intensity according to the range contrast of the wire harness image data, the dynamic range of the image can be optimized, image details can be enhanced, especially in the transition area between high-reflectivity and low-reflectivity surfaces, the contrast of the image is improved, making the defects on the wire harness surface more obvious and facilitating subsequent defect detection; by precisely adjusting the angle and intensity of the light source, the lighting effect in the image can be adapted to the needs of different materials, improving the presentation accuracy of image details. This is crucial for high-precision detection and analysis, especially when dealing with complex wire harness structures, and can improve the accuracy of the defect detection system; the system has an adaptive light source adjustment function that can automatically adjust the light source parameters according to different wire harness surface materials and the reflection characteristics of the current image, reducing manual intervention; the hybrid lighting technology of ring LED light source and coaxial light source is adopted, which can combine the advantages of the two light sources and adapt to wire harnesses with different reflectivity materials;

[0131] By dynamically adjusting the smoothing factor in the similarity metric and combining the weight of edge intensity, the weight of the edge region is enhanced, enabling better preservation of edge information during the denoising process. This solves the problem that traditional denoising methods often blur image details, especially edge details, thus improving the quality of the denoised image and enhancing the edge sharpness of the image; an adaptive similarity metric formula is adopted, which adjusts its similarity metric according to the gray values and gradient information of different pixels by controlling the weights of gray difference and gradient information, thereby more accurately identifying the noise regions and detail regions in the image. The smoothing factor that controls the similarity metric range is dynamically adjusted through the smoothing factor constraint formula, enabling the algorithm to flexibly adjust the denoising intensity according to different characteristics of the image; based on the non-local means denoising formula, noise in the image is removed by weighted averaging the similarity weights of each pixel and its neighboring pixels, while maintaining the detail part of the image, especially the preservation of high-frequency signals such as texture and edge parts is more prominent;

[0132] By introducing the multi-scale Retinex algorithm and blurring the images at each scale, the details at different levels in the images are effectively enhanced. Combining with local contrast enhancement, the texture contrast of the wire harness images is further improved, making the tiny details in the images more prominent, which is helpful for subsequent defect detection and quality assessment. By processing the wire harness images at multiple scales, it can adapt to the details at different scales, thus achieving more comprehensive and accurate enhancement. It can solve the problems of illumination differences and prominent local features at different scales, and improve the clarity and contrast of the images at each scale. By introducing the local contrast enhancement formula, the method can finely adjust the contrast of local regions in the images, improve the detail performance, and avoid excessive adjustment of the overall brightness range. By weighted summing the image information at different scales, not only the detail information at each scale is fused, but also the weights of each scale in the final image can be adjusted according to the contribution values at different scales, making the enhancement effect more balanced.

[0133] Embodiment 2

[0134] Please refer to Figure 2 As shown, for the parts not described in detail in this embodiment, refer to the description in Embodiment 1. A method for detecting wire harness crimping defects based on machine vision is provided, including:

[0135] S1. Adopt a hybrid illumination of a ring-shaped LED light source and coaxial light, and adapt to different wire harness surface materials by dynamically adjusting the light source angle and intensity;

[0136] S2. Based on the dynamically adjusted light source angle and intensity, collect wire harness image data and crimping geometric data in real time;

[0137] S3. Use the non-local means denoising algorithm to denoise the wire harness image data by dynamically adjusting the smoothing factor to control the similarity measurement range; superimpose the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data and the crimping geometric data to construct a multi-modal feature map of the crimping part;

[0138] S4. Take the multi-modal feature map of the crimping part as the input of the crimping defect detection model, and predict the crimping defect probability; judge whether there is a defect in the wire harness crimping according to the predicted crimping defect probability;

[0139] S5. If there is a defect in the wire harness crimping, automatically collect the crimping defect data; input the crimping defect data into the trained defect level prediction model to predict the defect level of the wire harness crimping;

[0140] S6. Divide the alarm level according to the defect level of the wire harness crimping, and trigger an audible and visual alarm to the defect detection terminal.

[0141] Since the electronic device introduced in this embodiment is the electronic device used in the system and method for detecting wire harness crimping defects based on machine vision in the embodiments of the present application, based on the system and method for detecting wire harness crimping defects based on machine vision introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the specific implementation of how this electronic device implements the method in the embodiments of the present application will not be described in detail here. As long as those skilled in the art implement the electronic device used in the system and method for detecting wire harness crimping defects based on machine vision in the embodiments of the present application, it falls within the scope of protection of the present application.

[0142] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to obtain a formula that is closest to the actual situation. The preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.

[0143] The above are only the preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the concept of the present invention belong to the scope of protection of the present invention. It should be noted that for ordinary technical users in the technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the scope of protection of the present invention.

Claims

1. A wire harness crimping defect detection system based on machine vision, characterized in that, Including: A multi - spectral light source control module, which is used to adopt a hybrid illumination of a ring - shaped LED light source and coaxial light, and adapt to different wire harness surface materials by dynamically adjusting the light source angle and intensity; A multi - dimensional perception acquisition module, which is based on the dynamically adjusted light source angle and intensity, and acquires wire harness image data and crimping geometric data in real time; A vision enhancement and fusion module, which is used to denoise the wire harness image data by using the non - local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor; Overlaying the multi - scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fusing the wire harness image data and the crimping geometric data to construct a multi - modal feature map of the crimping part; A crimping defect detection module, which is used to take the multi - modal feature map of the crimping part as the input of the crimping defect detection model, predict the probability of crimping defects, and judge whether there are defects in the wire harness crimping according to the predicted probability of crimping defects; A defect level classification module, if there are defects in the wire harness crimping, it automatically collects crimping defect data, inputs the crimping defect data into the trained defect level prediction model, and predicts the defect level of the wire harness crimping; A real - time alarm module, which is used to classify the alarm level according to the defect level of the wire harness crimping and trigger an audible and visual alarm to the defect detection terminal; The method of adopting a hybrid illumination of a ring - shaped LED light source and coaxial light, dynamically adjusting the light source, and adapting to different wire harness surface materials includes: Obtain the wire harness image data through the camera, classify the surface materials of the wire harness, and preset the surface material categories of the wire harness as , where represents the copper core; represents the plating; represents the insulation layer; Considering the reflectivity of the wire harness surface materials, the wire harness surface materials are divided into high reflectivity materials, medium reflectivity materials, and low reflectivity materials; The reflectivity of the wire harness surface material is defined as: ; where is the reflectivity of the wire harness surface material; is the reflected light intensity; is the incident light intensity; Preset first reflectivity threshold and second reflectivity threshold , if , then it is determined that the surface material of the wire harness is a high-reflectivity material; if , then it is determined that the surface material of the wire harness is a medium-reflectivity material; if , then it is determined that the surface material of the wire harness is a low-reflectivity material; according to the imaging effects of the surface material of the wire harness under different light sources, different light source combinations are selected. Different light source combinations include ; where is a ring-shaped LED light source; is a coaxial light source; After identifying the surface material of the wire harness and selecting different light source combinations, the light source angle and intensity are dynamically adjusted according to the current extreme contrast of the wire harness image data through the light source angle adjustment formula and the light source intensity adjustment formula; the extreme contrast is: ; where is the maximum brightness value in the wire harness image data; is the minimum brightness value in the wire harness image data; The light source angle adjustment formula is as follows: ; where is the adjusted light source angle; is the initial light source angle; is the light source angle adjustment factor; is the target contrast ratio; is the current contrast ratio; The formula for adjusting the light source intensity is as follows: ; where is the adjusted light source intensity; is the initial light source intensity; is the reflectivity factor; is the reflectivity of the current wire harness surface material; is the light source intensity adjustment factor; The reflectivity factor is restricted and constrained by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is as follows: ; where is the number of wire harness surface material categories; is the weight factor of the current range contrast; is the weight factor of the number of wire harness surface material categories.

2. The wire harness crimping defect detection system based on machine vision according to claim 1, characterized in that, The wire harness image data includes the wire harness surface image, the crimping terminal image, and the crimping part image; the crimping geometric data includes the height, volume, crimping angle, thickness, and diameter of the terminal of the crimping part.

3. The wire harness crimping defect detection system based on machine vision according to claim 2, wherein The method of denoising the wire harness image data by using the non - local means denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor includes: For each pixel in the wire harness image data , calculate its edge intensity using an edge detection algorithm ; where is a measure of the edge intensity; is the pixel value at each pixel ; is the abscissa of each pixel is the ordinate of each pixel For any pair of pixel points in the image and , calculate the adaptive similarity metric through the similarity metric formula; The similarity metric formula is as follows: ; where represents the similarity metric between pixel points and ; is the pixel value of pixel point ; is the pixel value of pixel point ; is the gradient information at pixel point ; is the gradient information at pixel point ; is the adjustment factor for controlling the weights of gray - level difference and gradient information; is the smoothing factor for controlling the range of similarity metric; The smoothing factor that restricts the similarity metric range is dynamically restricted by the smoothing factor restriction formula. The smoothing factor restriction formula is as follows: ; where ; among which is the total number of pixels in the wire harness image data; is the sum of edge intensities; is the adjustment factor that controls the influence of the total number of pixels in the wire harness image data on the smoothing factor; is the adjustment factor that controls the influence of edge intensity on the smoothing factor. Based on edge intensity , modify the similarity measure by modifying the similarity measure formula to enhance the weight of the edge region; Modify the similarity metric formula to be: ; where is the Euclidean distance between pixel points and ; is the edge intensity at pixel point ; is the adjustment factor that controls the edge weighting effect. Use the non-local means denoising formula to perform weighted averaging based on the similarity weights of each pixel and its neighboring pixels, thereby removing noise; the non-local means denoising formula is: ; where represents the value of the pixel in the denoised wire harness image data ; represents the position of the pixel currently being processed; represents each pixel in the neighborhood of the pixel ; represents the pixel value of the neighboring pixel ; 4. A wire harness crimping defect detection system based on machine vision according to claim 3, characterized in that The method of overlaying the multi - scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast includes; Blur the wire harness image data through Gaussian filters of different scales to obtain multi-scale images, and for each scale generate a blurred image; implement the blurring of the wire harness image data using convolution operations: where represents the pixel value of the blurred image processed by the Gaussian filter at scale at pixel point ; represents the Gaussian kernel function; represents the convolution operation; For each scale , calculate the ratio between the pixel value at each pixel and : ; where is the ratio image at scale ; Based on the ratio calculation, the ratio image is adjusted through the local contrast enhancement formula; the local contrast enhancement formula is: ; where is the image after local contrast enhancement; is the parameter factor for adjusting the enhancement intensity; Weighted summation is performed on ratio images from different scales to fuse the enhanced information at each scale, obtaining a multi-scale enhanced image; the multi-scale enhanced image is: ; where is the multi-scale enhanced image; is the number of scales; is the scale under the ratio image; is the scale weight coefficient; is the th scale; is the scale index; Using inverse logarithmic mapping to restore the dynamic range of the multi - scale enhanced image, and through exponential mapping of the multi - scale enhanced image, restoring the multi - scale enhanced image to the brightness range of the wire harness image data.

5. The wire harness crimping defect detection system based on machine vision according to claim 4, characterized in that, The method of fusing the wire harness image data and the crimping geometric data to construct a multi - modal feature map of the crimping part includes: Applying the ICP algorithm to perform point cloud alignment on the wire harness image data and the crimping geometric data iteratively, continuously adjusting the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometric data, and finding the best transformation that minimizes the distance between the two sets of point clouds; Presetting an error threshold. In each iteration process, the ICP algorithm calculates the distance error between the two sets of point clouds under the current transformation and judges whether it reaches the preset error threshold or the maximum number of iterations. If so, the iteration ends; after alignment, the point clouds of the wire harness image data and the crimping geometric data will be fused into a whole, thus generating a multi - modal feature map of the crimping part.

6. The wire harness crimping defect detection system based on machine vision according to claim 5, characterized in that The construction method of the crimping defect detection model includes: Divide the dataset into a training set, a validation set, and a test set; construct a crimping defect detection model. The sample set is a subset of the dataset, and each sample set includes the multi-modal feature map of the historical crimping part and the corresponding crimping defect probability. The crimping defect detection model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; the input layer is the multi-modal feature map of the historical crimping part, and the output layer is the crimping defect probability; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model. Use the mean squared error as the loss function to measure the error between the predicted value and the actual value of the model; use the training set to train the hazard probability prediction model, and update the model parameters through the backpropagation algorithm and the gradient descent method to minimize the loss function; use the validation set to evaluate the performance of the crimping defect detection model by calculating the coefficient of determination, and tune the hyperparameters of the model. Select the Adam optimization algorithm as the optimizer, adjust the hyperparameters of the model until the performance no longer improves or reaches the preset number of iterations and then stop; use the test set to evaluate the performance of the model in the prediction task, and use the trained crimping defect detection model to predict the multi-modal feature map of the current crimping part to obtain the crimping defect probability.

7. A wire harness crimping defect detection system based on machine vision according to claim 6, characterized in that, The method for judging whether a harness crimping has a defect according to the predicted crimping defect probability includes: Preset a crimping defect probability threshold, and compare the predicted crimping defect probability with the preset crimping defect probability threshold. If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is judged that the harness crimping has no defect. If the predicted crimping defect probability is greater than or equal to the preset crimping defect probability threshold, it is judged that the harness crimping has a defect.

8. A wire harness crimping defect detection system based on machine vision according to claim 7, characterized in that, The training method of the defect level prediction model includes: Divide the dataset into a training set, a test set, and a validation set, construct a defect level prediction model. The defect level prediction model includes an input layer, a hidden layer, and an output layer; the input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the harness crimping; the output layer sets neurons equal to the number of defect levels of the harness crimping, and each neuron corresponds to the prediction probability of a type of defect level of the harness crimping, and uses the softmax function as the activation function; the defect level prediction model is a multi-layer perceptron MLP model. Use multi-class cross-entropy as the loss function of the model to measure the difference between the predicted value and the actual value of the model; use the training set to train the error type recognition model, and update the model parameters through the backpropagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy index. Select the SGD optimization algorithm as the optimizer, tune the model according to the performance feedback of the validation set, and adjust the model parameters until the performance no longer improves or reaches the preset stop condition and then stop; use the test set to evaluate the performance of the model in the prediction task, and use the trained defect level prediction model to predict the current crimping defect data to obtain the defect level of the harness crimping.

9. A wire harness crimping defect detection method based on machine vision, which is used to implement a wire harness crimping defect detection system according to any one of claims 1 to 8, characterized in that, Include: S1. Adopt a hybrid lighting of a ring-shaped LED light source and coaxial light, and adapt to different surface materials of wire harnesses by dynamically adjusting the light source angle and intensity. S2. Based on the dynamically adjusted light source angle and intensity, collect wire harness image data and crimp geometry data in real time. S3. Use the non-local means denoising algorithm to denoise the wire harness image data by dynamically adjusting the similarity metric range through a smoothing factor. Overlay the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data and the crimp geometry data to construct a multi-modal feature map of the crimping part. S4. Use the multi-modal feature map of the crimping part as the input of the crimping defect detection model to predict the probability of crimping defects; judge whether there are defects in the wire harness crimping according to the predicted probability of crimping defects. S5. If there are defects in the wire harness crimping, automatically collect the crimping defect data; input the crimping defect data into the trained defect level prediction model to predict the defect level of the wire harness crimping. S6. Divide the alarm level according to the defect level of the wire harness crimping, and trigger an audible and visual alarm to the defect detection terminal.

Images