Wire harness crimping defect detection system and method based on machine vision

Abstract

The invention belongs to the technical field of computer vision, and discloses a wire harness crimping defect detection system and method based on machine vision, and the system comprises a multispectral light source control module which is used for employing an annular LED light source and coaxial light mixed illumination and adapting to different wire harness surface materials by dynamically adjusting the angle and intensity of the light source; the multi-dimensional perception acquisition module is used for acquiring wire harness image data and crimping geometric data in real time based on the dynamically adjusted light source angle and intensity; the visual enhancement fusion module is used for de-noising the wire harness image data by using a non-local mean de-noising algorithm and dynamically adjusting and controlling a similarity measurement range through a smooth factor; superposing a multi-scale Retinex algorithm, and enhancing the texture contrast of the wire harness image data through the local contrast; the wire harness image data and the crimping geometric data are fused, and a crimping part multi-mode characteristic graph is constructed; the accuracy of defect detection is facilitated, and potential problems can be found and solved in time.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and more specifically, to a wire harness crimping defect detection system and method based on machine vision. Background Art

[0002] The patent with the patent publication number CN118760105A discloses a crimping detection system and method for wire harness connector processing. The system includes: a parameter optimization module based on a Gaussian process model, which collects crimping data of the wire harness connector and records the crimping terminal type, wire type characteristics and corresponding crimping quality during each crimping. In the present invention, by introducing data collection and analysis based on a Gaussian process model, the initial crimping parameters are optimized, the control accuracy of contact resistance and crimping quality during the crimping process is improved, and real-time monitoring and experimental feedback loops are used to ensure that the crimping parameters are continuously updated, significantly improve the crimping consistency and success rate, and finely adjust the crimping force, speed and time to further ensure the integrity and strength of the crimping interface. These optimization measures significantly reduce the crimping defect rate, improve production efficiency and product reliability, and systematically collect and analyze crimping quality data, so that the production process can quickly adapt to different production needs and material changes.

[0003] The existing wire harness crimping defect detection system and method mainly have the following problems: In the existing technology, the fixed light source configuration often cannot adapt to the reflective characteristics of different wire harness surface materials, resulting in overexposure or underexposure of some material surfaces, affecting image quality; in traditional lighting solutions, due to the fixed and unsuitable light source, the difference in reflectivity of different materials may cause the image contrast to be too low, unable to highlight surface details, affecting subsequent defect detection; traditional lighting methods do not take into account the diversity of wire harness surface materials, and the lighting effect of all surface materials under the same light source may not be ideal; many existing systems require manual adjustment of the angle and intensity of the light source, which is complicated to operate and does not have adaptive capabilities; different surface materials have different lighting requirements, and the use of inappropriate light source configuration may result in uneven lighting on the wire harness surface, thereby affecting the accuracy of defect detection; In traditional image denoising methods, image details, especially edge details, are often ignored during the denoising process, resulting in blurred or distorted image edges; many denoising algorithms in the prior art use fixed similarity metrics and cannot make adaptive adjustments for different image regions. In particular, when there are different materials or textures in the image, fixed metrics may not accurately reflect the similarities between regions; traditional denoising methods often lose some detail information during the denoising process, especially when dealing with high-frequency noise during the denoising process, which can easily remove important texture and structural information in the image at the same time; many existing denoising algorithms are unstable in the face of different types of noise, which may result in incomplete noise removal or the generation of new noise; some existing denoising methods may have the problem of excessive computational effort when dealing with complex or large-scale images, and many existing algorithms have fixed denoising strengths and cannot be flexibly adjusted according to different image features; In traditional image processing methods, line beam images may have brightness differences due to uneven lighting, affecting the accurate extraction of details; conventional image enhancement methods may cause loss of details or excessive blurring in the image when enhancing contrast; traditional image processing methods usually process images based on a single scale and cannot effectively process details of different sizes and textures; in traditional methods, the intensity of contrast enhancement is usually adjusted globally, which may cause certain areas of the image to be over-enhanced or have insufficient contrast.

[0004] In view of this, the present invention proposes a wire harness crimping defect detection system and method based on machine vision to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a wire harness crimping defect detection system based on machine vision, comprising: Multi-spectral light source control module, which uses a ring-shaped LED light source and coaxial light mixed lighting, and dynamically adjusts the light source angle and intensity to adapt to different harness surface materials; Multi-dimensional sensing acquisition module, which collects wire harness image data and crimping geometry data in real time based on dynamically adjusted light source angle and intensity; The visual enhancement fusion module is used to denoise the wire harness image data by using the non-local mean denoising algorithm and dynamically adjusting the smoothing factor to control the similarity measurement range; superimpose the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data with the crimping geometry data to construct a multimodal feature map of the crimping part; The crimping defect detection module is used to use the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the probability of crimping defects; based on the predicted crimping defect probability, it is determined whether the wire harness crimping has defects; Defect grade classification module: if there is a defect in the crimping of the wire harness, the crimping defect data will be automatically collected; the crimping defect data will be input into the trained defect grade prediction model to predict the defect grade of the crimping of the wire harness; The real-time alarm module is used to classify the alarm level according to the defect level of the wire harness crimping and trigger the sound and light alarm to the defect detection terminal; each module is connected by wire and / or wireless means.

[0006] Preferably, the method of using a ring-shaped LED light source and coaxial light mixed lighting, dynamically adjusting the light source, and adapting to different harness surface materials includes: The wire harness image data is obtained through the camera, and the wire harness surface material is classified. The preset wire harness surface material categories are ,in, Indicates copper core; Indicates coating; Indicates the insulation layer; considering the reflectivity of the harness surface material, the harness surface material is divided into high reflectivity material, medium reflectivity material and low reflectivity material; the reflectivity of the harness surface material is defined as: ;in, is the reflectivity of the harness surface material; is the reflected light intensity; is the incident light intensity; Preset reflectivity first threshold and reflectivity second threshold ,like , then the surface material of the harness is determined to be a high reflectivity material; if , then the surface material of the harness is determined to be a medium reflectivity material; if , then the surface material of the wire harness is determined to be a low reflectivity material; according to the different imaging effects of the surface material of the wire harness under different light sources, different light source combinations are selected. Different light source combinations include ;in, It is a ring-shaped LED light source; It is a coaxial light source; After identifying the surface material of the wire harness and selecting different light source combinations, the light source angle and light source intensity are dynamically adjusted according to the current extreme contrast of the wire harness image data through the light source angle adjustment formula and the light source intensity adjustment formula; the extreme contrast is: ;in, is the maximum brightness value in the line beam image data; is the minimum brightness value in the line beam image data; The light source angle adjustment formula is: ;in, is the angle of the light source after adjustment; is the initial light source angle; is the light source angle adjustment factor; is the target extreme contrast; is the current extreme contrast; The light source intensity adjustment formula is: ;in, is the intensity of the light source after adjustment; is the initial light source intensity; is the reflectivity factor; is the reflectivity of the current harness surface material; is the light source intensity adjustment factor; The reflectivity factor is restricted by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is: ;in, is the number of material categories on the harness surface; is the weight factor of the current extreme contrast; A weighting factor for the number of material categories on the harness surface.

[0007] Preferably, the wire harness image data includes a wire harness surface image, a crimping terminal image and a crimping location image; the crimping geometry data includes the height, volume, crimping angle, thickness and diameter of the crimping location.

[0008] Preferably, the method of denoising the line beam image data by using a non-local mean denoising algorithm and controlling the similarity measurement range by dynamically adjusting a smoothing factor comprises: For each pixel in the line beam image data , use edge detection algorithm to calculate its edge strength ;in, is a measure of edge strength; For each pixel The pixel value at ; is the horizontal coordinate of each pixel; is the ordinate of each pixel; For any pair of pixels in the image and , the adaptive similarity measure is calculated by the similarity measure formula; The similarity measurement formula is: ;in, Represents pixel and Similarity measure between ; Pixel The pixel value of Pixel The pixel value of Pixel Gradient information at ; Pixel Gradient information at ; It is the adjustment factor to control the grayscale difference and gradient information weight; is a smoothing factor to control the range of similarity measurement; The smoothing factor that controls the range of similarity measurement through the smoothing factor restriction formula Dynamic limitation is performed, and the smoothing factor limitation formula is: ;in, is the total number of pixels of the line beam image data; is the sum of edge strengths; is a regulating factor for controlling the influence of the total number of pixels of the line beam image data on the smoothing factor; It is a regulating factor to control the effect of edge strength on the smoothing factor; Based on edge strength ,The similarity measure is modified to enhance the weight of the edge area by modifying the similarity measure formula; Modify the similarity measurement formula to: ;in, Pixel and The Euclidean distance between Pixel The edge strength at is a regulatory factor to control the influence of edge weighting; The non-local mean denoising formula is used to remove noise by performing weighted averaging based on the similarity weights of each pixel and its neighboring pixels. The non-local mean denoising formula is: ;in, Represents the pixels in the denoised line image data The value of Indicates the pixel position currently being processed; Represents pixels Every pixel in the neighborhood of ; Represents the neighborhood pixels The pixel value of .

[0009] Preferably, the method of superimposing a multi-scale Retinex algorithm to enhance the texture contrast of line beam image data by local contrast comprises: The line beam image data is blurred by Gaussian filters of different scales to obtain multi-scale images. Generate a blurred image; use a convolution operation to blur the line image data: in, Indicates the scale The blurred image after Gaussian filter processing is at pixel point The pixel value on ; represents the Gaussian kernel function; Represents the convolution operation; For each scale , calculate each pixel The pixel value at and The ratio between: ;in, For scale Ratio image below; Based on the ratio calculation, the contrast image is adjusted by the local contrast enhancement formula; the local contrast enhancement formula is: ;in, is the image after local contrast enhancement; is a parameter factor for adjusting the enhancement strength; The ratio images from different scales are weighted and summed, and the enhancement information of each scale is integrated to obtain a multi-scale enhanced image; the multi-scale enhanced image is: ;in, Enhance images at multiple scales; is the number of scales; For scale Ratio image below; For scale The weight coefficient of For the scale; is the index of the scale; The dynamic range of the multi-scale enhanced image is restored by using inverse logarithmic mapping, and the multi-scale enhanced image is restored to the brightness range of the line beam image data by performing exponential mapping on the multi-scale enhanced image.

[0010] Preferably, the method of fusing the wire harness image data with the crimping geometry data to construct a multimodal feature map of the crimping part comprises: Apply the ICP algorithm to iteratively align the point clouds of the wire harness image data and the crimping geometry data, continuously adjust the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometry data, and find the optimal transformation that minimizes the distance between the two sets of point clouds; The error threshold is preset. During each iteration, the ICP algorithm calculates the distance error between the two groups of point clouds under the current transformation and determines whether the preset error threshold or the maximum number of iterations is reached. If reached, the iteration is terminated. After the alignment is completed, the point cloud of the wire harness image data and the crimping geometry data will be fused into a whole, thereby generating a multimodal feature map of the crimping part.

[0011] Preferably, the method for constructing the crimping defect detection model includes: The data set is divided into a training set, a validation set, and a test set; a crimping defect detection model is constructed, where the sample set is a subset of the data set, and each sample set includes a multimodal feature map of the historical crimping parts and the corresponding crimping defect probability; The crimping defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is a multimodal feature map of historical crimping parts, and the output layer is the crimping defect probability; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model; The mean square error is used as the loss function to measure the error between the model's predicted value and the actual value. The hazard probability prediction model is trained using the training set, and the model parameters are updated through the back propagation algorithm and the gradient descent method to minimize the loss function. The performance of the crimping defect detection model is evaluated by calculating the coefficient of determination using the validation set, and the hyperparameters of the model are tuned. The Adam optimization algorithm is selected as the optimizer, and the hyperparameters of the model are adjusted until the performance no longer improves or the preset number of iterations is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained crimping defect detection model is used to predict the multimodal feature map of the current crimping part to obtain the crimping defect probability.

[0012] Preferably, the method for judging whether a defect occurs in the crimping of the wiring harness according to the predicted probability of crimping defects comprises: A crimping defect probability threshold is preset, and the predicted crimping defect probability is compared with the preset crimping defect probability threshold; If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is determined that no defect occurs in the crimping of the wire harness; If the predicted crimping defect probability is greater than or equal to a preset crimping defect probability threshold, it is determined that the wire harness crimping has a defect.

[0013] Preferably, the training method of the defect level prediction model includes: The data set is divided into a training set, a test set and a validation set, and a defect level prediction model is constructed. The defect level prediction model includes an input layer, a hidden layer and an output layer. The input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the wire harness crimping. The output layer is set with neurons equal to the number of defect levels of the wire harness crimping, each neuron corresponds to the prediction probability of a defect level type of the wire harness crimping, and a softmax function is used as an activation function. The defect level prediction model is a multi-layer perceptron MLP model. Use multi-classification cross entropy as the model's loss function to measure the difference between the model's predicted value and the actual value; use the training set to train the error type recognition model, and update the model parameters through the back propagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy index; The SGD optimization algorithm is selected as the optimizer, and the model is tuned according to the performance feedback of the validation set. The model parameters are adjusted until the performance no longer improves or the preset stopping condition is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained defect level prediction model is used to predict the current crimping defect data to obtain the defect level of the wire harness crimping.

[0014] A wire harness crimping defect detection method based on machine vision, comprising: S1. Adopting mixed lighting of annular LED light source and coaxial light, dynamically adjusting the angle and intensity of the light source to adapt to different harness surface materials; S2, based on the dynamically adjusted light source angle and intensity, real-time collection of wire harness image data and crimping geometry data; S3. Using the non-local mean denoising algorithm, the similarity measurement range is controlled by dynamically adjusting the smoothing factor to denoise the wire harness image data; the multi-scale Retinex algorithm is superimposed to enhance the texture contrast of the wire harness image data by local contrast; the wire harness image data is fused with the crimping geometry data to construct a multimodal feature map of the crimping part; S4, using the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the crimping defect probability; judging whether the wire harness crimping has defects according to the predicted crimping defect probability; S5. If a defect occurs in the crimping of the wire harness, the crimping defect data is automatically collected; the crimping defect data is input into the trained defect level prediction model to predict the defect level of the crimping of the wire harness; S6. Classify the alarm level according to the defect level of the wire harness crimping, and trigger the sound and light alarm to the defect detection terminal.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention dynamically adjusts the angle and intensity of the light source so that the surface materials of the wire harness with different reflectivity can obtain the best lighting conditions, ensuring that clear and accurate image data can be obtained for materials with high reflectivity, medium reflectivity or low reflectivity, avoiding image blur or loss of details caused by excessive or weak light. By adjusting the angle and intensity of the light source according to the extreme contrast of the wire harness image data, the dynamic range of the image can be optimized and the image details can be enhanced, especially in the transition area between high-reflectivity and low-reflectivity surfaces, the image contrast is improved, and the defects on the surface of the wire harness are more obvious, which is helpful for subsequent defect detection. By accurately adjusting the angle and intensity of the light source, the lighting effect in the image is adapted to the needs of different materials, and the presentation accuracy of the image details is improved. This is crucial for high-precision detection and analysis, especially when dealing with complex wire harness structures, it can improve the accuracy of the defect detection system. The system has an adaptive light source adjustment function, which can automatically adjust the light source parameters according to the different wire harness surface materials and the reflective characteristics of the current image, reducing manual intervention. The hybrid lighting technology of the annular LED light source and the coaxial light source is adopted, which can combine the advantages of the two light sources and adapt to wire harnesses with different reflectivity materials. By dynamically adjusting the smoothing factor in the similarity metric and combining it with the weight of the edge strength, the weight of the edge area is enhanced, so that the edge information can be better preserved during the denoising process. This solves the problem that traditional denoising methods often blur image details, especially edge details, thereby improving the quality of the denoised image and enhancing the edge clarity of the image; an adaptive similarity metric formula is used to control the weight of grayscale differences and gradient information, and adjust the similarity metric according to the grayscale values ​​and gradient information of different pixels, so as to more accurately identify the noise area and detail area in the image. The smoothing factor that controls the similarity metric range is dynamically adjusted through the smoothing factor restriction formula, so that the algorithm can flexibly adjust the denoising intensity according to the different characteristics of the image; based on the non-local mean denoising formula, the noise in the image is removed by weighted averaging the similarity weights of each pixel and its neighboring pixels, while maintaining the details of the image, especially the retention of high-frequency signals such as textures and edges. By introducing the multi-scale Retinex algorithm, the image blurring processing at each scale effectively enhances the details at different levels in the image. Combined with local contrast enhancement, the texture contrast of the wire harness image is further improved, making the tiny details in the image more prominent, which is helpful for subsequent defect detection and quality assessment. By processing the wire harness image at multiple scales, it can adapt to the details of different scales, thereby achieving more comprehensive and accurate enhancement. It can solve the prominent problems of illumination differences and local features at different scales, and improve the clarity and contrast of the image at all scales. By introducing the local contrast enhancement formula, the method can finely adjust the contrast of the local area of ​​the image, improve the detail performance, and avoid excessive adjustment of the overall brightness range. By weighted summation of image information at different scales, not only the detail information of each scale is integrated, but also the weight of each scale in the final image can be adjusted according to the contribution value of different scales, so that the enhancement effect is more balanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural schematic diagram of a wire harness crimping defect detection system based on machine vision of the present invention; Figure 2 A schematic flow chart of a wire harness crimping defect detection method based on machine vision according to the present invention; Figure 3 A flow chart of a method for constructing a multimodal feature map of a crimping part provided by the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1

[0019] See also Figure 1 As shown, this embodiment 1 further describes a wire harness crimping defect detection system based on machine vision proposed by the present invention, including: With the development of industrial automation and intelligence, wire harnesses are increasingly used in various electronic devices, especially in the fields of automobiles, aviation, and electronic products. The quality of wire harnesses directly affects the safety and stability of the equipment. The crimping quality of wire harnesses is crucial to the reliability of electrical connections. Therefore, it is particularly important to accurately and quickly detect wire harness crimping defects.

[0020] At present, traditional wire harness crimping defect detection methods mostly rely on manual visual inspection or automatic detection based on traditional image processing. These methods have some significant limitations, such as low efficiency of manual inspection, easy interference from human factors, and inability to meet the real-time requirements of large-scale production; and traditional automatic detection methods, especially those based on image processing, are often affected by factors such as lighting conditions, differences in wire harness surface materials, image noise, etc., resulting in inaccurate detection results, even missed detection or false detection, and cannot meet the requirements of high precision and high efficiency.

[0021] In the existing technology, although there are some detection systems based on machine vision, they still face the following problems: Poor light source adaptability: Different wire harness surface materials (such as copper core, coating, insulation layer, etc.) have different light reflection characteristics. Traditional light sources cannot be flexibly adapted, resulting in poor image quality and affecting the accuracy of subsequent detection.

[0022] Noise interference: Noise in the image caused by factors such as uneven lighting and environmental noise may have an adverse effect on image quality. Traditional image denoising methods have limited effects and it is difficult to effectively remove noise in complex backgrounds.

[0023] Low detection accuracy: Traditional defect detection algorithms are usually based on simple feature extraction and comparison, which makes it difficult to accurately identify subtle crimping defects, especially in complex crimping locations and variable wire harness surface materials, which are prone to missed detection or false detection.

[0024] Poor real-time performance: In the environment of an efficient production line, traditional wire harness defect detection methods often have slow processing speeds and cannot meet the needs of real-time production monitoring, increasing the risk in the production process.

[0025] In order to effectively solve the above problems, the present invention proposes a wire harness crimping defect detection system based on machine vision, comprising: Multi-spectral light source control module, which uses a ring-shaped LED light source and coaxial light mixed lighting, and dynamically adjusts the light source angle and intensity to adapt to different harness surface materials; Multi-dimensional sensing acquisition module, which collects wire harness image data and crimping geometry data in real time based on dynamically adjusted light source angle and intensity; The visual enhancement fusion module is used to denoise the wire harness image data by using the non-local mean denoising algorithm and dynamically adjusting the smoothing factor to control the similarity measurement range; superimpose the multi-scale Retinex algorithm to enhance the texture contrast of the wire harness image data through local contrast; fuse the wire harness image data with the crimping geometry data to construct a multimodal feature map of the crimping part; The crimping defect detection module is used to use the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the probability of crimping defects; based on the predicted crimping defect probability, it is determined whether the wire harness crimping has defects; Defect grade classification module: if there is a defect in the crimping of the wire harness, the crimping defect data will be automatically collected; the crimping defect data will be input into the trained defect grade prediction model to predict the defect grade of the crimping of the wire harness; The real-time alarm module is used to classify the alarm level according to the defect level of the wire harness crimping and trigger the sound and light alarm to the defect detection terminal; each module is connected by wire and / or wireless means.

[0026] Methods for using mixed lighting of annular LED light source and coaxial light, dynamically adjusting the light source, and adapting to different harness surface materials include: The wire harness image data is obtained through the camera, and the wire harness surface material is classified. The preset wire harness surface material categories are ,in, Indicates copper core; Indicates coating; Indicates the insulation layer; considering the reflectivity of the harness surface material, the harness surface material is divided into high reflectivity material, medium reflectivity material and low reflectivity material; the reflectivity of the harness surface material is defined as: ;in, is the reflectivity of the harness surface material; is the reflected light intensity; is the incident light intensity; Preset reflectivity first threshold and reflectivity second threshold ,like , then the surface material of the harness is determined to be a high reflectivity material; if , then the surface material of the harness is determined to be a medium reflectivity material; if , then the surface material of the wire harness is determined to be a low reflectivity material; according to the different imaging effects of the surface material of the wire harness under different light sources, different light source combinations are selected. Different light source combinations include ;in, It is a ring-shaped LED light source; Coaxial light source is a kind of light source illumination method, in which the light source is collinear with the optical axis of the camera lens. In other words, the light and the camera's viewing angle are propagated along the same axis. The advantage of this illumination method is that it can reduce the interference of reflected light on the surface of the object, making the surface texture and details clearer, especially suitable for imaging tasks that need to highlight the surface details of the object; After identifying the surface material of the wire harness and selecting different light source combinations, the light source angle and light source intensity are dynamically adjusted according to the current extreme contrast of the wire harness image data through the light source angle adjustment formula and the light source intensity adjustment formula; the extreme contrast is: ;in, is the maximum brightness value in the line beam image data; is the minimum brightness value in the line beam image data; The light source angle adjustment formula is: ;in, is the angle of the light source after adjustment; is the initial light source angle (based on the harness surface material type Preset); The light source angle adjustment factor mainly adjusts the angle of the light source, that is, how to adjust the angle of the light source relative to the beam to achieve the best imaging effect; is the target extreme contrast; is the current extreme contrast; The light source intensity adjustment formula is: ;in, is the intensity of the light source after adjustment; is the initial light intensity (based on the harness surface material type Preset); is the reflectivity factor, which is used to control the effect of reflectivity on the intensity of the light source; is the reflectivity of the current harness surface material; It is the light source intensity adjustment factor, which mainly adjusts the intensity of the light source, that is, it improves the extreme contrast of the image by adjusting the brightness of the light source; The reflectivity factor is restricted by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is: ;in, is the number of material categories on the harness surface; is the weight factor of the current extreme contrast; is the weight factor for the number of material categories on the harness surface; When the reflectivity is high, the intensity of light reflected from the surface is strong, and you may need to reduce the intensity of the light source to avoid overexposure; when the reflectivity is low, you may need to increase the intensity of the light source to make up for the insufficient reflection; The number of types of materials on the surface of the harness (such as copper, plating, insulation, etc.) affects the reflectivity and quality of the final image. Under the influence of different materials, the intensity of the light source needs to be adjusted differently. By dynamically adjusting the weight factors of the number of material types and the extreme contrast, the challenge of adjusting the intensity of the light source for multiple materials can be solved to ensure the adaptation of each material; the extreme contrast of the image (i.e. the brightness difference of the image) also affects the adjustment of the intensity of the light source; a higher contrast may mean that no strong light source adjustment is required, while a lower contrast may require an increase in the intensity of the light source.

[0027] Under different reflectivity and image contrast conditions, the adjustment range of the light source intensity should be appropriately limited. If the reflectivity factor is too large or too small, it may cause the image to be overexposed or too dark, affecting the quality. By setting the constraints of the reflectivity factor, excessive or insufficient adjustments can be avoided to ensure that the light source intensity is always within a reasonable range; For example, the current extreme contrast 10, the number of harness surface material categories 3, the weight factor of the current extreme contrast 0.6; weight factor for the number of material categories on the harness surface is 0.4; then the reflectivity factor after limitation is .

[0028] The wire harness image data includes the wire harness surface image, the crimping terminal image and the crimping part image; the crimping geometry data includes the height, volume, crimping angle, thickness and diameter of the crimping part.

[0029] Using the non-local mean denoising algorithm, the similarity measurement range is controlled by dynamically adjusting the smoothing factor, and the method for denoising the wire harness image data includes: For each pixel in the line beam image data , use edge detection algorithm to calculate its edge strength ;in, is a measure of edge strength; For each pixel The pixel value at , that is, the brightness or color intensity of that point in the image; is the horizontal coordinate of each pixel; is the ordinate of each pixel; For any pair of pixels in the image and ,The adaptive similarity metric is calculated by the similarity metric formula, which considers the pixel value difference and gradient information; The similarity measurement formula is: ;in, Represents pixel and The similarity measure between the two pixels indicates the similarity between the two pixels. The larger the value, the more similar the two pixels are, and the smaller the value, the greater the difference between them. In the denoising process, the similarity between the pixels determines their weight in weighted averaging; Pixel The pixel value of Pixel The pixel value of Pixel Gradient information at ; Pixel Gradient information at ; It is the adjustment factor to control the grayscale difference and gradient information weight; is a smoothing factor to control the range of similarity measurement; The smoothing factor that controls the range of similarity measurement through the smoothing factor restriction formula Dynamic limitation is performed, and the smoothing factor limitation formula is: ;in, is the total number of pixels of the line beam image data; is the sum of edge strengths; is a regulating factor for controlling the influence of the total number of pixels of the line beam image data on the smoothing factor; It is a regulating factor to control the effect of edge strength on the smoothing factor; For example, the total number of pixels of wire harness image data is 100000, the sum of the edge strengths The adjustment factor is 20000, which controls the influence of the total number of pixels of the wire harness image data on the smoothing factor. 0.0001, which is the adjustment factor that controls the effect of edge strength on the smoothing factor is 1; then the smoothing factor after restriction is .

[0030] Based on edge strength ,The similarity measure is modified by modifying the similarity measure formula to enhance the weight of the edge area, ensuring that the noise in the edge area is reduced while the details are preserved; Modify the similarity measurement formula to: ;in, Pixel and The Euclidean distance between Pixel The edge strength at is a regulatory factor to control the weighted influence of the edge; The non-local mean denoising formula is used to remove noise by performing weighted averaging based on the similarity weights of each pixel and its neighboring pixels. The non-local mean denoising formula is: ;in, Represents the pixels in the denoised line image data The value of The weighted average of the surrounding neighborhood pixels is used to eliminate noise; Indicates the pixel position currently being processed, which refers to a pixel point in the image. It indicates the position in the image and can be regarded as a point in the image coordinate system (but in the formula, the specific coordinate value is not written, and it is directly used Indicates location); Represents pixels Every pixel in the neighborhood of ; Represents the neighborhood pixels The pixel value of .

[0031] Methods for enhancing the texture contrast of line beam image data by local contrast by superimposing a multi-scale Retinex algorithm include: The line beam image data is blurred by Gaussian filters of different scales to obtain multi-scale images. Generate a blurred image; use a convolution operation to blur the line image data: in, Indicates the scale The blurred image after Gaussian filter processing is at pixel point The pixel value on ; represents the Gaussian kernel function, which is a two-dimensional Gaussian function used to blur the image; Represents the convolution operation; For each scale , calculate each pixel The pixel value at and The ratio between: ;in, For scale The ratio image below shows the enhancement effect of the image at this scale; Based on the ratio calculation, the contrast image is adjusted by the local contrast enhancement formula to enhance the high-frequency details in the image; the local contrast enhancement formula is: ;in, is the image after local contrast enhancement; is a parameter factor for adjusting the enhancement strength; The ratio images from different scales are weighted and summed, and the enhancement information of each scale is integrated to obtain a multi-scale enhanced image; the multi-scale enhanced image is: ;in, It is a multi-scale enhanced image, that is, by weighted summation of ratio images of different scales The obtained enhanced image; is the number of scales; For scale The ratio image below: For scale The weight coefficient of For the scale; is the index of the scale, ranging from 1 to ; The dynamic range of the multi-scale enhanced image is restored by using inverse logarithmic mapping, and the multi-scale enhanced image is restored to the brightness range of the line beam image data by performing exponential mapping on the multi-scale enhanced image.

[0032] Fusion of wire harness image data with crimp geometry data, such as Figure 3 As shown, the method for constructing a multi-modal feature map of a crimping part includes: Apply the ICP algorithm to iteratively align the point clouds of the wire harness image data and the crimping geometry data, continuously adjust the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometry data, and find the optimal transformation that minimizes the distance between the two sets of point clouds; The error threshold is preset. During each iteration, the ICP algorithm calculates the distance error between the two groups of point clouds under the current transformation and determines whether the preset error threshold or the maximum number of iterations is reached. If reached, the iteration is terminated. After the alignment is completed, the point cloud of the wire harness image data and the crimping geometry data will be fused into a whole, thereby generating a multimodal feature map of the crimping part.

[0033] The construction method of the crimping defect detection model includes: The data set is divided into a training set, a validation set, and a test set; a crimping defect detection model is constructed, where the sample set is a subset of the data set, and each sample set includes a multimodal feature map of the historical crimping parts and the corresponding crimping defect probability; The crimping defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer; the input layer is a multimodal feature map of historical crimping locations, and the output layer is the probability of crimping defects; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model; The mean square error is used as the loss function to measure the error between the model's predicted value and the actual value. The hazard probability prediction model is trained using the training set, and the model parameters are updated through the back propagation algorithm and the gradient descent method to minimize the loss function. The performance of the crimping defect detection model is evaluated by calculating the coefficient of determination using the validation set, and the hyperparameters of the model are tuned. The Adam optimization algorithm is selected as the optimizer, and the hyperparameters of the model are adjusted until the performance no longer improves or the preset number of iterations is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained crimping defect detection model is used to predict the multimodal feature map of the current crimping part to obtain the crimping defect probability.

[0034] Based on the predicted probability of crimping defects, the method for determining whether a wire harness crimping defect occurs includes: A crimping defect probability threshold is preset, and the predicted crimping defect probability is compared with the preset crimping defect probability threshold; If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is determined that no defect occurs in the crimping of the wire harness; If the predicted crimping defect probability is greater than or equal to a preset crimping defect probability threshold, it is determined that the wire harness crimping has a defect.

[0035] The training method of the defect grade prediction model includes: The data set is divided into training set, test set and validation set, and a defect level prediction model is constructed. The defect level prediction model includes an input layer, a hidden layer and an output layer. The input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the wire harness crimping. The output layer is set with neurons equal to the number of defect levels of the wire harness crimping, and each neuron corresponds to the prediction probability of a defect level type of the wire harness crimping, and the softmax function is used as the activation function. The defect level prediction model is a multi-layer perceptron MLP model. Use multi-classification cross entropy as the model's loss function to measure the difference between the model's predicted value and the actual value; use the training set to train the error type recognition model, and update the model parameters through the back propagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy index; The SGD optimization algorithm is selected as the optimizer, and the model is tuned according to the performance feedback of the validation set. The model parameters are adjusted until the performance no longer improves or the preset stopping condition is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained defect level prediction model is used to predict the current crimping defect data to obtain the defect level of the wire harness crimping.

[0036] The method of dividing the alarm level according to the defect level of the wire harness crimping includes: setting the corresponding alarm level according to the defect level, each defect level corresponds to an alarm level, for example, a minor defect corresponds to a low-level alarm, a medium defect corresponds to a medium-level alarm, and a serious defect corresponds to a high-level alarm; once a defect is detected, the system will compare the defect level with the preset alarm level.

[0037] If the defect level exceeds or equals a certain alarm level, an alarm is triggered; when the alarm condition is met, the system transmits the alarm information to the defect detection terminal through the sound and light alarm device. The sound and light alarm can be carried out by sound, flashing light or screen prompt.

[0038] The preset crimping defect probability threshold is set by the staff. Different crimping defect probabilities are collected through the defect detection terminal, and the average value of multiple crimping defect probabilities is taken as the preset crimping defect probability threshold; the preset error threshold is set similarly.

[0039] In this embodiment, by dynamically adjusting the angle and intensity of the light source, the surface materials of the wire harness with different reflectivity can obtain the best lighting conditions, ensuring that clear and accurate image data can be obtained for materials with high reflectivity, medium reflectivity or low reflectivity, avoiding image blur or loss of details caused by too strong or too weak light; by adjusting the angle and intensity of the light source according to the extreme contrast of the wire harness image data, the dynamic range of the image can be optimized and the image details can be enhanced, especially in the transition area between high-reflectivity and low-reflectivity surfaces, the image contrast is improved, and the defects on the surface of the wire harness are more obvious, which is helpful for subsequent defect detection; by accurately adjusting the angle and intensity of the light source, the lighting effect in the image can adapt to the needs of different materials, and the presentation accuracy of image details is improved. This is crucial for high-precision detection and analysis, especially when dealing with complex wire harness structures, it can improve the accuracy of the defect detection system; the system has an adaptive light source adjustment function, which can automatically adjust the light source parameters according to the different wire harness surface materials and the reflective characteristics of the current image, reducing manual intervention; the hybrid lighting technology of the ring LED light source and the coaxial light source can combine the advantages of the two light sources and adapt to wire harnesses with different reflectivity materials; By dynamically adjusting the smoothing factor in the similarity metric and combining it with the weight of the edge strength, the weight of the edge area is enhanced, so that the edge information can be better preserved during the denoising process. This solves the problem that traditional denoising methods often blur image details, especially edge details, thereby improving the quality of the denoised image and enhancing the edge clarity of the image; an adaptive similarity metric formula is used to control the weight of grayscale differences and gradient information, and adjust the similarity metric according to the grayscale values ​​and gradient information of different pixels, so as to more accurately identify the noise area and detail area in the image. The smoothing factor that controls the similarity metric range is dynamically adjusted through the smoothing factor restriction formula, so that the algorithm can flexibly adjust the denoising intensity according to the different characteristics of the image; based on the non-local mean denoising formula, the noise in the image is removed by weighted averaging the similarity weights of each pixel and its neighboring pixels, while maintaining the details of the image, especially the retention of high-frequency signals such as textures and edges. By introducing the multi-scale Retinex algorithm, the image blurring processing at each scale effectively enhances the details at different levels in the image. Combined with local contrast enhancement, the texture contrast of the wire harness image is further improved, making the tiny details in the image more prominent, which is helpful for subsequent defect detection and quality assessment. By processing the wire harness image at multiple scales, it can adapt to the details of different scales, thereby achieving more comprehensive and accurate enhancement. It can solve the prominent problems of illumination differences and local features at different scales, and improve the clarity and contrast of the image at all scales. By introducing the local contrast enhancement formula, the method can finely adjust the contrast of the local area of ​​the image, improve the detail performance, and avoid excessive adjustment of the overall brightness range. By weighted summation of image information at different scales, not only the detail information of each scale is integrated, but also the weight of each scale in the final image can be adjusted according to the contribution value of different scales, so that the enhancement effect is more balanced.

[0040] Example 2

[0041] See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, and a method for detecting wire harness crimping defects based on machine vision is provided, comprising: S1. Adopting mixed lighting of annular LED light source and coaxial light, dynamically adjusting the angle and intensity of the light source to adapt to different harness surface materials; S2, based on the dynamically adjusted light source angle and intensity, real-time collection of wire harness image data and crimping geometry data; S3. Using the non-local mean denoising algorithm, the similarity measurement range is controlled by dynamically adjusting the smoothing factor to denoise the wire harness image data; the multi-scale Retinex algorithm is superimposed to enhance the texture contrast of the wire harness image data by local contrast; the wire harness image data is fused with the crimping geometry data to construct a multimodal feature map of the crimping part; S4, using the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the crimping defect probability; judging whether the wire harness crimping has defects according to the predicted crimping defect probability; S5. If a defect occurs in the crimping of the wire harness, the crimping defect data is automatically collected; the crimping defect data is input into the trained defect level prediction model to predict the defect level of the crimping of the wire harness; S6. Classify the alarm level according to the defect level of the wire harness crimping, and trigger the sound and light alarm to the defect detection terminal.

[0042] Since the electronic device introduced in this embodiment is an electronic device used to implement a wire harness crimping defect detection system and method based on machine vision in the embodiment of this application, based on the wire harness crimping defect detection system and method based on machine vision introduced in the embodiment of this application, the technical personnel in this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be introduced in detail here. As long as the technical personnel in this field implement the electronic device used by a wire harness crimping defect detection system and method based on machine vision in the embodiment of this application, it belongs to the scope of protection of this application.

[0043] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0044] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A wire harness crimping defect detection system based on machine vision, characterized in that: include: Multi-spectral light source control module, which uses a ring-shaped LED light source and coaxial light mixed lighting, and dynamically adjusts the light source angle and intensity to adapt to different harness surface materials; Multi-dimensional sensing acquisition module, which collects wire harness image data and crimping geometry data in real time based on dynamically adjusted light source angle and intensity; A visual enhancement fusion module is used to denoise the wire harness image data using a non-local mean denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor; The multi-scale Retinex algorithm is superimposed to enhance the texture contrast of the wire harness image data through local contrast; the wire harness image data is fused with the crimping geometry data to construct a multi-modal feature map of the crimping part; The crimping defect detection module is used to use the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the probability of crimping defects; based on the predicted crimping defect probability, it is determined whether the wire harness crimping has defects; Defect grade classification module: if there is a defect in the crimping of the wire harness, the crimping defect data will be automatically collected; the crimping defect data will be input into the trained defect grade prediction model to predict the defect grade of the crimping of the wire harness; The real-time alarm module is used to classify the alarm level according to the defect level of the wire harness crimping and trigger the sound and light alarm to the defect detection terminal.

2. The wire harness crimping defect detection system based on machine vision according to claim 1 is characterized in that: The method of using a ring-shaped LED light source and coaxial light mixed lighting, dynamically adjusting the light source, and adapting to different harness surface materials includes: The wire harness image data is obtained through the camera, and the wire harness surface material is classified. The preset wire harness surface material categories are ,in, Indicates copper core; Indicates coating; Indicates the insulation layer; considering the reflectivity of the harness surface material, the harness surface material is divided into high reflectivity material, medium reflectivity material and low reflectivity material. Material; The reflectivity of the harness surface material is defined as: ;in, is the reflectivity of the harness surface material; is the reflected light intensity; is the incident light intensity; Preset reflectivity first threshold and reflectivity second threshold ,like , then the surface material of the harness is determined to be a high reflectivity material; if , then the surface material of the harness is determined to be a medium reflectivity material; if , then the surface material of the wire harness is determined to be a low reflectivity material; according to the different imaging effects of the surface material of the wire harness under different light sources, different light source combinations are selected. Different light source combinations include ;in, It is a ring-shaped LED light source; It is a coaxial light source; After identifying the surface material of the wire harness and selecting different light source combinations, the light source angle and light source intensity are dynamically adjusted according to the current extreme contrast of the wire harness image data through the light source angle adjustment formula and the light source intensity adjustment formula; the extreme contrast is: ;in, is the maximum brightness value in the line beam image data; is the minimum brightness value in the line beam image data; The light source angle adjustment formula is: ;in, is the angle of the light source after adjustment; is the initial light source angle; is the light source angle adjustment factor; is the target extreme contrast; is the current extreme contrast; The light source intensity adjustment formula is: ;in, is the intensity of the light source after adjustment; is the initial light source intensity; is the reflectivity factor; is the reflectivity of the current harness surface material; is the light source intensity adjustment factor; The reflectivity factor is restricted by the reflectivity factor restriction formula, and the reflectivity factor restriction formula is: ;in, is the number of material categories on the harness surface; is the weight factor of the current extreme contrast; A weighting factor for the number of material categories on the harness surface.

3. The wire harness crimping defect detection system based on machine vision according to claim 2 is characterized in that: The wire harness image data includes a wire harness surface image, a crimping terminal image and a crimping location image; the crimping geometry data includes the height, volume, crimping angle, thickness and diameter of the crimping location.

4. The wire harness crimping defect detection system based on machine vision according to claim 3 is characterized in that: The method for denoising the wire harness image data by using the non-local mean denoising algorithm and controlling the similarity measurement range by dynamically adjusting the smoothing factor includes: For each pixel in the line beam image data , use edge detection algorithm to calculate its edge strength ;in, is a measure of edge strength; For each pixel The pixel value at ; is the horizontal coordinate of each pixel; is the ordinate of each pixel; For any pair of pixels in the image and , the adaptive similarity measure is calculated by the similarity measure formula; The similarity measurement formula is: ;in, Represents pixel and Similarity measure between ; Pixel The pixel value of Pixel The pixel value of Pixel Gradient information at ; Pixel Gradient information at ; It is the adjustment factor to control the grayscale difference and gradient information weight; is a smoothing factor to control the range of similarity measurement; The smoothing factor that controls the range of similarity measurement through the smoothing factor restriction formula Dynamic limitation is performed, and the smoothing factor limitation formula is: ;in, is the total number of pixels of the line beam image data; is the sum of edge strengths; is a regulating factor for controlling the influence of the total number of pixels of the line beam image data on the smoothing factor; It is a regulating factor to control the effect of edge strength on the smoothing factor; Based on edge strength ,The similarity measure is modified to enhance the weight of the edge area by modifying the similarity measure formula; Modify the similarity measurement formula to: ;in, Pixel and The Euclidean distance between Pixel The edge strength at is a regulatory factor to control the weighted influence of the edge; The non-local mean denoising formula is used to remove noise by performing weighted averaging based on the similarity weights of each pixel and its neighboring pixels. The non-local mean denoising formula is: ;in, Represents the pixels in the denoised line image data The value of Indicates the pixel position currently being processed; Represents pixels Every pixel in the neighborhood of ; Represents the neighborhood pixels The pixel value of .

5. The machine vision-based wire harness crimping defect detection system according to claim 4, characterized in that: The method of superimposing a multi-scale Retinex algorithm to enhance the texture contrast of line beam image data by local contrast includes: The line beam image data is blurred by Gaussian filters of different scales to obtain multi-scale images. Generate a blurred image; use a convolution operation to blur the line image data: in, Indicates the scale The blurred image after Gaussian filter processing is at pixel point The pixel value on ; represents the Gaussian kernel function; Represents the convolution operation; For each scale , calculate each pixel The pixel value at and The ratio between: ;in, For scale The ratio image below: Based on the ratio calculation, the contrast image is adjusted by the local contrast enhancement formula; the local contrast enhancement formula is: ;in, is the image after local contrast enhancement; Parameter factor for adjusting the enhancement strength; The ratio images from different scales are weighted and summed, and the enhancement information of each scale is integrated to obtain a multi-scale enhanced image; the multi-scale enhanced image is: ;in, Enhance images at multiple scales; is the number of scales; For scale The ratio image below: For scale The weight coefficient of For the scale; is the index of the scale; The dynamic range of the multi-scale enhanced image is restored by using inverse logarithmic mapping, and the multi-scale enhanced image is restored to the brightness range of the line beam image data by performing exponential mapping on the multi-scale enhanced image.

6. The machine vision-based wire harness crimping defect detection system according to claim 5, characterized in that: The method of fusing the wire harness image data with the crimping geometry data to construct a multimodal feature map of the crimping part includes: Apply the ICP algorithm to iteratively align the point clouds of the wire harness image data and the crimping geometry data, continuously adjust the relative position and rotation angle between the two point clouds of the wire harness image data and the crimping geometry data, and find the optimal transformation that minimizes the distance between the two sets of point clouds; The error threshold is preset. During each iteration, the ICP algorithm calculates the distance error between the two groups of point clouds under the current transformation and determines whether the preset error threshold or the maximum number of iterations is reached. If reached, the iteration is terminated. After the alignment is completed, the point cloud of the wire harness image data and the crimping geometry data will be fused into a whole, thereby generating a multimodal feature map of the crimping part.

7. The machine vision-based wire harness crimping defect detection system according to claim 6, characterized in that: The method for constructing the crimping defect detection model includes: The data set is divided into a training set, a validation set, and a test set; a crimping defect detection model is constructed, where the sample set is a subset of the data set, and each sample set includes a multimodal feature map of the historical crimping parts and the corresponding crimping defect probability; The crimping defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is a multimodal feature map of historical crimping parts, and the output layer is the crimping defect probability; the output layer uses a linear activation function, and the crimping defect detection model is a convolutional neural network model; The mean square error is used as the loss function to measure the error between the model's predicted value and the actual value. The hazard probability prediction model is trained using the training set, and the model parameters are updated through the back propagation algorithm and the gradient descent method to minimize the loss function. The performance of the crimping defect detection model is evaluated by calculating the coefficient of determination using the validation set, and the hyperparameters of the model are tuned. The Adam optimization algorithm is selected as the optimizer, and the hyperparameters of the model are adjusted until the performance no longer improves or the preset number of iterations is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained crimping defect detection model is used to predict the multimodal feature map of the current crimping part to obtain the crimping defect probability.

8. The machine vision-based wire harness crimping defect detection system according to claim 7, characterized in that: The method for judging whether a wire harness crimping defect occurs according to the predicted crimping defect probability includes: A crimping defect probability threshold is preset, and the predicted crimping defect probability is compared with the preset crimping defect probability threshold; If the predicted crimping defect probability is less than the preset crimping defect probability threshold, it is determined that no defect occurs in the crimping of the wire harness; If the predicted crimping defect probability is greater than or equal to a preset crimping defect probability threshold, it is determined that the wire harness crimping has a defect.

9. The machine vision-based wire harness crimping defect detection system according to claim 8, characterized in that: The training method of the defect level prediction model includes: The data set is divided into a training set, a test set and a validation set, and a defect level prediction model is constructed. The defect level prediction model includes an input layer, a hidden layer and an output layer. The input layer of the model is used to input historical crimping defect data, and the output layer is used to output the defect level of the wire harness crimping. The output layer is set with neurons equal to the number of defect levels of the wire harness crimping, each neuron corresponds to the prediction probability of a defect level type of the wire harness crimping, and a softmax function is used as an activation function. The defect level prediction model is a multi-layer perceptron MLP model. Use multi-classification cross entropy as the model's loss function to measure the difference between the model's predicted value and the actual value; use the training set to train the error type recognition model, and update the model parameters through the back propagation algorithm to minimize the loss function; use the validation set to evaluate the performance of the defect level prediction model by calculating the accuracy index; The SGD optimization algorithm is selected as the optimizer, and the model is tuned according to the performance feedback of the validation set. The model parameters are adjusted until the performance no longer improves or the preset stopping condition is reached. The test set is used to evaluate the performance of the model in the prediction task, and the trained defect level prediction model is used to predict the current crimping defect data to obtain the defect level of the wire harness crimping.

10. A wire harness crimping defect detection method based on machine vision, used to implement the wire harness crimping defect detection system based on machine vision as claimed in any one of claims 1 to 9, characterized in that: include: S1. Adopting mixed lighting of annular LED light source and coaxial light, dynamically adjusting the angle and intensity of the light source to adapt to different harness surface materials; S2, based on the dynamically adjusted light source angle and intensity, real-time collection of wire harness image data and crimping geometry data; S3, using a non-local mean denoising algorithm to control the similarity measurement range by dynamically adjusting the smoothing factor to denoise the wire harness image data; The multi-scale Retinex algorithm is superimposed to enhance the texture contrast of the wire harness image data through local contrast; the wire harness image data is fused with the crimping geometry data to construct a multi-modal feature map of the crimping part; S4, using the multimodal feature map of the crimping part as the input of the crimping defect detection model to predict the crimping defect probability; judging whether the wire harness crimping has defects according to the predicted crimping defect probability; S5. If a defect occurs in the crimping of the wire harness, the crimping defect data is automatically collected; the crimping defect data is input into the trained defect level prediction model to predict the defect level of the crimping of the wire harness; S6. Classify the alarm level according to the defect level of the wire harness crimping, and trigger the sound and light alarm to the defect detection terminal.

Images