Intelligent visual assembly guidance system and method for wire harness

Abstract

The invention belongs to the technical field of control, and discloses an intelligent visual assembly guidance system and method for a wire harness, and the method comprises the steps: collecting geometric feature data and environment data; based on the geometric feature data and the environmental data, using an SVM model to detect the qualification of the assembly process; when detecting that the assembly process is unqualified, judging the defect type by using a GS-DNN model; wherein the probability of each defect type is calculated by using a regulatory-Gumbel-Softmax activation function, and the probability of each defect type is calculated by using the regulatory-Gumbel-Softmax activation function; when the defect type is wrong wiring, a motion control system is adjusted by using a chaotic mapping adaptive PID control model, and the movement amount of the wiring harness is adjusted; when the defect type is a reverse line, a motion control system is adjusted by using a chaotic mapping adaptive PID control model, and the direction angle of the wire harness is adjusted; automatic correction in the wire harness assembling process is achieved, the assembling precision is improved, and the production efficiency and the product quality can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a wire harness intelligent visual assembly guidance system and method. Background Art

[0002] The existing intelligent visual assembly system for wire harnesses has many problems. First, it relies on ordinary cameras and simple image processing algorithms, such as Canny edge detection, and cannot extract the wire harness contour from complex wire harness images. Secondly, existing intelligent assembly guidance systems for wire harnesses usually only focus on visual information during the assembly process, ignoring the influence of environmental factors, which can affect the accuracy of the wire harnesses during the actual assembly process; In addition, the existing intelligent assembly guidance system for wire harnesses relies on simple judgment criteria to determine the defect type and cannot determine the specific defect type; Finally, for defects that occur during the assembly process, the existing wiring harness intelligent assembly guidance system can only perform post-detection and cannot control other systems in time to make adjustments.

[0003] In view of this, the present invention proposes a wire harness intelligent visual assembly guidance system and method to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution, a wiring harness intelligent visual assembly guidance system, comprising: Information capture unit: Use a camera to capture the wire harness image and perform preliminary processing to obtain a preliminary wire harness image; use ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; based on the wire harness contour, use geometric extraction technology to extract geometric feature data; among them, environmental data is collected in real time using sensors; Qualification judgment unit: Based on geometric feature data and environmental data, the SVM model is used to detect the qualification of the assembly process; Intelligent control unit: When an unqualified assembly process is detected, the GS-DNN model is used to determine the defect type; the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the exponential decay-adaptive method is used to set the temperature coefficient in the adjustment-Gumbel-Softmax activation function; when the defect type is a wrong line, the chaotic mapping adaptive PID control model is used to adjust the motion control system and adjust the movement of the wiring harness; when the defect type is a reverse line, the chaotic mapping adaptive PID control model is used to adjust the motion control system and adjust the direction angle of the wiring harness.

[0005] Furthermore, the specific method of using a camera to capture the wire harness image and performing preliminary processing to obtain the preliminary wire harness image includes: The collected wire beam image is denoised using a Gaussian blur model to obtain a denoised wire beam image; the denoised wire beam image is contrast enhanced using a histogram equalization technique to obtain a preliminary wire beam image.

[0006] Furthermore, the specific method of using the ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image includes: Step aa1, using the Sobel operator to calculate the gradient of each pixel in the preliminary line beam image in the horizontal direction and the vertical direction; introducing the Laplacian operator based on the gradient in the horizontal direction and the vertical direction to calculate the gradient amplitude; Step aa2: Dynamically adjust the high and low thresholds by calculating the mean and standard deviation of the gradient amplitude. The formula is: , ,in, is the mean value of the gradient amplitude, is the standard deviation of the gradient amplitude, is the regulating factor, is the high threshold, is the low threshold; Step aa3: When the gradient amplitude of a pixel point is greater than the high threshold, it is a strong edge point and is directly retained; when the gradient amplitude of a pixel point is between the low threshold and the high threshold, step aa4 is performed to determine whether to retain or suppress; when the pixel point is less than the low threshold, it is a weak edge point and is suppressed; Step aa4, calculate the gradient direction based on the gradients in the horizontal and vertical directions; for each pixel point, use bilinear interpolation to calculate the interpolation gradient amplitude of two adjacent pixel points in the gradient direction, and compare the interpolation gradient amplitude with the gradient amplitude of the pixel point; when the gradient amplitude of the pixel point is greater than or equal to the interpolation gradient amplitude, retain the pixel point; when the gradient amplitude of the pixel point is less than the interpolation gradient amplitude, suppress the pixel point.

[0007] Furthermore, the specific method of extracting geometric feature data by using geometric extraction technology based on the wire harness profile includes: The geometric feature data includes wire harness length data, wire harness curvature data, and wire harness angle data; Based on the extracted wire harness contour, a contour tracking algorithm is used to obtain the wire harness contour point set: ,in, represents the ath harness contour point, represents the nth line bundle contour point, n represents the total number of line bundle contour point sets, and a represents the index of the line bundle contour point set; Through the wire harness contour point set, the distance between adjacent points is calculated using the Euclidean technique to obtain the wire harness length data. The formula is: ,in, and Indicates The horizontal and vertical coordinates of the line contour points, Indicates harness length data; The discrete second-order derivative method is used to calculate the bundle curvature data, and the formula is: ,in, represents the curvature of the a-th contour point; and Indicates the contour point from the ath contour point to the right The displacement of and Indicates the contour point a to the left of the contour point The displacement of Represents the square of the Euclidean distance from the ath contour point to the right contour point; By using the vector geometry method, the angle data of the line bundle between the ath contour point and the left contour point and the right contour point is calculated; first, the contour point is calculated To contour point The vector of is: , calculate the contour points To contour point The vector of is: ; Calculate the contour points again and contour points Harness angle data between: .

[0008] Furthermore, the specific method of using the SVM model to detect the eligibility of the assembly process based on the geometric feature data and the environmental data includes: Based on the geometric feature data and environmental data, the interpolation method is used to process missing values ​​and outliers, and the Min-Max method is used for normalization to obtain the geometric environment feature data set; the geometric environment feature data set is used as the input of the SVM model; Use grid search to initialize hyperparameters, including penalty coefficient C, kernel function, gamma parameter, and maximum number of iterations; use random method to initialize weight vector and the bias term ; Calculate the initial loss function value of the SVM model ,in, represents regularization, represents the classification error term, is the penalty coefficient, is the total number of samples in the input data set, is the sample index, is the label of the fth sample, is the input data of the fth sample; In each iteration, use The optimizer updates the weight vector and bias term, and calculates the new loss function during the update process; When the maximum number of iterations is reached, the iteration is stopped and the final weight vector is output. and the bias term , using the final weight vector and the bias term Calculate the decision function value ,in, For input data, For input data The decision function value of , indicating that the assembly process is qualified. , indicating that the assembly process is unqualified.

[0009] Furthermore, when the assembly process is detected to be unqualified, the specific method of using the GS-DNN model to determine the defect type includes: The input layer takes the geometric feature data and environmental data of the unqualified assembly process as input; the number of neurons is set to be equal to the number of input data; Hidden layer, set L_l hidden layers; pass the input data from the input layer to the first hidden layer, obtain the output of the first hidden layer through linear transformation and SinReLu activation function, pass the output of the first hidden layer to the next hidden layer, repeatedly use SinReLu activation function, calculate the output of the next hidden layer, until the end of the last hidden layer, obtain the output of the last hidden layer; The output layer passes the output of the last hidden layer to the output layer, uses linear transformation to project the hidden layer output to the 2D defect type space, and outputs the output value of the 2D defect type; the first-dimensional defect type is a broken line, and the second-dimensional defect type is a reverse line; Calculate the probability of defect type. Based on the output value of the 2D defect type output by the output layer, use the Adjust-Gumbel-Softmax activation function to calculate the probability of the 2D defect type. Determine the final defect type. When the set number of iterations is reached, stop the iteration and output the final defect type probability. Compare the final probabilities of the two-dimensional defect types and take the one with the larger probability as the main defect of the assembly process. When the final probability of the first-dimensional defect type is greater than the second-dimensional defect type, the defect type is a misalignment, and the one-hot encoding is expressed as ; When the final probability of the defect type in the first dimension is less than that of the defect type in the second dimension, the defect type is reversed, and the one-hot encoding is expressed as .

[0010] Furthermore, the specific method of using the adjustment-Gumbel-Softmax activation function to calculate the probability of each defect type includes: Using the Adjustment-Gumbel-Softmax activation function, the probability of each defect type is calculated as follows: ,in, Indicates the defect type index, Indicated in The output layer outputs the Output value of defect type, Indicates In the iteration The probability of a defect type, Indicates In the iteration Gubel noise for each defect type, Indicates In the iteration The adjustment temperature coefficient of each defect type is set using the exponential decay-adaptive method. is the normalization factor, Indicates the defect type index, Indicates The output layer outputs the Output value of each defect type, No. In the iteration Gubel noise for each defect type, Indicates In the iteration The adjustment temperature coefficient for each defect type is The index of the iteration number.

[0011] Furthermore, the specific method of using the exponential decay-adaptive method to set the adjustment temperature coefficient includes: The temperature parameter is calculated using the exponential decay method, the formula is: ,in, For the In the iteration Exponential temperature coefficients for each defect type, is the initial temperature coefficient, is the attenuation coefficient, which controls the change rate of the temperature coefficient; The temperature parameters are calculated using the adaptive method, and the formula is: ,in, Indicates In the iteration Adaptive temperature coefficients for each defect type, is the adjustment coefficient, Indicates Iteration No. The entropy of defect types is: , Indicates In the iteration The probability of each defect type is calculated by the Softmax function; The adjustment temperature coefficient is calculated by combining the exponential temperature coefficient and the adaptive temperature coefficient. The formula is: ,in, Indicates In the iteration Adjustment temperature coefficient for each defect type.

[0012] Furthermore, when the defect type is a misaligned line, the motion control system is adjusted using a chaotic mapping adaptive PID control model to adjust the movement amount of the wire harness in a specific manner including: When the defect type is misalignment, a laser displacement sensor is used to measure the actual position of the current harness, and the positioning error is calculated by the actual position and the standard position. The formula is: , ,in, and The horizontal and vertical coordinates represent the actual position. is the positioning error of the horizontal coordinate of the actual position, and The horizontal and vertical coordinates represent the standard position, is the positioning error of the actual position ordinate; according to the positioning error, the chaotic mapping adaptive PID control model is used to adjust the movement of the motion control system to move the harness back to the standard position.

[0013] Furthermore, when the defect type is a reverse line, the motion control system is adjusted using a chaotic mapping adaptive PID control model, and the specific method of adjusting the direction angle of the wire beam includes: Use magnetic sensors to detect the magnetic field components on the horizontal and vertical coordinates of the harness, and calculate the actual harness direction angle through the magnetic field components. The formula is: ,in, and represents the components in the ordinate and abscissa directions, Indicates the actual harness direction angle. By calculating the actual harness direction angle deviation, the formula is: ,in, is the actual harness direction angle, is the standard harness direction angle, The chaos mapping adaptive PID control model is used to adjust the direction angle control value of the motion control system to adjust the direction of the wire harness for the deviation of the actual position direction angle.

[0014] A wire harness intelligent visual assembly guidance method, which is applied to the wire harness intelligent visual assembly guidance system, comprises: Step SS1, using a camera to capture a wire harness image, and performing preliminary processing to obtain a preliminary wire harness image; using ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; based on the wire harness contour, using geometric extraction technology to extract geometric feature data; wherein, environmental data is collected in real time using sensors; Step SS2: Based on the geometric feature data and the environmental data, the SVM model is used to detect the eligibility of the assembly process; Step SS3, when it is detected that the assembly process is unqualified, the GS-DNN model is used to determine the defect type; wherein, the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the temperature coefficient in the adjustment-Gumbel-Softmax activation function is set using the exponential decay-adaptive method; when the defect type is a wrong line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the movement of the harness; when the defect type is a reverse line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the direction angle of the harness.

[0015] The technical effects and advantages of the intelligent visual assembly guidance system and method of the wiring harness of the present invention are as follows: The present invention can accurately obtain the morphological characteristics of the wire harness and improve the accuracy of defect detection by combining high-resolution cameras, ACED edge detection technology, Euclidean technology, discrete second-order derivative method and vector geometry method; The SVM model is used to perform conformity testing based on geometric feature data and environmental data, effectively improving the robustness and generalization ability of the test; The GS-DNN model is used to judge the defect type, combined with the adjustment-Gumbel-Softmax activation function, which can accurately distinguish between wrong and reverse lines and improve the classification accuracy; The temperature parameters are dynamically adjusted through the exponential decay-adaptive method to optimize the defect classification process, making the model more adaptive during the training process; The actual position of the misaligned line is measured by a laser displacement sensor, and the motion control system is adjusted using the chaotic mapping adaptive PID control model to achieve high-precision correction of the harness position. A magnetic sensor is used to detect the magnetic field component of the reverse line, and the deviation between the actual direction angle and the standard direction angle is calculated. The direction of the harness is then accurately adjusted using the chaotic mapping adaptive PID control model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a wiring harness intelligent visual assembly guidance system according to the present invention; Figure 2 It is a flow chart of the GS-DNN model of the present invention; Figure 3 It is a schematic diagram of a wire harness intelligent visual assembly guidance method of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1

[0019] See also Figure 1 As shown, this embodiment provides a wiring harness intelligent visual assembly guidance system, including: Information capture unit: Use a camera to capture the wire harness image and perform preliminary processing to obtain a preliminary wire harness image; use ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; based on the wire harness contour, use geometric extraction technology to extract geometric feature data; among them, environmental data is collected in real time using sensors; Qualification judgment unit: Based on geometric feature data and environmental data, the SVM model is used to detect the qualification of the assembly process; Intelligent control unit: When an unqualified assembly process is detected, the GS-DNN model is used to determine the defect type; the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the exponential decay-adaptive method is used to set the temperature coefficient in the adjustment-Gumbel-Softmax activation function; when the defect type is a wrong line, the chaotic mapping adaptive PID control model is used to adjust the motion control system and adjust the movement of the wiring harness; when the defect type is a reverse line, the chaotic mapping adaptive PID control model is used to adjust the motion control system and adjust the direction angle of the wiring harness.

[0020] The specific methods of capturing the wire harness image using a camera and performing preliminary processing to obtain the preliminary wire harness image include: The collected wire beam image is denoised using a Gaussian blur model to obtain a denoised wire beam image; the denoised wire beam image is contrast enhanced using a histogram equalization technique to obtain a preliminary wire beam image; The Gaussian blur model is an image smoothing technology that uses a Gaussian function to perform a convolution operation on an image, thereby suppressing the noise in the image. By replacing each pixel with the weighted average of its neighborhood, the random noise in the image is effectively reduced. Gaussian blur can retain the overall structure of the image while reducing detail noise. It is suitable for removing high-frequency noise in the image, improving the image quality, and providing a cleaner image foundation for subsequent processing. Histogram equalization technology is an image processing method that enhances image contrast. It adjusts the grayscale distribution of the image to make the brightness or grayscale value distribution of the image more uniform. This technology expands the dynamic range of the image by stretching the grayscale range of the image, making the details of the image more obvious, especially the details of the dark and bright areas are highlighted, thereby enhancing the visual effect and improving the overall visibility of the image.

[0021] The specific methods of using ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image include: Step aa1: Use the Sobel operator to calculate the horizontal and vertical gradients of each pixel in the preliminary line beam image. Based on the horizontal and vertical gradients, the Laplacian operator is introduced to calculate the gradient amplitude. The formula is: ,in, is the adjustment factor, with a value range of 0.5 to 1, and is selected using a random method. is the gradient amplitude, and are the gradients in the horizontal and vertical directions respectively, Indicates the preliminary harness image at position Laplacian gradient at ; and The formula is: , ,in, and are the horizontal and vertical coordinates of the preliminary line beam image, and Represents the horizontal and vertical offsets respectively. and Respectively represent the position of the Sobel horizontal convolution kernel and the vertical convolution kernel The coefficient value at Indicated in The pixel value at ; in, , ; The Laplacian formula is: ,in, ,in, Indicates the preliminary harness image at position The Laplace gradient at Represents the position of the convolution kernel of the Laplacian operator The coefficient value of The coefficient value is the value at each position in the convolution kernel; , and A typical horizontal Sobel operator , vertical Sobel operator and the Laplacian operator; The gradient magnitude indicates the intensity of the brightness change of the preliminary line beam image, reflecting the strength of the edge in the image. Usually, an operator like Sobel is used to calculate the gradients in the horizontal and vertical directions, and then the gradient magnitude is obtained by calculating the synthesis of these two gradients. However, when using Sobel algorithm for edge detection, weak edges or low-contrast edges are easily lost. Weak edges and low-contrast edges represent subtle changes in the image. Although their brightness changes are not as significant as strong edges, they still contain important details of the image. Therefore, the Laplacian operator is introduced to calculate the gradient amplitude in order to enhance the edge information in the image, especially for those edges with weak changes or low contrast; by combining the second-order derivative characteristics of the Laplacian operator, the details and edges in the preliminary line image can be highlighted, the influence of noise can be reduced, and subtle and low-contrast edges can be detected more accurately, thereby improving the accuracy and robustness of the overall edge detection; Step aa2: Dynamically adjust the high and low thresholds by calculating the mean and standard deviation of the gradient amplitude. The formula is: , ,in, is the mean value of the gradient amplitude, is the standard deviation of the gradient amplitude, is the adjustment factor, ranging from 1.5 to 2. is the high threshold, is the low threshold; Existing edge detection algorithms rely on fixed high and low thresholds, which requires constant parameter adjustment when processing different images. Fixed thresholds cannot adapt to changes in image content, resulting in poor results for images with more noise, lower contrast, or complex edges. Therefore, by calculating the mean and standard deviation of the gradient amplitude to dynamically adjust the high and low thresholds, the algorithm has adaptability and can automatically adjust the threshold according to the gradient changes in different image areas, thereby improving the accuracy of edge detection. Step aa3: When the gradient amplitude of a pixel point is greater than the high threshold, it is a strong edge point and is directly retained; when the gradient amplitude of a pixel point is between the low threshold and the high threshold, step aa4 is performed to determine whether to retain or suppress; when the pixel point is less than the low threshold, it is a weak edge point and is suppressed; Step aa4, calculate the gradient direction based on the gradients in the horizontal and vertical directions; for each pixel point, use bilinear interpolation to calculate the interpolation gradient amplitude of two adjacent pixel points in the gradient direction, and compare the interpolation gradient amplitude with the gradient amplitude of the pixel point; when When , the pixel is retained; when When , the pixel is suppressed; The gradient direction formula is: ; The interpolation gradient amplitude formula is: ,in, Represents the weight factor, which ranges from 0 to 1 and is selected randomly. Represents the gradient magnitude of the current pixel position after interpolation; When the gradient amplitude of a pixel is between the low threshold and the high threshold, the existing edge detection algorithm usually uses the method of connecting edges to determine whether to retain the pixel. This method is simple, but is easily disturbed by noise and local gradient changes, resulting in inaccurate edge judgment, especially in areas with gentle gradient changes or more noise, which may mistakenly retain some unreal edges or ignore details; In comparison, calculating the gradient amplitude of adjacent pixels through bilinear interpolation provides a more detailed and adaptive processing method. By comparing the interpolated gradient with the gradient amplitude of the pixel point, the true strength of the edge can be dynamically determined.

[0022] Based on the wire harness profile, the specific methods of using geometric extraction technology to extract geometric feature data include: The geometric feature data includes wire harness length data, wire harness curvature data, and wire harness angle data; Based on the extracted wire harness contour, a contour tracking algorithm is used to obtain the wire harness contour point set: ,in, represents the ath harness contour point, represents the nth line bundle contour point, n represents the total number of line bundle contour point sets, and a represents the index of the line bundle contour point set; Through the wire harness contour point set, the distance between adjacent points is calculated using the Euclidean technique to obtain the wire harness length data. The formula is: ,in, and Indicates The horizontal and vertical coordinates of the line contour points, Indicates harness length data; The discrete second-order derivative method is used to calculate the bundle curvature data, and the formula is: ,in, represents the curvature of the a-th contour point, The power keeps the curvature relatively stable at different scales and does not change with distance; and Indicates the contour point from the ath contour point to the right The displacement of and Indicates the contour point a to the left of the contour point The displacement of Represents the square of the Euclidean distance from the ath contour point to the right contour point; By using the vector geometry method, the angle data of the line bundle between the ath contour point and the left contour point and the right contour point is calculated; first, the contour point is calculated To contour point The vector of is: , calculate the contour points To contour point The vector of is: ; Calculate the contour points again and contour points Harness angle data between: ; Contour tracking is an image processing technique used to extract boundary point sets of wire harnesses. The core idea is to gradually track all boundary points along the wire harness contour to form a complete contour point set.

[0023] Based on geometric feature data and environmental data, the specific methods of using the SVM model to detect the eligibility of the assembly process include: Environmental data Environmental data includes temperature data, humidity data and electrostatic data, which are collected by installing temperature sensors, humidity sensors and capacitive sensors in the assembly environment; Based on the geometric feature data and environmental data, the interpolation method is used to process missing values ​​and outliers, and the Min-Max method is used for normalization to obtain the geometric environment feature data set; the geometric environment feature data set is used as the input of the SVM model; Use grid search to initialize hyperparameters, including penalty coefficient C, kernel function, gamma parameter, and maximum number of iterations; use random method to initialize weight vector and the bias term ; Calculate the initial loss function value of the SVM model ,in, represents regularization, represents the classification error term, is the penalty coefficient, is the total number of samples in the input data set, is the sample index, is the label of the fth sample, is the input data of the fth sample; In each iteration, use The optimizer updates the weight vector and bias term, and calculates the new loss function during the update process; When the maximum number of iterations is reached, the iteration is stopped and the final weight vector is output. and the bias term , using the final weight vector and the bias term Calculate the decision function value ,in, For input data, For input data The decision function value of , indicating that the assembly process is qualified. , indicating that the assembly process is unqualified; Among them, temperature changes will affect the physical properties of the wiring harness materials, especially the expansion and contraction of the wires and insulation layers. Under high or low temperature conditions, the geometry and flexibility of the wiring harness will change, which will affect the accuracy and stability of the assembly process. If these temperature changes are not taken into account, it will lead to assembly errors and thus cause unqualified inspections. Excessive humidity can cause electrical short circuits, signal attenuation or reflection changes, etc., which can increase the noise in the sensor output signal and interfere with the accuracy of direction judgment; Static electricity is a common environmental factor in the manufacturing and assembly process. Especially in a dry environment, the accumulation of static electricity will seriously affect the safety and assembly accuracy of electronic components. Under the action of static electricity, wiring harness materials and wires may undergo slight displacement or charge accumulation, which will affect the accuracy and stability of the assembly process.

[0024] See also Figure 2 As shown, the process of the GS-DNN model in this embodiment includes: When an assembly process is detected to be unqualified, the specific methods of using the GS-DNN model to determine the defect type include: The input layer takes the geometric feature data and environmental data of the unqualified assembly process as input; the number of neurons is set to be equal to the number of input data; Hidden layer, set L_l hidden layers; pass the input data from the input layer to the first hidden layer, and obtain the output of the first hidden layer through linear transformation and SinReLu activation function. The formula is: ,in, is the output of the first hidden layer, is the weight of the first hidden layer, The geometric feature data and environmental data input to the input layer, is the bias term of the first hidden layer; the output of the first hidden layer is passed to the next hidden layer, and the SinReLu activation function is repeatedly used to calculate the output of the next hidden layer until the last hidden layer ends and the output of the last hidden layer is obtained ; The output layer passes the output of the last hidden layer to the output layer and uses a linear transformation to transform the hidden layer output Projected into the 2D defect type space, output the output value of the 2D defect type, , Indicates the output layer Output value of each defect type, Indicates the output layer The weight of each defect type, The output layer The first dimension defect type is line breakage, and the second dimension defect type is line reversal; Calculate the probability of defect type. Based on the output value of the 2D defect type output by the output layer, use the Adjust-Gumbel-Softmax activation function to calculate the probability of the 2D defect type. Determine the final defect type. When the set number of iterations is reached, stop the iteration and output the final defect type probability. Set the defect type with the highest probability as the main defect in the assembly process. When the output misalignment probability is the highest, the defect type is misalignment, and the one-hot encoding is expressed as ; When the probability of outputting a reverse line is the largest, the defect type is a reverse line, and the one-hot encoding is expressed as .

[0025] Using the Adjustment-Gumbel-Softmax activation function, the specific method of calculating the probability of each defect type includes: Using the Adjustment-Gumbel-Softmax activation function, the probability of each defect type is calculated as follows: ,in, Indicates the defect type index, Indicated in The output layer outputs the Output value of each defect type, Indicates In the iteration The probability of a defect type, Indicates In the iteration Gubel noise for each defect type, Indicates In the iteration The adjustment temperature coefficient of each defect type is set using the exponential decay-adaptive method. is the normalization factor, Indicates the defect type index, Indicates The output layer outputs the Output value of each defect type, No. In the iteration Gubel noise for each defect type, Indicates In the iteration The adjustment temperature coefficient for each defect type is is the index of the iteration number; in, The calculation formula is: , It is a random number collected from the range 0 to 1 using a random method; Similarly; The temperature parameter here is a hyperparameter and has nothing to do with the temperature data collected by the temperature sensor; The Softmax activation function is often used for classification problems. It converts the output value of the output layer into a probability distribution. However, one drawback of the Softmax function is that it relies on the global maximum value for normalization, which can lead to over-smoothing of smaller categories (i.e., categories with lower probabilities). Especially when the gap between categories is small, it may make it difficult to distinguish between categories with small differences. For example, the output values ​​of the defect types output by the output layer are 3 and 2.9, and the probabilities calculated by the Softmax activation function are 0.511 and 0.488. The difference between the two is very small, making it difficult to ensure that the model can accurately determine which defect type it is.

[0026] The Adjustment-Gumbel-Softmax activation function combines Gumbel noise and temperature adjustment mechanisms, overcoming the smoothing problem of Softmax and providing more flexible control. The use of the adjustment temperature coefficient can adjust the temperature according to the different iterations during the training process, so that the training process can focus more accurately on the more informative categories, thereby effectively improving the model's ability to distinguish subtle differences in categories.

[0027] The specific methods of using the exponential decay-adaptive method to set the adjustment temperature coefficient include: The temperature parameter is calculated using the exponential decay method, the formula is: ,in, For the In the iteration Exponential temperature coefficients for each defect type, is the initial temperature coefficient, determined using the random method, is the attenuation coefficient, which controls the rate of change of the temperature coefficient and is determined using the random method; The temperature parameters are calculated using the adaptive method, and the formula is: ,in, Indicates In the iteration Adaptive temperature coefficients for each defect type, is the adjustment coefficient, which is determined by random method. Indicates Iteration No. The entropy of defect types is: , Indicates In the iteration The probability of each defect type is calculated by the Softmax function; The adjustment temperature coefficient is calculated by combining the exponential temperature coefficient and the adaptive temperature coefficient. The formula is: ,in, Indicates In the iteration Adjustment temperature coefficient for each defect type; The existing temperature parameters are set using fixed values, usually set to 1 or 0.5. Fixed temperature parameters cannot be adaptively adjusted for different data; Using the exponential decay method to calculate the temperature parameter can gradually reduce the temperature coefficient as the training progresses. The exponential decay method can help the model maintain a large degree of exploration in the early stage, and gradually reduce the degree of exploration as the training progresses, avoiding the problem of overfitting in the early stage and helping the model converge better. The temperature parameters are calculated using the adaptive method. The temperature parameters can be dynamically adjusted according to the entropy value of each defect type during the training process. When the model has a large uncertainty about a certain category, the adaptive method will increase the temperature value, allowing more exploration.

[0028] When the defect type is misalignment, the motion control system is adjusted using the chaotic mapping adaptive PID control model. The specific methods for adjusting the movement of the wire harness include: When the defect type is misalignment, a laser displacement sensor is used to measure the actual position of the current harness, and the positioning error is calculated by the actual position and the standard position. The formula is: , ,in, and The horizontal and vertical coordinates represent the actual position. is the positioning error of the horizontal coordinate of the actual position, and The horizontal and vertical coordinates represent the standard position, is the positioning error of the actual position ordinate; according to the positioning error, the chaotic mapping adaptive PID control model is used to adjust the movement of the motion control system to move the harness back to the standard position.

[0029] When the defect type is reverse line, the motion control system is adjusted using the chaotic mapping adaptive PID control model. The specific methods of adjusting the direction angle of the wire beam include: Use magnetic sensors to detect the magnetic field components on the horizontal and vertical coordinates of the harness, and calculate the actual harness direction angle through the magnetic field components. The formula is: ,in, and represents the components in the ordinate and abscissa directions, Indicates the actual harness direction angle. By calculating the actual harness direction angle deviation, the formula is: ,in, is the actual harness direction angle, is the standard harness direction angle, The chaotic mapping adaptive PID control model is used to adjust the direction angle control amount of the motion control system to adjust the direction of the wire harness for the deviation of the actual position direction angle. The control effect of the PID control model is highly dependent on the selection of three coefficients: proportional coefficient, integral coefficient and differential coefficient. In practical applications, selecting appropriate PID coefficients often requires a lot of experiments and adjustments. Once it is determined that the entire operation step uses a fixed PID coefficient; different working conditions and system characteristics may require different coefficients, which makes the adjustment process of the PID controller very cumbersome and prone to errors. Chaotic mapping is used to adjust the coefficients of the PID control model to obtain a chaotic mapping adaptive PID control model. The specific methods include: The formula of the chaotic map adaptive PID control model is: ,in, Indicates the control amount, Indicates positioning error or deviation, that is, , or , represents the proportionality coefficient, is the integration coefficient, is the differential coefficient; Among them, for The proportional coefficient is dynamically updated using Logistic mapping, and the formula is: ,in, is the initial proportionality factor, is the adjustment factor, ranging from 0 to 1, and is selected using the random method. For the The state of the generation number, To control the parameters, the range is from 1 to 4, and the random method is used for calling; against The integral coefficient is dynamically updated using the Henon mapping, and the formula is: ,in, is the initial integration coefficient, is the adjustment factor, ranging from 0 to 1, and is selected using the random method. and To control the parameters, The range is 1 to 2. The range is from 0.1 to 0.9, and the random method is used for selection; against The differential coefficient is dynamically updated using Tent mapping, and the formula is: ,in, is the initial differential coefficient, is the adjustment factor, ranging from 0 to 1, and is selected using the random method. To control the parameters, the range is from 1 to 4, and the random method is used for calling; Among them, the initial proportional coefficient, integral coefficient and differential coefficient are taken as a smaller value by empirical method, the initial state value is between 0 and 1, and a random number is selected as the initial state by random method; The PID controller is composed of three core coefficients: proportional coefficient, integral coefficient and differential coefficient. These coefficients determine the control system's response to errors. In traditional PID controllers, the three coefficients are usually fixed, which may not provide the best control effect when facing dynamic changes or nonlinear control systems. Therefore, using chaotic mapping to dynamically adjust these coefficients can improve the robustness, stability and adaptability of the control system. The proportional coefficient controls the impact of the current positioning error or deviation on the response of the motion control system, affecting the sensitivity and response speed of the motion control system. For the proportional coefficient, a chaotic mapping with high sensitivity and fast response characteristics should be selected so that it can quickly adapt to the positioning error or deviation changes of the system. Logistic mapping is one of the most classic chaotic mappings, with strong nonlinearity and sensitivity to initial conditions, and is suitable for dynamic adjustment of the proportional coefficient. It can provide a fast and sensitive response for the proportional coefficient, so that the motion control system can flexibly adjust the control output according to the change of positioning error or deviation. This is very useful for fast-changing motion control systems, especially for situations where fast response is required when the positioning error or deviation is large. The integral coefficient determines the response of the motion control system to long-term positioning errors or deviations. It is crucial to eliminate steady-state positioning errors or deviations, but it may also cause the integral wind-up of the motion control system (when the positioning error or deviation accumulates too much, it may cause the control output to be too large); for the integral coefficient, it is necessary to select a chaotic mapping that can produce smooth and slow changes to avoid affecting the stability of the system; Henon mapping is a classic two-dimensional discrete chaotic mapping with strong stability and low sensitivity. It is suitable for controlling the integral parameter because it can produce slow and smooth changes, avoiding excessive increase of the integral parameter when the system positioning error or deviation has not been eliminated for a long time, and preventing the occurrence of the integral wind-up problem; The differential coefficient controls the response of the motion control system to the positioning error or the rate of change of the deviation, and is mainly used to slow down the oscillation of the motion control system and improve the stability of the motion control system; the adjustment of the differential coefficient requires fast response and greater sensitivity to ensure that the system can respond to the positioning error or the rate of change of the deviation in a timely manner. Since the differential coefficient is usually sensitive to the changes in the motion control system, it is necessary to select a chaotic mapping with high sensitivity and fast response characteristics; Tent mapping is a chaotic mapping with a simple form and strong nonlinearity. It changes quickly and can respond quickly to small input changes. It is suitable for controlling the differential coefficient because it can quickly adjust the differential coefficient when controlling the positioning error or deviation change of the motion control system, thereby improving the system's response speed to dynamic changes; The motion control system is a mechatronic system used to precisely control the position and direction of the wiring harness, usually consisting of a servo motor, a driver, a controller, and a sensor. The system receives real-time position and direction data from a laser displacement sensor or a magnetic sensor, calculates the adjustment amount using a chaotic mapping adaptive PID control model, and then drives the actuator (such as a motor or a robotic arm) to move and rotate precisely, so that the actual position and direction of the wiring harness gradually approaches the standard position and direction, thereby correcting defects such as misaligned or reversed wiring and ensuring accurate installation and arrangement of the wiring harness.

[0030] In this embodiment, by combining a high-resolution camera, ACED edge detection technology, Euclidean technology, discrete second-order derivative method, and vector geometry method, the morphological characteristics of the wire harness can be accurately obtained, thereby improving the accuracy of defect detection; The SVM model is used to perform conformity testing based on geometric feature data and environmental data, effectively improving the robustness and generalization ability of the test; The GS-DNN model is used to judge the defect type, combined with the adjustment-Gumbel-Softmax activation function, which can accurately distinguish between wrong and reverse lines and improve the classification accuracy; The temperature parameters are dynamically adjusted through the exponential decay-adaptive method to optimize the defect classification process, making the model more adaptive during the training process; The actual position of the misaligned line is measured by a laser displacement sensor, and the motion control system is adjusted using the chaotic mapping adaptive PID control model to achieve high-precision correction of the harness position. A magnetic sensor is used to detect the magnetic field component of the reverse line, and the deviation between the actual direction angle and the standard direction angle is calculated. The direction of the harness is then accurately adjusted using the chaotic mapping adaptive PID control model.

[0031] Example 2

[0032] See also Figure 3 As shown, the part not described in detail in this embodiment is described in Example 1, which provides a wiring harness intelligent visual assembly guidance method, including: Step SS1, using a camera to capture a wire harness image, and performing preliminary processing to obtain a preliminary wire harness image; using ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; based on the wire harness contour, using geometric extraction technology to extract geometric feature data; wherein, environmental data is collected in real time using sensors; Step SS2: Based on the geometric feature data and the environmental data, the SVM model is used to detect the eligibility of the assembly process; Step SS3, when it is detected that the assembly process is unqualified, the GS-DNN model is used to determine the defect type; wherein, the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the temperature coefficient in the adjustment-Gumbel-Softmax activation function is set using the exponential decay-adaptive method; when the defect type is a wrong line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the movement of the harness; when the defect type is a reverse line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the direction angle of the harness.

[0033] Example 3

[0034] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the intelligent visual assembly guidance system and method for wiring harnesses provided above is implemented.

[0035] Since the electronic device introduced in this embodiment is an electronic device used to implement a wire harness intelligent visual assembly guidance system and method in the embodiment of the present application, based on the wire harness intelligent visual assembly guidance system and method introduced in the embodiment of the present application, the technical personnel of the field can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as the technical personnel of the field implement the electronic device used in the wire harness intelligent visual assembly guidance system and method in the embodiment of the present application, it belongs to the scope of protection of this application.

[0036] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0037] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A wire harness intelligent visual assembly guidance system, characterized in that: include: Information capture unit: Use a camera to capture the wire harness image and perform preliminary processing to obtain a preliminary wire harness image; the preliminary processing steps include denoising and contrast enhancement; use ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; Based on the wire harness profile, geometric feature data is extracted using geometric extraction technology; geometric extraction technology includes Euclidean technology, discrete second-order derivative method and vector geometry method; among them, environmental data is collected in real time using sensors; Qualification judgment unit: Based on geometric feature data and environmental data, the SVM model is used to detect the qualification of the assembly process; Intelligent control unit: When an assembly process is detected to be unqualified, the GS-DNN model is used to determine the defect type; in the GS-DNN model, the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the temperature coefficient in the adjustment-Gumbel-Softmax activation function is set using the exponential decay-adaptive method; When the defect type is misalignment, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the movement of the wire harness; when the defect type is reversed, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the direction angle of the wire harness; Among them, in the chaotic map adaptive PID control model, for The proportional coefficient is dynamically updated using Logistic mapping; for The integral coefficient is dynamically updated using the Henon map; for The differential coefficients are dynamically updated using Tent mapping.

2. The intelligent visual assembly guidance system for wiring harnesses according to claim 1 is characterized in that: The specific method of using a camera to capture the wire harness image and performing preliminary processing to obtain the preliminary wire harness image includes: The collected wire beam image is denoised using a Gaussian blur model to obtain a denoised wire beam image; the denoised wire beam image is contrast enhanced using a histogram equalization technique to obtain a preliminary wire beam image.

3. The intelligent visual assembly guidance system for wiring harnesses according to claim 2 is characterized in that: The specific method of using the ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image includes: Step aa1, using the Sobel operator to calculate the gradient of each pixel in the preliminary line beam image in the horizontal direction and the vertical direction; introducing the Laplacian operator based on the gradient in the horizontal direction and the vertical direction to calculate the gradient amplitude; Step aa2, dynamically adjusting the high and low thresholds by calculating the mean and standard deviation of the gradient amplitude; Step aa3: When the gradient amplitude of a pixel point is greater than the high threshold, it is a strong edge point and is directly retained; when the gradient amplitude of a pixel point is between the low threshold and the high threshold, step aa4 is performed to determine whether to retain or suppress; when the pixel point is less than the low threshold, it is a weak edge point and is suppressed; Step aa4, calculate the gradient direction based on the gradients in the horizontal and vertical directions; for each pixel point, use bilinear interpolation to calculate the interpolation gradient amplitude of two adjacent pixel points in the gradient direction, and compare the interpolation gradient amplitude with the gradient amplitude of the pixel point; when the gradient amplitude of the pixel point is greater than or equal to the interpolation gradient amplitude, retain the pixel point; when the gradient amplitude of the pixel point is less than the interpolation gradient amplitude, suppress the pixel point.

4. The intelligent visual assembly guidance system for wiring harnesses according to claim 3 is characterized in that: The specific method of extracting geometric feature data by using geometric extraction technology based on the wire harness contour includes: The geometric feature data includes wire harness length data, wire harness curvature data, and wire harness angle data; Based on the extracted wire harness contour, a contour tracking algorithm is used to obtain a wire harness contour point set; Through the wire harness contour point set, the distance between adjacent points is calculated using the Euclidean technique to obtain the wire harness length data; Based on the displacement from the ath contour point to the left contour point and the square of the Euclidean distance from the ath contour point to the right contour point, the discrete second-order derivative method is used to calculate the bundle curvature data; By using the vector geometry method, the beam angle data between the ath contour point and the left contour point and the right contour point are calculated.

5. The intelligent visual assembly guidance system for wire harnesses according to claim 4 is characterized in that: The specific method of using the SVM model to detect the eligibility of the assembly process based on the geometric feature data and the environmental data includes: Based on the geometric feature data and environmental data, the interpolation method is used to process missing values ​​and outliers, and the Min-Max method is used for normalization to obtain the geometric environment feature data set; the geometric environment feature data set is used as the input of the SVM model; Use grid search to initialize hyperparameters, including penalty coefficient C, kernel function, gamma parameter, and maximum number of iterations; use random method to initialize weight vector and the bias term ; Calculate the initial loss function value of the SVM model; In each iteration, use The optimizer updates the weight vector and bias term, and calculates the new loss function during the update process; When the set maximum number of iterations is reached, the iteration is stopped, the final weight vector and bias term are output, and the decision function value is calculated using the final weight vector and bias term. When the decision function value is greater than 0, it indicates that the assembly process is qualified. When the decision function value is less than or equal to 0, it indicates that the assembly process is unqualified.

6. The intelligent visual assembly guidance system for wire harnesses according to claim 5 is characterized in that: When the assembly process is detected to be unqualified, the specific method of using the GS-DNN model to determine the defect type includes: The input layer takes the geometric feature data and environmental data of the unqualified assembly process as input; the number of neurons is set to be equal to the number of input data; Hidden layer, set L_l hidden layers; pass the input data from the input layer to the first hidden layer, obtain the output of the first hidden layer through linear transformation and SinReLu activation function, pass the output of the first hidden layer to the next hidden layer, repeatedly use SinReLu activation function, calculate the output of the next hidden layer, until the end of the last hidden layer, obtain the output of the last hidden layer; The output layer passes the output of the last hidden layer to the output layer, uses linear transformation to project the hidden layer output to the 2D defect type space, and outputs the output value of the 2D defect type; the first-dimensional defect type is a broken line, and the second-dimensional defect type is a reverse line; Calculate the probability of defect type. Based on the output value of the 2D defect type output by the output layer, use the Adjust-Gumbel-Softmax activation function to calculate the probability of the 2D defect type. Determine the final defect type. When the set number of iterations is reached, stop the iteration and output the final defect type probability. Compare the final probabilities of the two-dimensional defect types and take the one with the larger probability as the main defect of the assembly process. When the final probability of the first-dimensional defect type is greater than the second-dimensional defect type, the defect type is a misalignment, and the one-hot encoding is represented as ; When the final probability of the defect type in the first dimension is less than that of the defect type in the second dimension, the defect type is reversed, and the one-hot encoding is expressed as .

7. The intelligent visual assembly guidance system for wire harnesses according to claim 6 is characterized in that: The specific method of using the adjustment-Gumbel-Softmax activation function to calculate the probability of each defect type includes: For the output layer output The output value of the defect type, The Gubel noise of the first defect type and The adjustment temperature coefficient of the defect type is calculated using the adjustment-Gumbel-Softmax activation function. The probability of a defect type; The calculation steps of the adjustment temperature coefficient are as follows: for the initial temperature coefficient, using the exponential decay method to calculate the exponential temperature coefficient; using the adaptive method to calculate the adaptive temperature coefficient; and combining the exponential temperature coefficient and the adaptive temperature coefficient to calculate the adjustment temperature coefficient.

8. The intelligent visual assembly guidance system for wire harnesses according to claim 7 is characterized in that: When the defect type is a misaligned line, the specific method of adjusting the movement amount of the wire harness by adjusting the motion control system using the chaotic mapping adaptive PID control model includes: When the defect type is misalignment, a laser displacement sensor is used to measure the actual position of the current wire harness, and the positioning error is calculated through the actual position and the standard position. According to the positioning error, a chaotic mapping adaptive PID control model is used to adjust the movement amount of the motion control system to move the wire harness back to the standard position.

9. The intelligent visual assembly guidance system for wire harnesses according to claim 8, characterized in that: When the defect type is a reverse line, the motion control system is adjusted using a chaotic mapping adaptive PID control model, and the specific method of adjusting the direction angle of the wire beam includes: Magnetic sensors are used to detect the magnetic field components on the horizontal and vertical coordinates of the harness, and the actual harness direction angle is calculated through the magnetic field components; the actual harness direction angle deviation is calculated based on the actual harness direction angle and the standard harness direction angle; based on the actual harness direction angle deviation, the chaotic mapping adaptive PID control model is used to calculate the direction angle control amount, and the direction angle control amount of the motion control system is adjusted to adjust the harness direction.

10. A wire harness intelligent visual assembly guidance method, applied to the wire harness intelligent visual assembly guidance system according to any one of claims 1 to 9, characterized in that: include: Step SS1, using a camera to capture a wire harness image, and performing preliminary processing to obtain a preliminary wire harness image; using ACED edge detection technology to outline the wire harness contour from the preliminary wire harness image; based on the wire harness contour, using geometric extraction technology to extract geometric feature data; wherein, environmental data is collected in real time using sensors; Step SS2: Based on the geometric feature data and the environmental data, the SVM model is used to detect the eligibility of the assembly process; Step SS3, when it is detected that the assembly process is unqualified, the GS-DNN model is used to determine the defect type; wherein, the adjustment-Gumbel-Softmax activation function is used to calculate the probability of each defect type; the temperature coefficient in the adjustment-Gumbel-Softmax activation function is set using the exponential decay-adaptive method; when the defect type is a wrong line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the movement of the harness; when the defect type is a reverse line, the motion control system is adjusted using the chaotic mapping adaptive PID control model to adjust the direction angle of the harness.

Images