A visually assisted method for detecting industrial wiring harness quality

Abstract

The present invention belongs to the field of industrial automation detection technology, and provides a visually assisted industrial wire harness quality detection method, including: setting multiple industrial cameras to shoot industrial wire harnesses, obtaining a preliminary screening image group, using edge detection and convolutional neural networks to identify damaged areas, and judging whether the industrial wire harnesses are severely damaged. If the industrial wire harnesses are not damaged or slightly damaged, the industrial wire harnesses are quality tested. During the test, cameras are set to shoot industrial wire harness images to construct a time-series quality inspection image sequence, the image sequence is analyzed by the optical flow method to determine the damaged area and the superimposed degradation speed, a time-series prediction model is constructed and trained, the damage condition of the industrial wire harnesses is predicted, and whether to stop the quality test is determined according to the prediction results. If the quality test cycle meets the requirements, a comprehensive evaluation of the industrial wire harness quality is triggered, and the quality of the industrial wire harnesses is evaluated and graded based on the comprehensive visual damage indicators and physical and chemical test indicators.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial automation detection, and in particular relates to a visually assisted industrial wire harness quality detection method. Background Art

[0002] In modern industrial production, industrial wiring harnesses are key components that connect various parts in the electrical system. Their quality directly affects the performance, reliability and safety of the entire equipment or system. With the continuous improvement of industrial automation and intelligence, higher requirements are placed on the accuracy, efficiency and comprehensiveness of industrial wiring harness quality inspection.

[0003] Traditional industrial wire harness quality inspection mainly relies on manual inspection and some simple automated inspection methods. Manual inspection has many disadvantages. Inspectors are prone to visual fatigue after working for a long time, resulting in frequent missed inspections and false inspections, making it difficult to ensure the accuracy and consistency of inspections. In addition, manual inspection is inefficient and cannot meet the needs of large-scale industrial production. Simple automated inspection methods, such as those based on contact sensors, can only detect some physical parameters of the wire harness, and are difficult to effectively detect problems such as surface damage and internal structural defects of the wire harness.

[0004] With the development of machine vision technology, vision-based industrial wire harness detection methods have gradually been applied. However, in the existing technology, some detection methods only use a single visual algorithm to identify wire harness damage. For example, relying solely on edge detection algorithms is easily affected by noise interference in complex industrial environments, resulting in misjudgment of damaged areas and inability to accurately identify minor damage or potential damage areas. On the other hand, existing methods often only focus on the state detection of the wire harness at a certain moment, lack the ability to dynamically monitor and predict its quality degradation process, and cannot predict in advance the quality change trend of the wire harness during subsequent use, making it difficult to take effective measures in time during the quality testing stage. In addition, in terms of comprehensive quality assessment, most of the existing detection methods process visual inspection results and physical and chemical test results separately, and fail to establish a reasonable evaluation model to effectively combine the two, resulting in the quality assessment results being unable to fully and accurately reflect the actual quality status of industrial wire harnesses.

[0005] In order to solve the above problems, the present invention proposes a visually assisted industrial wiring harness quality detection method. Summary of the Invention

[0006] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0007] The technical solution adopted by the present invention to solve the technical problem is: a visually assisted industrial wiring harness quality detection method, comprising:

[0008] Obtain a preliminary screening image set of industrial wiring harnesses and preliminarily screen out those with severe damage;

[0009] Specifically, a preliminary screening image group of industrial wiring harnesses is obtained, and industrial wiring harness damage is identified on the images in the preliminary screening image group through edge detection and convolutional neural network. The weight of each damaged area in the industrial wiring harness image is calculated according to the damage category and recognition confidence. The weighted calculation is combined with the damage degree of the damaged area to obtain the image damage value of the industrial wiring harness image.

[0010] If the image damage value of any industrial wiring harness image in the industrial wiring harness image group is greater than the preset damage threshold, it is determined that the industrial wiring harness is severely damaged and the quality of the industrial wiring harness is unqualified;

[0011] Start the quality test and collect industrial wiring harness images in real time to construct a time-series quality inspection image sequence. Combined with the optical flow method, the superimposed degradation rate of the industrial wiring harness is calculated in real time. The time-series degradation rate sequence and time-series image damage sequence are established. The time-series prediction model is trained to predict the degradation trend of the industrial wiring harness. The quality test is dynamically terminated. The duration of the quality test cycle is used to determine whether to trigger a comprehensive assessment of the industrial wiring harness quality.

[0012] Specifically, if the industrial wiring harness is not damaged or is slightly damaged, the industrial wiring harness is subjected to a quality test, and during the quality test, images of the industrial wiring harness are collected in real time to construct a time-series quality inspection image sequence;

[0013] The optical flow method is used to analyze the time-series quality inspection image sequence and calculate the actual motion speed of each pixel at the current detection moment. If the actual motion speed is greater than the motion speed threshold, it is marked as a moving pixel.

[0014] Taking each moving pixel as the center, check whether there is a moving pixel in its 5×5 pixel neighborhood. If there is, all the moving pixels in the neighborhood are grouped into the same damage area. If not, clear the pixel mark.

[0015] Obtain all moving pixels on the industrial wiring harness image at the current detection moment, normalize and calculate the actual movement speed of the moving pixels, and obtain the superimposed degradation speed of the industrial wiring harness at the current moment;

[0016] Obtain all superimposed degradation rates from the quality test period to the current detection moment, and convert them into a time series degradation rate sequence according to the time series set;

[0017] Obtain industrial wire harness images from a time-series quality inspection image sequence, input them into a trained convolutional neural network, identify the damaged area in each industrial wire harness image, and calculate the image damage value of each industrial wire harness image. The image damage sequence is then generated based on the time series set.

[0018] Obtain the time-series degradation rate sequence and time-series image damage sequence, build and train a gated recurrent unit model to predict the degradation rate and damage of industrial wiring harnesses, and obtain the predicted superimposed degradation rate and predicted image loss value of the industrial wiring harness at different viewing angles at each detection moment within the prediction period;

[0019] Obtain the predicted stacking degradation rate and predicted image loss value within the prediction period for analysis;

[0020] If any predicted superposition degradation speed is greater than the degradation speed threshold within the prediction period, or any predicted image loss value is greater than the preset damage threshold, the quality test of the industrial wiring harness is stopped and the quality test cycle ends;

[0021] Get the duration of the quality test cycle. If the duration of the quality test cycle is greater than or equal to the standard duration of the quality test cycle, trigger the comprehensive quality assessment of the industrial wiring harness.

[0022] If triggered, the system will integrate the timing degradation speed sequence, timing image damage sequence and quality test data, and use multi-index weighted evaluation to calculate the quality assessment score of the industrial wiring harness, thus achieving the quantitative classification of the quality level of the industrial wiring harness;

[0023] Specifically, if the comprehensive quality assessment of industrial wiring harnesses is triggered, each test in the quality test is divided into numerical tests and Boolean tests according to the test requirements. Based on the Boolean test, the test results are directly mapped to 0 or 1;

[0024] If there is a test result mapped to 0 in all Boolean tests, the industrial wiring harness quality is judged to be unqualified, otherwise, the Boolean test result is judged to be qualified;

[0025] Obtain the Boolean test results. If the Boolean test results are qualified, jointly analyze the visual damage indicators and the physical and chemical test indicators. Calculate the interaction weights of the visual damage indicators and the physical and chemical test indicators through the cross-modal attention mechanism. Based on the interaction weights, weighted fusion of the visual damage indicators and the physical and chemical test indicators is performed to calculate the quality assessment score of the industrial wiring harness.

[0026] The beneficial effects of the present invention are as follows:

[0027] 1. The present invention acquires a preliminary screening image group by setting up multiple industrial cameras, and combines edge detection with convolutional neural networks to identify damaged areas, which can more comprehensively capture various damage characteristics of industrial wiring harnesses and significantly improve the accuracy of damage identification. During the quality testing process, a time-series quality inspection image sequence is constructed and analyzed using the optical flow method. The pixel movement speed can be accurately calculated, the damaged area and the superimposed degradation speed can be determined, and detailed data support can be provided for subsequent analysis. The gated recurrent unit model predicts the degradation speed and image damage value, so that the detection is no longer limited to the current state, and the damage development trend can be predicted in advance, providing a more scientific and comprehensive basis for industrial wiring harness quality assessment, effectively ensuring product quality.

[0028] 2. This method optimizes the industrial wire harness inspection process. From initial screening to quality testing and then to comprehensive evaluation, each link is closely connected and efficient. Through intelligent image processing and analysis, manual intervention is reduced and inspection efficiency is improved. At the same time, the quality assessment score is calculated based on the visual damage index and physical and chemical test indicators within the comprehensive quality testing cycle, making the evaluation results more objective and accurate. The grading based on the evaluation score not only provides a clear definition of product quality, but also provides a strong reference for subsequent production improvements and quality control, helping enterprises improve their overall production level and market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart of the steps of a visually assisted industrial wiring harness quality inspection method according to an embodiment of the present invention;

[0031] Figure 2 This is a flowchart of the specific steps for establishing a time-series degradation speed sequence in a visually assisted industrial wiring harness quality inspection method according to an embodiment of the present invention;

[0032] Figure 3 This is a flowchart of the specific steps for determining whether to trigger a comprehensive evaluation of industrial wiring harness quality in a visually assisted industrial wiring harness quality detection method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0034] Example 1 Please refer to Figure 1 As shown, a visually assisted industrial wiring harness quality detection method according to an embodiment of the present invention includes the following steps:

[0035] S1: Set up multiple industrial cameras to shoot industrial wiring harnesses, obtain a preliminary screening image group, use edge detection and convolutional neural networks to identify damaged areas, calculate image damage values, and determine whether the industrial wiring harnesses are severely damaged based on the image damage values;

[0036] Set up several high-resolution industrial cameras with fixed camera positions and different shooting angles. The plane formed by the cameras is parallel to the plane of the industrial wire harness's motion trajectory. Use the set cameras to shoot the industrial wire harness. Collect all images taken of the same industrial wire harness into a preliminary screening image group to ensure that all parts of the industrial wire harness can be clearly imaged. Adjust the camera parameters to make them consistent to ensure uniform image quality.

[0037] Performing edge detection processing on images in the primary screening image group;

[0038] Specifically, the images in the initial screening image group are grayscale smoothed to obtain the corresponding preprocessed quality inspection images. For all preprocessed quality inspection images, each pixel of the preprocessed quality inspection image is traversed and the Sobel operator is used to calculate the gradient components of the pixel in the horizontal and vertical directions respectively. and , through the formula

[0039]

[0040] Calculate the gradient amplitude G of the pixel point and use the formula

[0041]

[0042] Calculate the gradient direction of the pixel;

[0043] Compare the pixel gradient amplitude with the adjacent pixels in the gradient direction. If the pixel gradient amplitude is the maximum value among the adjacent pixels in the gradient direction, retain it. Otherwise, set the pixel gradient amplitude to 0. Compare the pixel gradient amplitude obtained after traversal with the gradient amplitude high threshold and gradient amplitude low threshold.

[0044] If the pixel gradient amplitude is greater than or equal to the gradient amplitude high threshold, the pixel is marked as a true edge point;

[0045] If the pixel gradient amplitude is between the gradient amplitude high threshold and the gradient amplitude low threshold, the pixel is marked as a weak edge point;

[0046] If the pixel gradient amplitude is less than or equal to the gradient amplitude low threshold, it means that the pixel is not an edge point;

[0047] Taking each weak edge point as the center, check whether there is a real edge point in its 5×5 pixel neighborhood. If there is a real edge point in the neighborhood, mark the weak edge point as a real edge point and retain it. If there is no real edge point in the neighborhood, judge that the weak edge point is not an edge point and clear the pixel mark;

[0048] Based on the identified true edge points, the outline of the industrial wire harness in all pre-processed quality inspection images is obtained, the image within the outline of the industrial wire harness in the pre-processed quality inspection image is intercepted to obtain the corresponding industrial wire harness image, and the industrial wire harness images belonging to the same industrial wire harness are grouped into an industrial wire harness image group;

[0049] A convolutional neural network is used to train a large number of industrial wiring harness damage images based on big data. The industrial wiring harness image group is then input into the trained convolutional neural network to identify damaged areas in the industrial wiring harness images. The convolutional neural network outputs the range, damage category, and recognition confidence of each damaged area in the industrial wiring harness image.

[0050] Based on any industrial wiring harness image, the weight of each damaged area in the industrial wiring harness image is calculated according to the damage category and recognition confidence. The number of pixels contained in the damaged area is obtained according to the range of each damaged area. The number of pixels contained in each damaged area is normalized, and the normalized number of pixels is weighted based on the corresponding weight to calculate the image damage value of the industrial wiring harness image.

[0051] If the image damage value of any industrial wiring harness image in the industrial wiring harness image group is greater than the preset damage threshold, the industrial wiring harness is judged to be severely damaged; otherwise, the industrial wiring harness is judged to be undamaged or slightly damaged;

[0052] If the industrial wiring harness is severely damaged, it is judged that the quality of the industrial wiring harness is unqualified;

[0053] It should be noted that the purpose of this step is to obtain wire harness images through multi-angle industrial cameras, use edge detection and convolutional neural network technology to identify wire harness damage areas and calculate image damage values, and quickly determine whether the wire harness is severely damaged based on preset thresholds, thereby achieving preliminary screening of unqualified products, avoiding subsequent invalid inspection processes, and improving overall inspection efficiency.

[0054] S2: If the industrial wiring harness is undamaged or slightly damaged, perform a quality test on the industrial wiring harness. During the test, set a camera to capture images of the industrial wiring harness, perform edge detection and contour extraction, construct a time-series quality inspection image sequence, analyze the image sequence using the optical flow method, calculate the pixel point motion speed, determine the damaged area and superimpose the degradation speed, and obtain a time-series degradation speed sequence and a time-series image damage sequence.

[0055] like Figure 2As shown, the specific steps for establishing the timing degradation speed sequence are as follows:

[0056] If the industrial wiring harness is not damaged or the damage is minor, further quality testing will be performed on the industrial wiring harness;

[0057] Specifically, quality testing includes electrical performance testing, mechanical performance testing, and environmental adaptability testing. The electrical performance testing includes continuity testing, insulation resistance testing, voltage withstand testing, and resistance measurement. The mechanical performance testing includes tensile testing, bending testing, torsion testing, and insertion force testing. The environmental adaptability testing includes high temperature testing, low temperature testing, damp heat testing, and salt spray testing.

[0058] It should be noted that the role of quality testing is to ensure the stability of current transmission and the safety of electricity use through electrical performance testing, to ensure that the wiring harness can withstand various external forces through mechanical performance testing, and to verify its reliability in different extreme environments through environmental adaptability testing. This will comprehensively test the quality of the wiring harness, ensure the stable operation of industrial equipment, reduce the risk of failure, and improve production safety and product quality.

[0059] The time required for any test in the quality test is marked as a quality test period, and the quality test periods required to complete all tests in the quality test are grouped together as a quality test cycle. Several detection moments are evenly selected in the quality test period, and the intervals between adjacent detection moments are all the same length;

[0060] Set up several high-resolution industrial cameras with fixed camera positions and different shooting angles. The plane formed by the cameras is parallel to the plane of the industrial harness motion trajectory to ensure that all parts of the industrial harness can be clearly imaged. Adjust the camera parameters to make them consistent to ensure uniform image quality.

[0061] At the detection moment, the industrial wiring harness is photographed by the camera, and the edge detection is performed on the captured image. After the image is grayscale smoothed, the Sobel operator is used to calculate the gradient components of each pixel in the processed image in the horizontal and vertical directions. and , according to the calculated gradient component and Calculate the gradient magnitude G and gradient direction of the pixel point, compare the gradient magnitude of the pixel point based on the gradient direction, and filter out the true edge points of the image;

[0062] Based on the identified real edge points, the outline of the industrial wire harness in the image is obtained. The image within the outline of the industrial wire harness in the image is intercepted to obtain the corresponding industrial wire harness image. The industrial wire harness image obtained by shooting and edge detection of the same industrial wire harness with a single camera within a quality test period to the current inspection time is obtained. The obtained industrial wire harness image is adjusted to the same image size and resolution, and is aggregated into a time-series quality inspection image sequence according to the time sequence;

[0063] The optical flow method is used to analyze the time-series quality inspection image sequence and analyze the deformation of industrial wiring harnesses during the test process;

[0064] Specifically, set the initial optical flow field. The optical flow field consists of two optical flow velocity components, u and v, which represent the movement speed of the pixel in the x and y directions respectively. Initialize u and v of the optical flow field to zero matrices, and the size of the matrix is ​​the same as the size of the image.

[0065] It should be noted that the reason for initializing the optical flow field is that before the iteration begins, there is no prior information about the motion speed of the pixel points, so the initial motion speed of all pixels is set to zero;

[0066] Construct an energy function and solve the optical flow field by minimizing an energy function containing data terms and smooth terms through the Horn-Schunck algorithm. The expression of the energy function is:

[0067]

[0068] in, is a data item, is the temporal gradient, which is obtained by performing the difference processing on the gradient amplitude G of the pixels at the same position in the adjacent industrial wire harness images in the time series quality inspection image sequence. It reflects the motion information of the pixels in the industrial wire harness image at two adjacent inspection moments;

[0069] in, is a smooth term, is the preset smooth parameter used to control the weight of the smooth term. and Represent the gradients of u and v in time:

[0070]

[0071]

[0072] here and Represent the partial derivatives of u in the x-direction and y-direction respectively. Similarly, and represent the partial derivatives of v in the x and y directions respectively, and Reflects the rate of change of the optical flow field in space;

[0073] It should be noted that the data term is based on the assumption of constant brightness, that is, it is assumed that in adjacent industrial wire harness images, if there is no motion, the grayscale value of the same pixel remains unchanged. If there is motion, the change in image grayscale is related to the motion speed u and v of the pixel. By minimizing the data term, the calculated optical flow field can meet the constant brightness assumption as much as possible. The role of introducing the smooth term is to make the optical flow field smoother and avoid local drastic changes. It is required that the optical flow speed of adjacent pixels changes as little as possible, so that the optical flow field has a certain degree of continuity.

[0074] By using the variational method to minimize the energy function E(u, v), we can obtain the iterative update formula:

[0075]

[0076]

[0077] in, and Respectively represent the values ​​of the two optical flow velocity components u and v at the nth iteration, and They are and The local average value of and The optical flow field can be further smoothed by averaging them in their neighborhoods.

[0078] Starting from the initialized optical flow field, the iterative update formula is continuously used for calculation until the difference between the optical flow fields of two adjacent iterations is less than the preset optical flow difference threshold. The convergence condition is determined to be met, and the iterative calculation is stopped to obtain the actual optical flow velocity component of each pixel point. Based on the actual optical flow velocity component, the actual motion speed of the pixel point at the current detection moment is calculated;

[0079] If the actual motion speed of the pixel is greater than the motion speed threshold, the pixel is judged to be in motion and marked as a moving pixel;

[0080] Analyze the moving pixels. With each moving pixel as the center, check whether there are any moving pixels within its 5×5 pixel neighborhood. If there are moving pixels in the neighborhood, all moving pixels in the neighborhood are marked as damage area points and classified into the same damage area set. If there are no moving pixels in the neighborhood, the moving pixel is considered to be a misjudgment and the pixel mark is removed.

[0081] Normalize the actual motion speed of all moving pixels on the industrial wiring harness image at the current detection moment. Based on any set of damaged areas, take the average of the normalized actual motion speeds within the damaged area set to obtain the normalized degradation speed of the damaged area. Sum all normalized degradation speeds on the industrial wiring harness image to obtain the superimposed degradation speed of the industrial wiring harness at the current moment.

[0082] Obtain the superimposed degradation rate of the industrial wiring harness at all detection moments within the quality test period up to the current detection moment, and convert it into a time series degradation rate sequence according to the time series set;

[0083] Obtain industrial wire harness images from a time-series quality inspection image sequence, input them into a trained convolutional neural network, identify the damaged area in each industrial wire harness image, and calculate the image damage value of each industrial wire harness image. The image damage sequence is then generated based on the time series set.

[0084] Analyze the time-series quality inspection image sequences captured by each camera, calculate several time-series degradation rate sequences and time-series image damage sequences, and label the corresponding sequence numbers of the time-series degradation rate sequences and time-series image damage sequences according to the camera sequence number. The time-series degradation rate sequences and time-series image damage sequences obtained by the same camera have the same sequence numbers.

[0085] It should be noted that this step is to conduct comprehensive quality tests on wiring harnesses without serious damage, including electrical, mechanical, and environmental adaptability. During the test, the camera is used to continuously capture images, and a time-series image sequence is constructed through edge detection. The optical flow method is used to analyze the wiring harness deformation, obtain the time-series degradation speed sequence and image damage sequence, and realize dynamic monitoring and quantitative analysis of the wiring harness state changes during the test.

[0086] S3: Arrange the time series degradation rate sequence and time series image damage sequence, build and train the gated recurrent unit model to predict the superimposed degradation rate and image damage value of the industrial wiring harness within the prediction period, and determine whether to stop the quality test based on the prediction results. After stopping the quality test, determine whether to trigger the comprehensive quality assessment of the industrial wiring harness;

[0087] like Figure 3 As shown, the specific steps of determining whether to trigger the comprehensive evaluation of industrial wiring harness quality are as follows;

[0088] At the current detection moment, the time series degradation rate sequence and time series image damage sequence are obtained, and a gated recurrent unit (GRU) model is constructed and trained to predict the degradation rate and damage condition of industrial wiring harnesses.

[0089] Specifically, the temporal degradation rate sequences and temporal image damage sequences are sorted, and the temporal degradation rate sequences and temporal image damage sequences with the same sequence number are combined into a visual aid dataset. The combined visual aid dataset is normalized so that the temporal degradation rate sequences and temporal image damage sequences in the visual aid dataset have the same scale. The normalized data is divided into a training set, a validation set, and a test set.

[0090] It should be noted that the training set is used to train the GRU model to learn the data patterns and regularities in the visual aid dataset. The validation set is used to evaluate the performance of the GRU model during training and adjust the hyperparameters of the GRU model to prevent overfitting. The test set is used to finally evaluate the prediction performance of the trained GRU model.

[0091] A gated recurrent unit model was constructed, consisting of an input layer, a GRU layer, and an output layer. The visual aid dataset contains two features: superposition degradation rate and image damage value. Therefore, the number of neurons in the input layer was set to 2. Similarly, the number of neurons in the output layer was set to 2. Based on the principle of balancing the expressiveness and complexity of the model, the number of GRU layers and the number of neurons in each GRU layer were determined and set through experiments and parameter adjustment. Hyperparameters including the learning rate, number of training rounds, and batch size were also set.

[0092] Based on any visual aid dataset, the training set, validation set and test set divided by the visual aid dataset are input into the gated recurrent unit model for training, and initial values ​​are given to the weights and biases of the gated recurrent unit model. The training set is input into the gated recurrent unit model in sequence. The gated recurrent unit model calculates the input data according to the current parameters, passes it from the input layer through the GRU layer to the output layer, and obtains the prediction result. The mean square error is used as the loss function to measure the loss value between the model prediction result and the true value. Based on the calculated loss value, the gradient of the loss value to each parameter of the gated recurrent unit model is calculated through the back propagation algorithm. The gradient represents the rate of change of the loss function in the parameter space. The gated recurrent unit model adjusts the parameters according to the direction and size of the gradient to reduce the loss value.

[0093] Use the stochastic gradient descent algorithm as the optimization algorithm to update the model parameters according to the calculated gradient. The optimization algorithm determines the amplitude of the parameter update based on the learning rate and gradient, and iterates continuously until the loss value of the gated recurrent unit model no longer decreases significantly or reaches the preset number of training rounds;

[0094] During the training process, the validation set is used to evaluate the model, and the loss value of the gated recurrent unit model on the validation set is calculated. If the loss value on the validation set increases while the loss value on the training set decreases, it indicates that the model may be overfitting. Regularization methods are used to address the overfitting problem, and the model's hyperparameters are adjusted based on the evaluation results of the validation set. After the gated recurrent unit model training and hyperparameter adjustment are completed, the test set is used to conduct a final evaluation of the model to determine whether the model's predictive ability and generalization performance in actual applications meet the standards. The training of the gated recurrent unit model is then completed.

[0095] The trained gated recurrent unit model is used to predict the stacking degradation rate and image damage value within the prediction period and perform denormalization processing to obtain the predicted stacking degradation rate and predicted image loss value at each detection moment within the prediction period.

[0096] The prediction period starts at the current detection time. The starting point of the prediction period changes with time. The length of the prediction period is determined based on the test requirements corresponding to the current quality test period. Within the same quality test period, the length of the prediction period remains unchanged.

[0097] Based on each visual aid dataset, the gated recurrent unit model is trained and predicted to obtain the predicted superimposed degradation rate and predicted image loss value of the industrial wiring harness under different viewing angles at each detection moment within the prediction period.

[0098] The stacking degradation rate and image loss value at the current detection moment are regarded as the predicted stacking degradation rate and predicted image loss value, and the predicted stacking degradation rate and predicted image loss value within the prediction period are analyzed;

[0099] If any predicted superposition degradation speed is greater than the degradation speed threshold within the prediction period, or any predicted image loss value is greater than the preset damage threshold, it is determined that the industrial wiring harness is over-damaged or degrades too quickly within the prediction period, and the quality test of the industrial wiring harness is stopped, ending the quality test cycle.

[0100] After the quality test cycle ends, obtain the duration of the quality test cycle;

[0101] If the duration of the quality test cycle is less than the standard duration of the quality test cycle, the industrial wiring harness is judged to be unqualified;

[0102] If the duration of the quality test cycle is greater than or equal to the standard duration of the quality test cycle, the comprehensive quality assessment of the industrial wiring harness is triggered;

[0103] It should be noted that the purpose of this step is to build a visually assisted dataset based on the collected time series data, train the gated recurrent unit model to predict the future degradation rate and damage value of the wiring harness, and determine whether there is a risk of over-damage or rapid degradation based on the prediction results. The test is terminated in time to avoid resource waste. At the same time, the subsequent comprehensive quality assessment is triggered based on the test cycle length, realizing intelligent control of the inspection process.

[0104] S4: If the comprehensive quality assessment of industrial wiring harnesses is triggered, the quality assessment score of the industrial wiring harnesses is calculated based on the visual damage indicators and physical and chemical test indicators within the comprehensive quality test cycle, and the quality of the industrial wiring harnesses is graded according to the quality assessment score;

[0105] If the comprehensive quality assessment of industrial wiring harnesses is triggered, the test results of the electrical performance test, mechanical performance test, and environmental adaptability test of the industrial wiring harnesses during the quality test cycle are obtained;

[0106] Divide each test in the quality test into numerical tests and Boolean tests according to the test requirements;

[0107] It should be noted that numerical tests require recording specific values ​​and comparing them with standard ranges, such as insulation resistance and resistance values, while Boolean tests only require determining whether the test has passed.

[0108] Based on numerical tests, a linear normalization formula is used to calculate the test parameter and the corresponding industry standard threshold value for ratio processing. If the ratio is greater than 1, the result is set to 1; otherwise, the ratio is retained and recorded;

[0109] Normalize the test parameters of the numerical test at each detection moment within the corresponding quality test period, and aggregate the normalized data into a time series physical and chemical parameter sequence according to the time series;

[0110] All numerical test results are normalized to obtain several time series of physical and chemical parameters;

[0111] The time-series degradation rate sequence and time-series image damage sequence of industrial wiring harnesses in each quality test period are normalized and marked as visual damage indicators, and the time-series physical and chemical parameter sequences are marked as physical and chemical test indicators;

[0112] Based on Boolean testing, the test results are directly mapped to 0 or 1, where 0 indicates that the test fails and 1 indicates that the test passes;

[0113] If there are test results mapped to 0 in all Boolean tests, the industrial wiring harness is judged to be unqualified;

[0114] If the test result mappings of all Boolean tests are 1, the Boolean test results are judged to be qualified, and the visual damage index and physical and chemical test indexes are jointly analyzed to evaluate the quality level of the industrial wiring harness;

[0115] Specifically, the visual impairment index and the physical and chemical test index are matched one by one in the time dimension to construct a three-dimensional tensor of visual impairment index. Three-dimensional tensor of physical and chemical test indicators , where T represents the total number of detection moments in the quality test cycle, Indicates setting the number of cameras. Represents the dimension of visual damage indicators, including superimposed degradation speed and image damage value, Indicates the number of test parameters, Indicates the dimension of physical and chemical test indicators;

[0116] The interaction weights of visual impairment indicators and physical and chemical test indicators are calculated through the cross-modal attention mechanism. The formula is:

[0117]

[0118] in, , Represent the query key vectors of visual impairment index and physical and chemical test index respectively, and is a learnable weight matrix used to map visual impairment indicators and physical and chemical test indicators to the same dimension , Scaling factor to avoid Softmax gradient vanishing and ensure numerical stability;

[0119] The quality assessment score of industrial wiring harnesses is calculated by weighted fusion of visual damage indicators and physical and chemical test indicators based on the calculated interactive weights;

[0120] Compare the obtained quality assessment scores with grade thresholds, including qualified thresholds, good thresholds, and excellent thresholds;

[0121] If the quality assessment score is less than the qualified threshold, the industrial wiring harness is judged to be unqualified;

[0122] If the quality assessment score is greater than or equal to the qualified threshold and less than the good threshold, the industrial wiring harness is judged to be qualified;

[0123] If the quality assessment score is greater than or equal to the good threshold and less than the excellent threshold, the industrial wiring harness is judged to be of good quality;

[0124] If the quality assessment score is greater than or equal to the excellent threshold, the industrial wiring harness is judged to be of excellent quality;

[0125] It should be noted that this step integrates visual damage indicators and physical and chemical test indicators within the quality test cycle, and screens effective data features through data classification, normalization, and correlation analysis. A cross-modal attention mechanism is used to fuse strongly correlated data, calculate quality assessment scores, and classify wire harness quality levels based on level thresholds, providing a comprehensive and accurate comprehensive evaluation conclusion for industrial wire harness quality.

[0126] The technical solution of the embodiment of the present invention is as follows: multiple industrial cameras are set to shoot industrial wire harnesses, a preliminary screening image group is obtained, edge detection and convolutional neural network are used to identify the damaged area, image damage value is calculated, and whether the industrial wire harness is seriously damaged is judged according to the image damage value. If the industrial wire harness is not damaged or the damage is slightly damaged, the industrial wire harness is subjected to quality testing. During the test, a camera is set to shoot the industrial wire harness image, edge detection and contour extraction are performed, a time-series quality inspection image sequence is constructed, the image sequence is analyzed by the optical flow method, the pixel motion speed is calculated, the damaged area and the superimposed degradation speed are determined, and the time-series degradation speed is obtained. The time series degradation rate sequence and time series image damage sequence are sorted, and a gated cyclic unit model is constructed and trained to predict the superimposed degradation rate and image damage value of the industrial wiring harness within the prediction period. The prediction results are used to determine whether to stop the quality test. After stopping the quality test, if the test cycle length meets the requirements, the comprehensive quality evaluation of the industrial wiring harness is triggered. If the comprehensive quality evaluation of the industrial wiring harness is triggered, the visual damage indicators and physical and chemical test indicators within the comprehensive quality test cycle are used to calculate the quality evaluation score of the industrial wiring harness. The quality of the industrial wiring harness is divided into evaluation levels according to the quality evaluation score.

[0127] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A visually assisted industrial wiring harness quality inspection method, characterized by: include: Obtain a preliminary screening image set of industrial wiring harnesses and preliminarily screen out those with severe damage; Start the quality test and collect industrial wiring harness images in real time to construct a time-series quality inspection image sequence. Combined with the optical flow method, the superimposed degradation rate of the industrial wiring harness is calculated in real time. The time-series degradation rate sequence and time-series image damage sequence are established. The time-series prediction model is trained to predict the degradation trend of the industrial wiring harness. The quality test is dynamically terminated. The duration of the quality test cycle is used to determine whether to trigger a comprehensive assessment of the industrial wiring harness quality. The method for establishing the timing degradation speed sequence is as follows: Obtain all moving pixels on the industrial wiring harness image at the current detection moment, normalize and calculate the actual movement speed of the moving pixels, and obtain the superimposed degradation speed of the industrial wiring harness at the current moment; Obtain all superimposed degradation rates from the quality test period to the current detection moment, and convert them into a time series degradation rate sequence according to the time series set; The method for obtaining the moving pixel points is as follows: If the industrial wiring harness is not damaged or is slightly damaged, perform a quality test on the industrial wiring harness, and collect industrial wiring harness images in real time during the quality test to construct a time-series quality inspection image sequence; The optical flow method is used to analyze the time-series quality inspection image sequence and calculate the actual motion speed of each pixel at the current detection moment. If the actual motion speed is greater than the motion speed threshold, it is marked as a moving pixel. Taking each moving pixel as the center, check whether there is a moving pixel in its 5×5 pixel neighborhood. If there is, all the moving pixels in the neighborhood are grouped into the same damage area. If not, clear the pixel mark. The time series image damage sequence is constructed as follows: Obtain industrial wire harness images from a time-series quality inspection image sequence, input them into a trained convolutional neural network, identify the damaged area in each industrial wire harness image, and calculate the image damage value of each industrial wire harness image. The image damage sequence is then generated based on the time series set. If triggered, the system will integrate the timing degradation speed sequence, timing image damage sequence and quality test data, and use multi-index weighted evaluation to calculate the quality assessment score of the industrial wiring harness, thus achieving the quantitative classification of the quality level of the industrial wiring harness; The quality assessment score is obtained as follows: If the comprehensive quality assessment of industrial wiring harnesses is triggered, each test in the quality test is divided into numerical tests and Boolean tests according to the test requirements; Based on numerical tests, a linear normalization formula is used to calculate the test parameter and the corresponding industry standard threshold value for ratio processing. If the ratio is greater than 1, the result is set to 1; otherwise, the ratio is retained and recorded; For all numerical tests, the recorded test parameter results are normalized separately and integrated according to the time sequence to obtain the time series physical and chemical parameter sequence corresponding to the numerical test; The time-series degradation speed sequence and time-series image damage sequence of industrial wiring harnesses are normalized and marked as visual damage indicators, and the time-series physical and chemical parameter sequences are marked as physical and chemical test indicators; Based on Boolean tests, the test results are directly mapped to 0 or 1; If there is a test result mapped to 0 in all Boolean tests, the industrial wiring harness quality is judged to be unqualified, otherwise, the Boolean test result is judged to be qualified; If the Boolean test result is qualified, the visual damage index and the physical and chemical test index are jointly analyzed, and the interaction weight of the visual damage index and the physical and chemical test index is calculated through the cross-modal attention mechanism. The visual damage index and the physical and chemical test index are weighted and fused based on the interaction weight to calculate the quality assessment score of the industrial wiring harness.

2. The visually assisted industrial harness quality inspection method according to claim 1, characterized in that: The method for initially screening out industrial wiring harnesses with severe damage is as follows: Obtain a preliminary screening image group of industrial wiring harnesses, perform industrial wiring harness damage identification on the images in the preliminary screening image group through edge detection and convolutional neural network, calculate the weight of each damaged area in the industrial wiring harness image based on the damage category and recognition confidence, and perform a weighted calculation based on the weight and the damage degree of the damaged area to obtain the image damage value of the industrial wiring harness image; If the image damage value of any industrial wiring harness image in the industrial wiring harness image group is greater than a preset damage threshold, it is determined that the industrial wiring harness is severely damaged and the quality of the industrial wiring harness is unqualified.

3. The visually assisted industrial harness quality inspection method according to claim 1, characterized in that: The method of dynamically ending the quality test is: Obtain the predicted stacking degradation rate and predicted image loss value within the prediction period for analysis; If any predicted superposition degradation speed within the prediction period is greater than the degradation speed threshold, or any predicted image loss value is greater than the preset damage threshold, the quality test of the industrial wiring harness is stopped and the quality test cycle ends.

4. The visually assisted industrial harness quality inspection method according to claim 3, characterized in that: The method for obtaining the predicted stacking degradation speed and the predicted image loss value is as follows: The time-series degradation rate sequence and time-series image damage sequence are obtained, and a gated recurrent unit model is constructed and trained to predict the degradation rate and damage condition of the industrial wiring harness. The predicted superimposed degradation rate and predicted image loss value of the industrial wiring harness at different perspectives at each detection moment within the prediction period are obtained.

5. The visually assisted industrial harness quality inspection method according to claim 1, characterized in that: The method for determining whether to trigger the comprehensive evaluation of industrial wiring harness quality is as follows: Get the duration of the quality test cycle. If the duration of the quality test cycle is greater than or equal to the standard duration of the quality test cycle, trigger the comprehensive quality assessment of the industrial wiring harness.

Images