A method, system, device and storage medium for detecting the quality of vehicle windshield wipers

Abstract

The present invention discloses a method, system, device and storage medium for detecting the quality of vehicle windshield wipers. The method includes: sampling to obtain a positive sample set and a negative sample set; respectively inputting the positive sample set and the negative sample set into a pre-constructed first convolutional neural network and a second convolutional neural network to train and obtain a first water column recognition model and a second water column recognition model; obtaining the water column image time series data of a third vehicle windshield wiper to be detected, and respectively inputting the water column image time series data into the first water column recognition model and the second water column recognition model for object detection to obtain the water column morphology time series data; comparing the water column morphology time series data with the positive sample time series data and the negative sample time series data, and determining the quality detection result of the third vehicle windshield wiper according to the comparison result. The present invention improves the accuracy of vehicle windshield wiper quality detection and can be widely applied to the field of computer vision technology.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a method, system, device and medium for detecting the quality of vehicle windshield wipers. Background Art

[0002] The quality inspection of windshield wipers is an important and indispensable link in the automobile production process. The method adopted by traditional technology is to first locate the position of the windshield wipers, obtain one picture of the water column of each of the left and right windshield wipers, input these two pictures into the resnet18 classification model, obtain the classification results of these two pictures, and determine whether the water columns are present or absent at the same time according to the classification results, so as to judge whether the vehicle windshield wipers meet the inspection standards. This processing method can generally ensure the accuracy of the inspection, but it is not the case in some special inspection environments. For example, when encountering accidental reflection of the water column during inspection, it is easy to cause obvious bright spots to appear locally on the water column, resulting in incorrect inspection results and reducing the accuracy of the vehicle windshield wiper quality inspection. Therefore, it is urgent to develop a vehicle windshield wiper quality inspection method with higher accuracy to solve the current problems. Summary of the Invention

[0003] The purpose of the present invention is to solve at least to some extent one of the technical problems existing in the prior art.

[0004] To this end, an object of an embodiment of the present invention is to provide a method for detecting the quality of vehicle windshield wipers, which improves the accuracy of the vehicle windshield wiper quality inspection.

[0005] Another object of an embodiment of the present invention is to provide a system for detecting the quality of vehicle windshield wipers.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0007] In a first aspect, an embodiment of the present invention provides a method for detecting the quality of vehicle windshield wipers, including the following steps:

[0008] Determine a plurality of first vehicle windshield wipers with good quality and a plurality of second vehicle windshield wipers with quality defects, sample the water column images of the first vehicle windshield wipers to obtain a positive sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set;

[0009] Input the positive sample set into a pre-constructed first convolutional neural network to train and obtain a first water column recognition model, and input the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model;

[0010] Obtain the time series data of the water column images of the third vehicle windshield wiper to be detected, and input the time series data of the water column images into the first water column recognition model and the second water column recognition model respectively for target detection to obtain the time series data of the water column morphology;

[0011] Determine the positive sample time series data of the water column morphology corresponding to the positive sample set and the negative sample time series data of the water column morphology corresponding to the negative sample set, compare the time series data of the water column morphology with the positive sample time series data and the negative sample time series data, and determine the quality detection result of the third vehicle windshield wiper according to the comparison result.

[0012] Further, in an embodiment of the present invention, the step of sampling the water column images of the first vehicle windshield wiper to obtain a positive sample set and sampling the water column images of the second vehicle windshield wiper to obtain a negative sample set specifically includes:

[0013] Continuously sample the water column images of the first vehicle windshield wiper at a preset sampling frequency to obtain a plurality of positive sample image sequences, and determine the first water column region label of the positive sample image sequences to obtain the positive sample set;

[0014] Continuously sample the water column images of the second vehicle windshield wiper at a preset sampling frequency to obtain a plurality of negative sample image sequences, and determine the second water column region label and the defect type label of the negative sample image sequences to obtain the negative sample set.

[0015] Further, in an embodiment of the present invention, the step of inputting the positive sample set into a pre-constructed first convolutional neural network to train and obtain a first water column recognition model specifically includes:

[0016] Input the positive sample set into a pre-constructed first convolutional neural network. The first convolutional neural network performs upsampling and feature fusion on the underlying features through FPN to obtain high-resolution water column feature information, and outputs a first water column region prediction result;

[0017] Determine the first loss value of the first convolutional neural network according to the first water column region prediction result and the first water column region label;

[0018] Update the parameters of the first convolutional neural network according to the first loss value through the backpropagation algorithm;

[0019] When the first loss value reaches a preset first threshold or the model accuracy reaches a preset second threshold, stop training to obtain a trained first water column recognition model.

[0020] Further, in an embodiment of the present invention, the step of inputting the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model specifically includes:

[0021] Input the negative sample set into a pre-constructed second convolutional neural network. The second convolutional neural network performs upsampling and feature fusion on the underlying features through FPN to obtain high-resolution water column feature information, and then performs reinforcement learning through a multi-scale dilated convolution module to obtain a water column feature map, and further outputs a second water column region prediction result;

[0022] Determine a second loss value of the second convolutional neural network according to the second water column region prediction result and the second water column region label;

[0023] Update the parameters of the second convolutional neural network according to the second loss value through the backpropagation algorithm;

[0024] When the second loss value reaches a preset third threshold or the model accuracy reaches a preset fourth threshold, stop training to obtain a trained second water column recognition model.

[0025] Further, in an embodiment of the present invention, the step of obtaining the water column image time series data of the third vehicle windshield to be detected, inputting the water column image time series data into the first water column recognition model and the second water column recognition model respectively for target detection, and obtaining the water column morphology time series data specifically includes:

[0026] Continuously sample the water column images of the third vehicle windshield at a preset sampling frequency to obtain water column image time series data, and the water column image time series data includes multiple frames of water column images of the third vehicle windshield arranged in the sampling order;

[0027] Input the water column image time series data into the first water column recognition model, determine the first water column morphology data of several frames of water column images according to the recognition result, input the water column image time series data into the second water column recognition model, and determine the second water column morphology data of several frames of water column images according to the recognition result;

[0028] Generate water column morphology time series data according to the sampling order of the water column images corresponding to the first water column morphology data and the second water column morphology data.

[0029] Further, in an embodiment of the present invention, the step of determining the positive sample time series data of the water column morphology corresponding to the positive sample set and the negative sample time series data of the water column morphology corresponding to the negative sample set specifically includes:

[0030] Input the forward sample image sequence into the first water column recognition model, and determine the forward sample time series data of the water column form corresponding to the forward sample set according to the recognition result;

[0031] Input the negative sample image sequence into the second water column recognition model, determine the negative sample time series data of the water column form corresponding to the negative sample set according to the recognition result, and label the negative sample time series data according to the defect type label.

[0032] Further, in an embodiment of the present invention, the step of comparing the water column form time series data with the forward sample time series data and the negative sample time series data, and determining the quality detection result of the third vehicle windshield wiper according to the comparison result specifically includes:

[0033] Calculate the Euclidean distances between the water column form time series data and each of the forward sample time series data and each of the negative sample time series data through the KNN algorithm, and further calculate the similarities between the water column form time series data and each of the forward sample time series data and each of the negative sample time series data;

[0034] When the average value of the similarities between the water column form time series data and each of the forward sample time series data is greater than or equal to a preset fifth threshold, it is determined that the quality of the third vehicle windshield wiper is good;

[0035] When the average value of the similarities between the water column form time series data and each of the forward sample time series data is less than the preset fifth threshold, determine the defect type label corresponding to the negative sample time series data with the greatest similarity to the water column form time series data, and further determine the defect type of the third vehicle windshield wiper.

[0036] In a second aspect, an embodiment of the present invention provides a vehicle windshield wiper quality detection system, including:

[0037] A sample acquisition module, configured to determine a plurality of first vehicle windshield wipers with good quality and a plurality of second vehicle windshield wipers with quality defects, sample the water column images of the first vehicle windshield wipers to obtain a forward sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set;

[0038] A water column recognition model training module, configured to input the forward sample set into a pre-constructed first convolutional neural network to train a first water column recognition model, and input the negative sample set into a pre-constructed second convolutional neural network to train a second water column recognition model;

[0039] A water column shape detection module, configured to obtain the water column image time series data of the third vehicle windshield wiper to be detected, input the water column image time series data into the first water column recognition model and the second water column recognition model respectively for target detection, and obtain the water column shape time series data;

[0040] A quality detection result determination module, configured to determine the positive sample time series data of the water column shape corresponding to the positive sample set and the negative sample time series data of the water column shape corresponding to the negative sample set, compare the water column shape time series data with the positive sample time series data and the negative sample time series data, and determine the quality detection result of the third vehicle windshield wiper according to the comparison result.

[0041] In a third aspect, an embodiment of the present invention provides a vehicle windshield wiper quality detection device, including:

[0042] At least one processor;

[0043] At least one memory, configured to store at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned vehicle windshield wiper quality detection method.

[0045] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the above-mentioned vehicle windshield wiper quality detection method when executed by the processor.

[0046] The advantages and beneficial effects of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present invention:

[0047] In the embodiment of the present invention, the first water column recognition model and the second water column recognition model for detecting the windshield wiper water column are respectively trained by the positive sample set and the negative sample set, and then the water column image time series data of the vehicle windshield wiper to be detected are respectively input into the first water column recognition model and the second water column recognition model for target detection to obtain the water column shape time series data. Finally, the water column shape time series data is compared with the positive sample time series data and the negative sample time series data, so as to obtain the quality detection result of the vehicle windshield wiper. In the embodiment of the present invention, the first water column recognition model and the second water column recognition model are respectively trained by the positive sample set and the negative sample set, which can accurately identify the water column characteristics in different situations, so as to realize the target detection of the windshield wiper water column; by forming the water column shape time series data and comparing it with the positive and negative sample time series data, the time series change law of the windshield wiper water column during detection is utilized, avoiding the contingency of using the water column image at a single time point as the judgment basis, strengthening the dependence on the time series characteristics of the windshield wiper water column, and improving the accuracy of vehicle windshield wiper quality detection. Brief Description of the Drawings

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following provides an introduction to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings in the following introduction are only for conveniently and clearly presenting some embodiments of the technical solutions in the present invention. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 It is a flowchart of the steps of a method for detecting the quality of vehicle windshield wipers provided by an embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of a multi-scale dilated convolution module provided by an embodiment of the present invention;

[0051] Figure 3 It is a block diagram of the structure of a system for detecting the quality of vehicle windshield wipers provided by an embodiment of the present invention;

[0052] Figure 4 It is a block diagram of the structure of a device for detecting the quality of vehicle windshield wipers provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0053] The following details the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0054] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of "first" and "second", it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention.

[0055] Referring to Figure 1 , an embodiment of the present invention provides a method for detecting the quality of vehicle windshield wipers, which specifically includes the following steps:

[0056] S101. Determine multiple first vehicle windshield wipers with good quality and multiple second vehicle windshield wipers with quality defects. Sample the water column images of the first vehicle windshield wipers to obtain a positive sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set.

[0057] Specifically, install a high-definition video acquisition camera in the vehicle quality inspection area, collect video image data of the first vehicle windshield wipers with good quality and the second vehicle windshield wipers with quality defects, and form a positive sample set and a negative sample set respectively according to the collected video image data.

[0058] Further, as an optional implementation manner, the step of sampling the water column images of the first vehicle windshield wipers to obtain a positive sample set and sampling the water column images of the second vehicle windshield wipers to obtain a negative sample set specifically includes:

[0059] S1021. Continuously sample the water column images of the first vehicle windshield wipers at a preset sampling frequency to obtain multiple positive sample image sequences, and determine the first water column area label of the positive sample image sequences to obtain a positive sample set;

[0060] S1022. Continuously sample the water column images of the second vehicle windshield wipers at a preset sampling frequency to obtain multiple negative sample image sequences, and determine the second water column area label and defect type label of the negative sample image sequences to obtain a negative sample set.

[0061] Specifically, for the vehicles with windshield wipers that have passed manual inspection and have good quality, reset them back to the detection production line, and collect image data of them at a preset sampling frequency (such as once every 3 seconds). Repeat this process multiple times with different vehicles until a certain number of positive sample image sequences are obtained. Perform corresponding annotation on the obtained positive sample images containing vehicle windshield wiper water column information, so that the framed area can represent the position of the windshield wiper water column in the image.

[0062] Similarly, for the vehicles with quality defects after manual inspection and classification, reset them back to the detection production line, and collect image data of them at a preset sampling frequency (such as once every 3 seconds). Repeat this process multiple times with different vehicles until a certain number of negative sample image sequences are obtained. Perform corresponding annotation on the obtained negative sample images containing vehicle windshield wiper water column information, so that the framed area can represent the position of the windshield wiper water column in the image, and at the same time, the defect type needs to be annotated.

[0063] S102. Input the positive sample set into a pre-constructed first convolutional neural network to train and obtain a first water column recognition model, and input the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model.

[0064] Specifically, in the embodiments of the present invention, the YOLO convolutional neural network is used to train the positive sample set and the negative sample set. Since the annotation boxes of the water column regions in the negative sample set are more irregular than those in the positive sample set, the training processes of the two are also different, which will be described separately below.

[0065] Further as an optional implementation manner, the step of inputting the positive sample set into a pre-constructed first convolutional neural network and training to obtain a first water column recognition model specifically includes:

[0066] S1021: Input the positive sample set into a pre-constructed first convolutional neural network. The first convolutional neural network performs upsampling and feature fusion on the underlying features through FPN to obtain high-resolution water column feature information, and outputs a first water column region prediction result;

[0067] S1022: Determine the first loss value of the first convolutional neural network according to the first water column region prediction result and the first water column region label;

[0068] S1023: Update the parameters of the first convolutional neural network according to the first loss value through the backpropagation algorithm;

[0069] S1024: When the first loss value reaches a preset first threshold or the model accuracy reaches a preset second threshold, stop training to obtain a trained first water column recognition model.

[0070] Specifically, the labeled positive sample set is input into the backbone network of Ghost-Bottleneck-CSP-Darknet53, and simplified convolutions are used to extract features from the wiper water column images. In the model, the sampled BiFPN results are used as the Neck part to improve the recognition of water column features.

[0071] At the same time, new anchor boxes are added in the wiper water column prediction layer, and 4-scale detection heads are sampled to achieve multi-scale detection, enhancing the feature extraction ability of the water column.

[0072] In addition, to alleviate the problem of high repetition rate of the feature maps obtained after convolution, the convolution process is simplified through the GhostNet structure to achieve model compression. The 3*3 convolutional linear operation adopted by the Ghost module is used to obtain a larger number of effective features. The Ghost modules are stacked and the feature information is fused and passed into the residual edge to replace the bottle-neck in yolo5, realizing the reduction of the computational amount and the compression of the model.

[0073] To improve the feature fusion effect, the embodiments of the present invention also utilize a Feature Pyramid Network (FPN) to enhance the performance of detecting small objects such as water columns. The FPN upsamples the features of the bottom layer and fuses them with the bottom-layer features to obtain high-resolution features.

[0074] The network directly modifies the original single network. After introducing the feature map of the next resolution scaled by two times for each resolution of the feature map, an element-wise addition operation is performed. Different scales and ratios of anchors are set on the feature map, and the scale information corresponds to the corresponding feature map (with areas set to 32^2, 64^2, 128^2, 256^2, 512^2 respectively).

[0075] Specifically, for the first water column recognition model, the accuracy of the object detection result can be measured by a loss function. The loss function is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of the single training data and the prediction result of the model for this training data. During actual training, a training data set has many training data, so generally a cost function is used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction errors of all training data, which can better measure the prediction effect of the model. For a general machine learning model, based on the aforementioned cost function, plus a regularization term that measures the model complexity, it can be used as the training objective function. Based on this objective function, the loss value of the entire training data set can be obtained. There are many types of commonly used loss functions. For example, the 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross-entropy loss function, etc. can all be used as the loss function of the machine learning model, which will not be elaborated one by one here.

[0076] In the embodiments of the present invention, the GIoU Loss is used as the loss function to determine the training loss value, which can eliminate the redundant wiper annotation boxes in the prediction result processing stage and find the optimal wiper feature detection position. Based on the training loss value, the backpropagation algorithm is used to update the parameters of the model, and after iterating a certain number of times, the trained first water column recognition model can be obtained.

[0077] Specifically, for positive samples, recall, precision, and mean average precision (mAP) are used as evaluation metrics. After multiple trainings, when the recall reaches 88%, the precision reaches 91%, and the mAP reaches 90%, the training is stopped, and the first water column recognition model is obtained.

[0078] Further, as an optional implementation, the step of inputting the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model specifically includes:

[0079] S1025: Input the negative sample set into the pre-constructed second convolutional neural network. The second convolutional neural network performs upsampling and feature fusion on the underlying features through the Feature Pyramid Network (FPN) to obtain high-resolution water column feature information, then performs reinforcement learning through a multi-scale dilated convolution module to obtain a water column feature map, and further outputs a second water column region prediction result;

[0080] S1026: Determine the second loss value of the second convolutional neural network according to the second water column region prediction result and the second water column region label;

[0081] S1027: Update the parameters of the second convolutional neural network according to the second loss value through the backpropagation algorithm;

[0082] S1028: When the second loss value reaches a preset third threshold or the model accuracy reaches a preset fourth threshold, stop the training to obtain a trained second water column recognition model.

[0083] Specifically, since the annotation bounding boxes of the water column regions in the negative sample set are more irregular than those in the positive sample set, an additional reinforcement feature learning is required after the Feature Pyramid Network (FPN).

[0084] Considering that the proportion of vehicle wiper targets with quality defects in the image is not fixed, which will affect the model's extraction of context information, the multi-scale dilated convolution module is introduced in the embodiments of the present invention for reinforcement learning. As Figure 2 shown is a schematic diagram of the multi-scale dilated convolution module provided by the embodiments of the present invention. In the multi-scale dilated convolution module, max-pooling operations of 5×5, 9×9, and 13×13 are performed on the input deep features in packets. Multi-scale processing with different sizes of convolution kernels can maintain the spatial information of the features; the size of the input feature map is represented as W×H. When the size of the convolution kernel is F and the stride is S, the scale calculation formula of the pooled output feature map is:

[0085]

[0086]

[0087] Perform dilated convolution operations on the cascaded feature tensors at different rates. Compared with standard convolution, the size of the convolution kernel in dilated convolution will increase. When padding the feature map, in order to keep the size of the feature map consistent with the original image, the number of padded pixels needs to be deduced and calculated. The calculation formula is as follows:

[0088]

[0089] Among them, K0 represents the size of the original convolution kernel, d is the dilation rate, and p is the number of pixels added to each side of the feature map. Finally, use the feature tensors obtained from feature maps at different levels as the input for the final detection to complete the detection task.

[0090] In addition, recall, precision, and mean average precision (mAP) are used as evaluation indicators for negative samples. However, due to the small number of negative samples in actual applications, when the recall reaches 70%, the precision reaches 72%, and the mean average precision (mAP) reaches 65%, the training can be stopped to obtain the second water column recognition model.

[0091] S103. Obtain the time-series data of the water column images of the third vehicle windshield to be detected, and input the time-series data of the water column images into the first water column recognition model and the second water column recognition model respectively for object detection to obtain the time-series data of the water column morphology.

[0092] Specifically, for the third vehicle windshield to be detected, the time-series sampling of the water column images is also performed at the same sampling frequency as during model training. The obtained time-series data of the water column images is input into the first water column recognition model and the second water column recognition model obtained in the previous steps respectively to obtain the water column morphology at each sampling time point, thereby generating the time-series data of the water column morphology. Step S103 specifically includes the following steps:

[0093] S1031. Continuously sample the water column images of the third vehicle windshield at a preset sampling frequency to obtain the time-series data of the water column images. The time-series data of the water column images includes multiple frames of water column images of the third vehicle windshield arranged in the sampling order.

[0094] S1032. Input the time-series data of the water column images into the first water column recognition model, and determine the first water column morphology data of several frames of water column images according to the recognition results. Input the time-series data of the water column images into the second water column recognition model, and determine the second water column morphology data of several frames of water column images according to the recognition results.

[0095] S1033. Generate the time-series data of the water column morphology according to the sampling order of the water column images corresponding to the first water column morphology data and the second water column morphology data.

[0096] Specifically, when the first water column shape data and the second water column shape data overlap in sampling time (that is, the same frame of image can be detected by two water column recognition models at the same time), the one with the higher predicted probability is selected from the two and added to the water column shape time series data.

[0097] S104. Determine the positive sample time series data of the water column shape corresponding to the positive sample set and the negative sample time series data of the water column shape corresponding to the negative sample set, compare the water column shape time series data with the positive sample time series data and the negative sample time series data, and determine the quality inspection result of the third vehicle windshield wiper according to the comparison result.

[0098] Specifically, after performing object detection on the third vehicle windshield wiper, the water column shape time series data during the entire detection process is accumulated. By comparing the similarity of the time series of the water column shapes of the positive and negative samples, the periodic characteristics of the water column and the change trend of the object detection data during the quality inspection of the vehicle windshield wiper can be mined, so that the quality inspection result of the third vehicle windshield wiper can be accurately obtained.

[0099] Further as an optional implementation manner, the step of determining the positive sample time series data of the water column shape corresponding to the positive sample set and the negative sample time series data of the water column shape corresponding to the negative sample set specifically includes:

[0100] S1041. Input the positive sample image sequence into the first water column recognition model, and determine the positive sample time series data of the water column shape corresponding to the positive sample set according to the recognition result;

[0101] S1042. Input the negative sample image sequence into the second water column recognition model, determine the negative sample time series data of the water column shape corresponding to the negative sample set according to the recognition result, and label the negative sample time series data according to the defect type label.

[0102] Specifically, input the positive sample image sequence and the negative sample image sequence into the first water column recognition model and the second water column recognition model respectively to obtain the positive sample time series data and the negative sample time series data. Among them, the negative sample time series data also needs to be labeled with the defect type.

[0103] Further as an optional implementation manner, the step of comparing the water column shape time series data with the positive sample time series data and the negative sample time series data, and determining the quality inspection result of the third vehicle windshield wiper according to the comparison result specifically includes:

[0104] S1043. Calculate the Euclidean distances between the time-series data of the water column morphology and the time-series data of each positive sample and each negative sample through the KNN algorithm, and then calculate the similarity between the time-series data of the water column morphology and the time-series data of each positive sample and each negative sample;

[0105] S1044. When the mean value of the similarities between the time-series data of the water column morphology and the time-series data of each positive sample is greater than or equal to a preset fifth threshold, determine that the quality of the third vehicle's windshield wiper is good;

[0106] S1045. When the mean value of the similarities between the time-series data of the water column morphology and the time-series data of each positive sample is less than the preset fifth threshold, determine the defect type label corresponding to the time-series data of the negative sample with the highest similarity to the time-series data of the water column morphology, and then determine the defect type of the third vehicle's windshield wiper.

[0107] Specifically, since there are certain differences in each target detection and recognition of different positive and negative samples, the data groups of their image data are not unique. In the embodiments of the present invention, the dynamic time warping (DTW) algorithm is used to calculate the similarity of arrays of different target recognition images.

[0108] In the embodiments of the present invention, the similarity comparison of the water column morphology is based on the time series. After the target detection and recognition and the dynamic time series calculation of the third vehicle's windshield wiper to be detected, the time-series data of the water column morphology can be obtained, and then the Euclidean distance between the time-series data of the water column morphology and the time-series data of the positive and negative samples is calculated by using the KNN algorithm, so as to calculate the similarity.

[0109] For the positive sample set, calculate the similarity between the time-series data of the water column morphology and the time-series data of each positive sample respectively, obtain more than 50 similarity values, and then take the mean value; when the mean value is greater than or equal to the fifth threshold (such as 0.85), it means that the water column time-series characteristics of the third vehicle's windshield wiper are highly similar to the water column time-series characteristics of the positive sample set, so it can be determined that the quality of the third vehicle's windshield wiper is good; when the mean value is less than the fifth threshold, determine the time-series data of the negative sample with the highest similarity to the time-series data of the water column morphology (that is, the time-series data of the negative sample with the closest Euclidean distance), and determine the defect type of the third vehicle's windshield wiper according to the corresponding defect type label.

[0110] In some optional embodiments, after detecting the defect type of the third vehicle's windshield wiper, corresponding quality repair operations can also be performed on it, which are not elaborated in the embodiments of the present invention.

[0111] The method steps of the embodiments of the present invention are described above. It can be understood that, by training the first water column recognition model and the second water column recognition model respectively with the positive sample set and the negative sample set, the embodiments of the present invention can accurately recognize the water column characteristics in different situations, so as to realize the target detection of the windshield wiper water column; by forming the water column shape time series data and comparing it with the positive and negative sample time series data, the time series change law of the windshield wiper water column during detection is utilized, avoiding the contingency of using the water column image at a single time point as the judgment basis, strengthening the dependence on the time series characteristics of the windshield wiper water column, and improving the accuracy of the vehicle windshield wiper quality detection.

[0112] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0113] 1) The YOLO convolutional neural network of the embodiments of the present invention adopts a one-stage algorithm, with fast network operation speed and small memory occupancy. Due to its more complex backbone network structure and training strategy skills, etc., it has better detection accuracy and faster inference speed;

[0114] 2) The embodiments of the present invention have increased the mAP value by more than 15% compared with the traditional video target recognition model, greatly improving the overall performance of the system, and also greatly improving the detection accuracy of small-scale targets such as windshield wiper water columns, and can realize the accurate recognition of the detection target;

[0115] 3) The embodiments of the present invention can effectively fuse the time series of positive and negative samples by using the KNN algorithm, which can greatly improve the accuracy of the vehicle windshield wiper quality detection, and the accuracy rate is increased by more than 20% compared with the traditional quality detection that only relies on target recognition.

[0116] Referring to Figure 3 , the embodiments of the present invention provide a vehicle windshield wiper quality detection system, including:

[0117] A sample acquisition module, configured to determine a plurality of first vehicle windshield wipers with good quality and a plurality of second vehicle windshield wipers with quality defects, sample the water column images of the first vehicle windshield wipers to obtain a positive sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set;

[0118] A water column recognition model training module, configured to input the positive sample set into a pre-constructed first convolutional neural network to train a first water column recognition model, and input the negative sample set into a pre-constructed second convolutional neural network to train a second water column recognition model;

[0119] A water column shape detection module, configured to obtain the water column image time series data of a third vehicle windshield wiper to be detected, input the water column image time series data into the first water column recognition model and the second water column recognition model respectively for target detection, and obtain the water column shape time series data;

[0120] A quality inspection result determination module is configured to determine the positive sample time series data of the water column shape corresponding to the positive sample set and the negative sample time series data of the water column shape corresponding to the negative sample set, compare the water column shape time series data with the positive sample time series data and the negative sample time series data, and determine the quality inspection result of the third vehicle windshield wiper according to the comparison result.

[0121] The content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0122] Referring to Figure 4 , an embodiment of the present invention provides a vehicle windshield wiper quality inspection device, including:

[0123] At least one processor;

[0124] At least one memory for storing at least one program;

[0125] When the above at least one program is executed by the above at least one processor, the above at least one processor implements the above vehicle windshield wiper quality inspection method.

[0126] The content in the above method embodiments is applicable to the device embodiments of the present invention. The functions specifically implemented by the device embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0127] An embodiment of the present invention also provides a computer-readable storage medium, in which a processor-executable program is stored. The processor-executable program is used to execute the above vehicle windshield wiper quality inspection method when executed by a processor.

[0128] A computer-readable storage medium according to an embodiment of the present invention can execute a vehicle windshield wiper quality inspection method provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0129] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 the method shown.

[0130] In some alternative embodiments, the functions / operations recited in the block diagrams may not occur in the order presented in the operational illustrations. For example, depending on the functions / operations involved, two blocks shown in succession may actually be executed substantially simultaneously or the blocks may sometimes be executed in reverse order. Further, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are envisioned in which the order of various operations is altered and in which sub-operations described as part of a larger operation are performed independently.

[0131] Moreover, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the above-described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It should also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skill of an engineer. Thus, those of ordinary skill in the art will be able to implement the present invention as set forth in the claims without undue experimentation. It should also be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0132] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as a USB flash drive, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc.

[0133] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered a definitional sequence of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. As used in this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device.

[0134] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the above program can be printed, because the above program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then storing it in a computer memory.

[0135] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0136] In the above description of this specification, the descriptions referring to the terms "one embodiment / example", "another embodiment / example", or "certain embodiments / examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0137] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

[0138] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting the quality of vehicle windshield wipers, characterized in that, Including the following steps: Determine a plurality of first vehicle windshield wipers with good quality and a plurality of second vehicle windshield wipers with quality defects. Sample the water column images of the first vehicle windshield wipers to obtain a positive sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set; Input the positive sample set into a pre-constructed first convolutional neural network to train and obtain a first water column recognition model. Input the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model; Obtain the time series data of the water column images of the third vehicle windshield wiper to be detected. Input the time series data of the water column images into the first water column recognition model and the second water column recognition model respectively for target detection to obtain the time series data of the water column morphology; Determine the positive sample time series data of the water column morphology corresponding to the positive sample set and the negative sample time series data of the water column morphology corresponding to the negative sample set. Compare the time series data of the water column morphology with the positive sample time series data and the negative sample time series data, and determine the quality detection result of the third vehicle windshield wiper according to the comparison result.

2. The vehicle wiper quality detection method according to claim 1, characterized in that, The step of sampling the water column images of the first vehicle windshield wipers to obtain a positive sample set and sampling the water column images of the second vehicle windshield wipers to obtain a negative sample set specifically includes: Continuously sample the water column images of the first vehicle windshield wipers at a preset sampling frequency to obtain a plurality of positive sample image sequences, and determine the first water column region labels of the positive sample image sequences to obtain the positive sample set; Continuously sample the water column images of the second vehicle windshield wipers at a preset sampling frequency to obtain a plurality of negative sample image sequences, and determine the second water column region labels and defect type labels of the negative sample image sequences to obtain the negative sample set.

3. A method for detecting the quality of vehicle windshield wipers according to claim 2, characterized in that, The step of inputting the positive sample set into a pre-constructed first convolutional neural network to train and obtain a first water column recognition model specifically includes: Input the positive sample set into a pre-constructed first convolutional neural network. The first convolutional neural network performs upsampling and feature fusion on the underlying features through FPN to obtain high-resolution water column feature information, and outputs a first water column region prediction result; Determine the first loss value of the first convolutional neural network according to the first water column region prediction result and the first water column region label; Update the parameters of the first convolutional neural network according to the first loss value through the backpropagation algorithm; When the first loss value reaches a preset first threshold or the model accuracy reaches a preset second threshold, stop training to obtain a trained first water column recognition model.

4. A method for detecting the quality of vehicle windshield wipers according to claim 2, characterized in that, The step of inputting the negative sample set into a pre-constructed second convolutional neural network to train and obtain a second water column recognition model specifically includes: Input the negative sample set into a pre-constructed second convolutional neural network. The second convolutional neural network performs upsampling and feature fusion on the underlying features through FPN to obtain high-resolution water column feature information, and then performs reinforcement learning through a multi-scale dilated convolution module to obtain a water column feature map, and further outputs a second water column region prediction result; Determine the second loss value of the second convolutional neural network according to the second water column region prediction result and the second water column region label; Update the parameters of the second convolutional neural network according to the second loss value through the backpropagation algorithm; When the second loss value reaches a preset third threshold or the model accuracy reaches a preset fourth threshold, stop training to obtain a trained second water column recognition model.

5. A method for detecting the quality of a vehicle windshield wiper according to claim 1, characterized in that, The step of obtaining the time series data of the water column image of the third vehicle windshield to be detected and inputting the time series data of the water column image into the first water column recognition model and the second water column recognition model for object detection to obtain the time series data of the water column morphology specifically includes: Continuously sample the water column images of the third vehicle windshield at a preset sampling frequency to obtain the time series data of the water column image, and the time series data of the water column image includes multiple frames of water column images of the third vehicle windshield arranged in the sampling order; Input the time series data of the water column image into the first water column recognition model, determine the first water column morphology data of several frames of water column images according to the recognition result, input the time series data of the water column image into the second water column recognition model, and determine the second water column morphology data of several frames of water column images according to the recognition result; Generate the time series data of the water column morphology according to the sampling order of the water column images corresponding to the first water column morphology data and the second water column morphology data.

6. The vehicle wiper quality inspection method according to claim 2, characterized in that, The step of determining the positive sample time series data of the water column morphology corresponding to the positive sample set and the negative sample time series data of the water column morphology corresponding to the negative sample set specifically includes: Input the positive sample image sequence into the first water column recognition model, and determine the positive sample time series data of the water column morphology corresponding to the positive sample set according to the recognition result; Input the negative sample image sequence into the second water column recognition model, determine the negative sample time series data of the water column morphology corresponding to the negative sample set according to the recognition result, and label the negative sample time series data according to the defect type label.

7. A method for detecting the quality of vehicle windshield wipers according to claim 6, characterized in that, The step of comparing the time series data of the water column morphology with the positive sample time series data and the negative sample time series data and determining the quality detection result of the third vehicle windshield according to the comparison result specifically includes: Calculate the Euclidean distance between the time series data of the water column morphology and each positive sample time series data and each negative sample time series data through the KNN algorithm, and further calculate the similarity between the time series data of the water column morphology and each positive sample time series data and each negative sample time series data; When the mean of the similarities between the time series data of the water column morphology and each positive sample time series data is greater than or equal to a preset fifth threshold, determine that the quality of the third vehicle windshield is good; When the mean of the similarities between the water column form time series data and each of the positive sample time series data is less than a preset fifth threshold, determine the defect type label corresponding to the negative sample time series data with the highest similarity to the water column form time series data, and further determine the defect type of the third vehicle windshield wiper.

8. A vehicle wiper quality detection system, characterized in that, Including: A sample acquisition module, configured to determine a plurality of first vehicle windshield wipers with good quality and a plurality of second vehicle windshield wipers with quality defects, sample the water column images of the first vehicle windshield wipers to obtain a positive sample set, and sample the water column images of the second vehicle windshield wipers to obtain a negative sample set; A water column recognition model training module, configured to input the positive sample set into a pre-constructed first convolutional neural network to train a first water column recognition model, and input the negative sample set into a pre-constructed second convolutional neural network to train a second water column recognition model; A water column form detection module, configured to obtain the water column image time series data of a third vehicle windshield wiper to be detected, input the water column image time series data into the first water column recognition model and the second water column recognition model respectively for target detection, and obtain the water column form time series data; A quality detection result determination module, configured to determine the positive sample time series data of the water column form corresponding to the positive sample set and the negative sample time series data of the water column form corresponding to the negative sample set, compare the water column form time series data with the positive sample time series data and the negative sample time series data, and determine the quality detection result of the third vehicle windshield wiper according to the comparison result.

9. A vehicle wiper quality detection device, characterized in that, Including: At least one processor; At least one memory, configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for detecting the quality of a vehicle windshield wiper according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that, The program executable by the processor, when executed by the processor, is used to execute a method for detecting the quality of a vehicle windshield wiper according to any one of claims 1 to 7.

Images