A method and system for detecting missing water-blocking covers for an automobile chassis

Abstract

The application provides a kind of detection method and system for the detection of missing water-blocking covers of automobile chassis, wherein the method comprises: obtaining sample images of vehicle chassis and labeling to obtain labeled cover quantity, performing data augmentation on the sample images to obtain augmented images, and making the sample images and augmented images into an image dataset together, inputting the image dataset into a yolov5s network for model training to obtain a detection and recognition model, detecting the water-blocking covers of the image to be tested of the chassis through the detection and recognition model to obtain the recognized cover quantity, comparing with the labeled cover quantity to determine whether there is cover missing in the image to be tested, when there is missing, obtaining a template image and loading all template detection boxes therein, matching the detection boxes of the image to be tested to obtain a matching result, and determining the missing position of the water-blocking cover according to the matching result.The application can realize real-time detection and positioning of missing water-blocking covers, and improve the detection and positioning accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile painting production, and in particular to a detection method and system for missing of waterproof plugs of an automobile chassis. BACKGROUND

[0002] In the automobile painting production process, in order to improve the overall protection performance of the vehicle body, the vehicle body accessories need to be installed through multiple processes, and then a secondary visual re-inspection is performed by manual to confirm whether the number and position of the installed accessories meet the requirements. The chassis is an important carrier of the vehicle body, and a plurality of waterproof plugs need to be assembled thereon. In the automobile production process, the number of installed plugs will be missed due to long-time work of workers, and the number of waterproof plugs of the chassis is large. During the long-time inspection of the number of plugs by the inspection personnel, visual fatigue is likely to occur, which causes such quality problems to flow to subsequent processes, and there is a large quality risk. Through research on the online detection method for missing and positioning of the waterproof plugs based on vision, the probability of missing of the waterproof plugs can be further reduced, and unnecessary frequent occurrence of automobile production quality problems can be avoided.

[0003] There are many holes on the automobile chassis, and the interference of foreign matters such as adhesive tapes and sealants, resulting in a relatively complex detection environment of the waterproof plugs. In addition, the plug and the surrounding chassis have a low contrast (same color), and therefore, it is difficult to accurately detect all the waterproof plugs by using the traditional image processing method which is suitable for detection of a single background and obvious object edge features, which causes a large amount of missed detection and false detection. In addition, the position of the automobile chassis on the flow production line is in a real-time change state, and such real-time position offset further increases the use limitation of the traditional image processing technology.

[0004] On the one hand, due to the real-time movement of the position of the chassis during online detection, the position positioning of the missing plug lacks coordinate reference, and large deviation is likely to occur in the position positioning of the missing plug. On the other hand, although the plug and the hole are in a one-to-one correspondence, the holes on the automobile chassis are very numerous, and if the relationship matching is performed, the position is likely to be mispositioned, and a large amount of calculation consumption is caused, which does not meet the real-time detection requirement.

[0005] Therefore, there is an urgent need for a method capable of improving the positioning accuracy and realizing real-time online detection of missing of the waterproof plugs of the automobile chassis. SUMMARY

[0006] Therefore, it is necessary to provide a detection method and system for missing of the waterproof plugs of the automobile chassis in view of the above technical problems.

[0007] A method for detecting missing water-blocking covers of an automobile chassis, comprising the following steps: obtaining a sample image of an effective detection vehicle chassis, labeling the water-blocking covers in the sample image, and counting the number of labeled covers; performing data enhancement processing on the sample image to obtain an enhanced image, and making the sample image and the enhanced image into an image dataset together, inputting the image dataset into a yolov5s network for model training to obtain a detection and recognition model; detecting the water-blocking covers of the to-be-detected chassis image through the detection and recognition model, counting the number of recognized covers, and comparing the number of recognized covers with the number of labeled covers to determine whether there is cover missing in the to-be-detected chassis image; when cover missing is detected in the to-be-detected chassis image, obtaining a template image, loading all template detection boxes in the template image, matching the detection boxes of the to-be-detected chassis image, and obtaining a matching result; and determining the missing position of the water-blocking cover in the to-be-detected chassis image according to the matching result.

[0008] In one of the embodiments, the data enhancement processing on the sample image to obtain an enhanced image and making the sample image and the enhanced image into an image dataset together, inputting the image dataset into a yolov5s network for model training to obtain a detection and recognition model, comprises: performing image symmetry operation on the sample image according to x-axis, y-axis and center to obtain first enhanced data; adjusting the chroma, contrast and brightness of the sample image according to a preset ratio to obtain second enhanced data; adding noise interference to the sample image to obtain third enhanced data; taking the center of the sample image as a rotation axis, rotating the sample image and filling the edges with gray pixels to obtain fourth enhanced data; dividing the first enhanced data, the second enhanced data, the third enhanced data, the fourth enhanced data and the sample image according to a preset division ratio to obtain a training set and a validation set; inputting the training set and the validation set into the yolov5s network for model training to obtain the detection and recognition model.

[0009] In one of the embodiments, the detection of the water-blocking covers of the to-be-detected chassis image through the detection and recognition model further comprises: obtaining a to-be-detected chassis image of a to-be-detected vehicle, detecting whether there is a vehicle body stud in the to-be-detected chassis image; if there is, the to-be-detected vehicle is determined as an effective detection vehicle; if not, the to-be-detected vehicle is determined as an ineffective detection vehicle.

[0010] In one of the embodiments, the detection of the waterproof plug in the to-be-tested chassis image by the detection identification model, and the counting of the identified plug quantity, and the judgment of whether there is plug omission in the to-be-tested chassis image detection according to the identified plug quantity and the labeled plug quantity, comprises: identifying and detecting the to-be-tested chassis image by the detection identification model, outputting the identified waterproof plug, and counting the identified plug quantity; comparing the identified plug quantity and the labeled plug quantity to obtain a comparison result, and judging whether there is plug omission according to the comparison result; wherein, when the identified plug quantity and the labeled plug quantity are equal, it is determined that there is no plug omission in the to-be-tested chassis image detection; when the identified plug quantity and the labeled plug quantity are not equal, it is determined that there is plug omission in the to-be-tested chassis image detection.

[0011] In one of the embodiments, when there is plug omission in the to-be-tested chassis image detection, a template image is obtained, comprising: in the initial detection, if there is plug omission, reading the initially reserved local template image as the template image; in the non-initial detection, if there is plug omission, the to-be-tested chassis image in which the labeled plug quantity is equal to the identified plug quantity in the last detection is taken as the template image.

[0012] In one of the embodiments, loading all template detection boxes of the template image and matching them with the detection boxes of the to-be-tested chassis image to obtain a matching result, comprises: obtaining all template detection boxes of the template image, constructing a template detection box information set, and the template detection box information set includes position information of template detection boxes of multiple plugs; obtaining all detection boxes of the to-be-tested chassis image, constructing a detection box information set of the to-be-tested chassis image online detection, and the detection box information set includes position information of detection boxes of a plurality of plugs; according to the template detection box information set and the detection box information set, performing detection box matching to obtain a matching result.

[0013] In one of the embodiments, according to the template detection box information set and the detection box information set, performing detection box matching to obtain a matching result, comprises: selecting a target template detection box from the template detection box information set, and calculating the intersection over union of the target template detection box and all detection boxes in the detection box information set; obtaining a pre-set matching threshold, comparing the relationship between the matching threshold and the intersection over union; when the intersection over union is greater than the matching threshold, it is determined that the matching is successful, and there is no waterproof plug omission in the position corresponding to the target template detection box; otherwise, it is determined that the matching fails, and there is waterproof plug omission in the position corresponding to the target template detection box, and the position index of the target template detection box is recorded; repeating the above steps until all template detection boxes are matched, obtaining a matching result and ending the matching.

[0014] In one of the embodiments, the determining the missing position of the waterproof plug cover in the to-be-tested chassis image according to the matching result comprises: obtaining the index according to the matching result, searching for the position information of the corresponding template detection frame in the template detection frame information set according to the index, and obtaining the missing position of the waterproof plug cover in the to-be-tested chassis image according to the position information.

[0015] A detection system for missing waterproof plug cover of automobile chassis is used to implement the detection method for missing waterproof plug cover of automobile chassis, comprising: a waterproof plug cover labeling module, which is used to sample images of vehicle chassis, label the waterproof plug cover in the sample images, and count the labeled plug cover quantity; a recognition model obtaining module, which is used to perform data enhancement processing on the sample images to obtain enhanced images, combine the enhanced images with the sample images to form an image data set, input the image data set into a yolov5s network for model training, and obtain a detection recognition model; a plug cover missing judgment module, which is used to detect the waterproof plug cover of a to-be-tested chassis image by using the detection recognition model, count the recognized plug cover quantity, compare the recognized plug cover quantity with the labeled plug cover quantity, and judge whether there is missing plug cover in the to-be-tested chassis image; a detection frame matching module, which is used to obtain a template image and load all template detection frames in the template image when it is detected that there is missing plug cover in the to-be-tested chassis image, match the detection frames of the to-be-tested chassis image with the template detection frames to obtain a matching result; and a missing position determining module, which is used to determine the missing position of the waterproof plug cover in the to-be-tested chassis image according to the matching result.

[0016] Compared with the prior art, the advantages and beneficial effects of the present application are as follows: sample images of vehicle chassis are obtained, labeled, and the labeled plug cover quantity in the sample images is obtained, data enhancement processing is performed on the sample images to obtain enhanced images, the enhanced images are combined with the sample images to form an image data set, the image data set is input into a yolov5s network for training to obtain a detection recognition model, the waterproof plug cover of a to-be-tested chassis image is detected by using the detection recognition model, the recognized plug cover quantity is counted, the recognized plug cover quantity is compared with the labeled plug cover quantity, and it is judged whether there is missing plug cover in the to-be-tested chassis image, when there is missing plug cover, a template image is obtained, all template detection frames in the template image are loaded, the detection frames of the to-be-tested chassis image are matched with the template detection frames to obtain a matching result, the missing position of the waterproof plug cover in the to-be-tested chassis image is determined according to the matching result, through the detection of the waterproof plug cover, the influence of the chassis hole on the missing detection is avoided, the detection precision of the missing plug cover is improved, the plug cover is positioned by using the template image, the adverse effects of real-time deviation of the vehicle on the positioning of the plug cover are avoided, the positioning precision of the plug cover is improved, real-time detection and positioning of the missing waterproof plug cover are realized, and the detection and positioning precision are improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of a method for detecting missing water-blocking plugs in an automobile chassis in an embodiment;

[0018] Figure 2 A labeling diagram of a water-blocking plug in an embodiment;

[0019] Figure 3 A flowchart of a setting process of a template in an embodiment;

[0020] Figure 4 A template detection frame of a water-blocking plug in an embodiment;

[0021] Figure 5 An online detection frame of a water-blocking plug in an embodiment;

[0022] Figure 6 A flowchart of matching a template detection frame with an online detection frame in an embodiment;

[0023] Figure 7 An effect diagram of missing plug detection in an embodiment;

[0024] Figure 8 A structural diagram of a system for detecting missing water-blocking plugs in an automobile chassis in an embodiment;

[0025] Figure 9 Effect diagrams of several data augmentations in an embodiment, wherein (a) is an effect diagram using a flipping method, (b) is an effect diagram using a color transformation method, (c) is an effect diagram using a noise interference method, and (d) is an effect diagram using a rotation method;

[0026] Figure 10 A curve diagram of a water-blocking plug detection result in an embodiment;

[0027] Figure 11 A comparison diagram of a labeled image and a detection result image in an embodiment, wherein (a) is a data labeling diagram, and (b) is a model detection result diagram. DETAILED DESCRIPTION

[0028] Before the specific embodiment of the present application is described, the overall concept of the present application is described as follows:

[0029] The present application is mainly developed based on the automobile painting production process. In the current automobile production process, multiple water-blocking plugs need to be manually assembled. However, in a long time of work, the number of installed plugs may be missed, which may cause a large quality risk. The existing image processing method is difficult to accurately detect all the water-blocking plugs, and there are a large number of missed detections and false detections, resulting in low detection accuracy.

[0030] Therefore, the present application provides a detection method for missing of waterproof plugs of automobile chassis, which comprises the following steps: obtaining a sample image of the vehicle chassis, labeling the sample image to obtain the number of labeled plugs in the sample image, performing data enhancement processing on the sample image to obtain an enhanced image, and making the sample image and the enhanced image into an image data set together, inputting the image data set into a yolov5s network for model training to obtain a detection and recognition model, detecting the waterproof plugs of the to-be-tested chassis image through the detection and recognition model, counting the number of recognized plugs, comparing the number of recognized plugs with the number of labeled plugs, and determining whether there is missing of plugs in the to-be-tested chassis image, when there is missing, obtaining a template image, loading all template detection boxes in the template image, matching the detection boxes of the to-be-tested chassis image to obtain a matching result, and determining the missing position of the waterproof plugs in the to-be-tested chassis image according to the matching result. Through the detection of the waterproof plugs, the influence of the chassis holes on the missing detection is avoided, and the detection accuracy of the missing of the plugs is improved. At the same time, the missing plugs are positioned by using the template image, which can avoid the adverse effects of real-time deviation of the vehicle on the positioning of the plugs, thereby improving the positioning accuracy of the plugs, realizing real-time detection and positioning of the missing of the waterproof plugs, and improving the detection and positioning accuracy.

[0031] After introducing the overall concept of the present application, in order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the present application will be further described in detail below through specific embodiments combined with the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0032] In one embodiment, as shown in Figure 1 A detection method for missing of waterproof plugs of automobile chassis is provided, comprising the following steps:

[0033] In step S101, a sample image of the vehicle chassis is obtained, and the waterproof plugs in the sample image are labeled to obtain the number of labeled plugs.

[0034] Specifically, when the waterproof plugs are missing, the holes covered by the waterproof plugs will be exposed. However, there are many holes on the automobile chassis. If the holes are used as the labeling basis for detection to determine whether the plugs are missing, the determination logic will be relatively complex, the detected holes need to be filtered, and the imaging of holes of different sizes at different angles is relatively similar, which will result in a high risk of misjudgment.

[0035] Therefore, in the embodiment, the waterproof plug is taken as the labeling basis, the vehicle after the waterproof plug installation processing by the worker is taken as the sample vehicle, the sample image of the chassis of the sample vehicle is obtained, the waterproof plug in the sample image is labeled, the number of the waterproof plug is counted according to the labeling condition, and the labeled plug number is obtained, so that the number of the labeled plug can be used to judge whether the plug is missed in the to-be-tested vehicle. When labeling, the process of image labeling is as shown in Figure 2

[0036] In step S102, the sample image is subjected to data enhancement processing to obtain an enhanced image, and the sample image and the enhanced image are made into an image data set together. The image data set is input into the yolov5s network for model training to obtain a detection and recognition model.

[0037] Specifically, since the sample images collected in the automobile production site are limited and the image features are relatively single, which is not conducive to the learning of the model, therefore, in the process of making the image data set, the existing sample images need to be subjected to data enhancement to improve the diversity of the obtained data set and improve the generalization ability of the model.

[0038] The chassis images collected historically are taken as sample images, the obtained sample images are subjected to data enhancement, the data enhancement can make the performance form of the image features more diverse, and is used for training the neural network model, thereby improving the robustness of the neural network model and reducing the adverse effects of various additional factors on the model detection. The enhanced image obtained after enhancement is combined with the sample image and input into the yolov5s network for model training to obtain the trained detection and recognition model, and the recognition accuracy of the detection and recognition model is improved through training.

[0039] In step S102, the sample image is subjected to data enhancement processing to obtain an enhanced image, and the sample image and the enhanced image are made into an image data set together. The image data set is input into the yolov5s network for model training to obtain a detection and recognition model.

[0040] ​Specifically, when data augmentation is performed, one or more of flipping, color transformation, noise interference, and rotation can be used to process the sample image. Among them, the sample image is symmetrically operated according to the x-axis, y-axis and center by flipping to obtain first enhanced data; the chroma, contrast and brightness of the sample image are appropriately adjusted according to a certain proportion to obtain second enhanced data; Gaussian, salt and pepper noise interference is added to the sample image to obtain third enhanced data; the sample image is rotated by a specified number of degrees with the center as the rotation axis, and the edges are filled with gray pixels to obtain fourth enhanced data; all enhanced data obtained by the above method and the sample image are divided according to the preset division ratio, for example, the preset division ratio is set to 8:2 to divide all enhanced data to obtain a training set and a validation set, completing the data set production and division process.

[0041] Among many target detection neural networks, the yolo neural network is widely used for its fast detection speed and high detection accuracy. In this embodiment, yolov5s is used as a basic detection model, and the training set and the validation set are input to obtain a detection and recognition model. The detection and recognition model can accurately recognize the waterproof cover of the automobile chassis under various conditions.

[0042] Step S103, detecting the waterproof cover of the to-be-tested chassis image through the detection and recognition model, counting the number of recognized covers, and comparing it with the number of labeled covers to determine whether the to-be-tested chassis image detection has cover omission.

[0043] Specifically, after the detection and recognition model is trained, the to-be-tested chassis image currently obtained is input into the detection and recognition model, and the waterproof cover recognized in the to-be-tested chassis image is output. The number of recognized covers is counted, so as to compare the number of labeled covers with the number of recognized covers to determine whether there is a situation of waterproof cover omission. The detection and recognition model can accurately determine whether there is a waterproof cover omission, thereby improving the accuracy of cover omission recognition.

[0044] Among them, the step of judging whether the vehicle corresponding to the to-be-tested chassis image is a valid detection vehicle through the detection and recognition model includes: obtaining the to-be-tested chassis image of the to-be-tested vehicle, detecting whether there is a vehicle body stud in the to-be-tested chassis image; if there is, the to-be-tested vehicle is determined as a valid detection vehicle; if not, the to-be-tested vehicle is determined as an invalid detection vehicle.

[0045] Specifically, in the actual detection process, some empty cars and repair cars do not need to be detected, otherwise they are considered as misjudgment. Therefore, when labeling, the labeling data of the vehicle body stud category can be added. Whether the to-be-tested vehicle is an empty car or a repair car can be determined by the vehicle body stud, because there is no vehicle body stud in the empty car and the repair car.

[0046] After obtaining the chassis image of the vehicle to be tested, it is determined whether the corresponding vehicle to be tested is a valid detection vehicle by detecting whether there is a vehicle body stud in the chassis image. When there is a vehicle body stud, the corresponding vehicle to be tested is determined to be a valid detection vehicle, and the detection of the waterproof plug cover of the corresponding vehicle is continued. Otherwise, the corresponding vehicle to be tested is determined to be an invalid detection vehicle, and the chassis detection of the corresponding vehicle is terminated.

[0047] In step S103, the detection recognition model is used to identify and detect the chassis image to be tested, output the identified waterproof plug cover, and count the number of identified plug covers. The number of identified plug covers is compared with the number of labeled plug covers to obtain a comparison result, and it is determined whether there is a missing plug cover according to the comparison result. When the number of identified plug covers is equal to the number of labeled plug covers, it is determined that there is no missing plug cover in the detection of the chassis image to be tested. When the number of identified plug covers is not equal to the number of labeled plug covers, it is determined that there is a missing plug cover in the detection of the chassis image to be tested.

[0048] Specifically, the chassis image to be tested is input into the detection recognition model to detect the waterproof plug cover in the chassis image to be tested, output the identified waterproof plug cover, and count the number of identified plug covers. The relationship between the number of identified plug covers and the number of labeled plug covers is compared to obtain a comparison result, and it is determined whether there is a missing plug cover in the chassis image to be tested according to the comparison result. When the number of identified plug covers is equal to the number of labeled plug covers, it is determined that there is no missing plug cover in the chassis image to be tested. Otherwise, it is determined that there is a missing plug cover in the chassis image to be tested. Thus, the judgment of whether there is a missing waterproof plug cover can be made, and accurate and stable judgment can be made in a complex detection environment. The number of waterproof plug covers is used to judge the missing, which effectively shields the interference of the chassis hole.

[0049] In step S104, when it is detected that there is a missing plug cover in the chassis image to be tested, a template image is obtained, and all template detection boxes of the template image are loaded and matched with the detection boxes of the chassis image to be tested to obtain a matching result.

[0050] Specifically, since the number of waterproof plug covers of each part of the automobile chassis is fixed, when it is found that the number of waterproof plug covers in a certain area is less than the reference number of labeled plug covers, it indicates that there is a missing plug cover. At this time, the missing plug cover needs to be located.

[0051] When locating the missing plug cover, the chassis image to be tested without missing is used as a template image, and all template detection boxes of the template image are labeled with all waterproof plug covers. All template detection boxes in the template image are loaded and matched with the detection boxes in the chassis image to be tested to obtain a corresponding matching result.

[0052] The step of obtaining the template image comprises: in initial detection, if there is a missing cover, reading a locally reserved initial template image as the template image; and in non-initial detection, if there is a missing cover, taking a to-be-tested chassis image in which the number of marked covers is equal to the number of recognized covers in the last detection as the template image.

[0053] Specifically, a detection count is performed during detection, and a detection counter is initialized, and the value of the counter is constantly increased as the detection process proceeds. The setting process of the template image is as shown in Figure 3 When the first detection is performed, if a missing cover occurs, the last cover position template image has not been generated at this time, and therefore, the locally reserved initial template image can be read as the template image. The last detection of the entire detection process saves the current cover position template image in a local folder, facilitating reading during the first detection of the next detection process.

[0054] In actual detection, since the template image is selected from an online detection image and the time interval between the current online detection image and the template image is short, the position movement of the automobile during the entire detection process can be ignored, so that the missing position of the waterproof cover can be accurately positioned.

[0055] In non-initial detection, if there is a missing cover, the to-be-tested chassis image in which the number of marked covers is equal to the number of recognized covers in the last detection is taken as the template image. Since the time interval between two detections is short, the position movement of the automobile can be ignored to facilitate accurate positioning of the missing waterproof cover. If there is no situation in which the number of marked covers is equal to the number of recognized covers before this detection, the template image of the last detection can be taken as the template image of the current detection.

[0056] The step of detection frame matching comprises: obtaining all template detection frames of the template image, constructing a template detection frame information set, the template detection frame information set comprising position information of template detection frames of a plurality of covers; obtaining all detection frames of the to-be-tested chassis image, constructing a detection frame information set of online detection of the to-be-tested chassis image, the detection frame information set comprising position information of detection frames of a plurality of covers; and performing detection frame matching according to the template detection frame information set and the detection frame information set to obtain a matching result.

[0057] Specifically, when the detection frame matching is performed, all template detection frames of the covers in the template image need to be loaded, as shown in Figure 4As shown, a template detection box information set B1 = {box0, box1, ..., boxN} is constructed using all template detection boxes; where boxN is the position information of the Nth blocking template detection box in the template diagram, and the data format of the template detection box is {x, y, w, h}; where x is the x-coordinate of the upper left corner of the detection box, y is the y-coordinate of the upper left corner of the detection box, w is the width of the detection box, and h is the height of the detection box.

[0058] Similarly, obtain all detection boxes in the image of the chassis to be tested, such as Figure 5 As shown, a set of detection box information for online detection of the chassis image under test is obtained by constructing all detection boxes, B2={Box0,Box1,…,BoxM}; where BoxM is the detection box position information of the Mth cover in the online detection image, and its data format is the same as that of the template detection box.

[0059] Based on the template detection box information set and the detection box information set, all detection boxes are matched one by one with the template detection boxes to obtain the corresponding matching results. This allows for the determination of the location of any missing waterproof plugs based on the matching results. Through the above steps, the location of any missing plugs can be quickly and effectively marked, while shielding the real-time offset interference of the vehicle's own position, thus improving the accuracy of plug location.

[0060] The steps for obtaining the matching result include: selecting a target template detection box from the template detection box information set, and calculating the intersection-union ratio (IUR) of the target template detection box with all detection boxes in the detection box information set; obtaining a pre-set matching threshold, and comparing the relationship between the matching threshold and the IUR; if the IUR is greater than the matching threshold, the matching is considered successful, and there is no waterproof plugging leak at the corresponding position of the target template detection box; otherwise, the matching is considered unsuccessful, and there is a waterproof plugging leak at the corresponding position of the target template detection box, and the position index of the target template detection box is recorded; repeating the above steps until all template detection boxes are matched, obtaining the matching result, and ending the matching process.

[0061] Specifically, when performing detection box matching, although an online detection map is used as a template map, in some special cases, the position of the car itself may be significantly offset at a certain moment. Therefore, it is necessary to repeatedly experiment and debug the pre-set matching threshold to adapt to special situations, expand the applicability of the detection box matching method, reduce false detections caused by some special reasons, and improve production efficiency.

[0062] like Figure 6 The diagram illustrates the detection box matching process. A target template detection box is selected from the constructed template detection box information set, and the intersection-over-union (IoU) ratio between the target template detection box and all detection boxes in the detection box information set is calculated. The calculation formula is as follows:

[0063]

[0064] Wherein, B1 represents a template detection frame, B2 represents an actual online detection frame; B1∩B2 represents the intersection of the two detection frames, and B1∪B2 represents the union of the two detection frames. When there is an overlap between the two detection frame regions, the intersection region is greater than 0; the larger the intersection region is, the smaller the union region is. Therefore, the numerical calculation range of IoU is 0-1, when IoU is 0, the two detection frames have no intersection, and the matching degree is the lowest; when IoU is 1, the two detection frames completely overlap, and the matching degree is the highest.

[0065] A matching threshold is obtained, for example, the matching threshold is set to 0.2, when the intersection over union is greater than the matching threshold, it is determined that the matching is successful, and there is no waterproof cover missing situation for the target template detection frame, otherwise, it is determined that the matching fails, and there is a waterproof cover missing situation for the target template detection frame, and the position index of the target template detection frame is recorded, so as to position the position of the target template detection frame according to the position index, thereby realizing accurate positioning of the missing position of the waterproof cover.

[0066] The above steps are repeated until all template detection frames in the template detection frame information set are matched, the matching result is obtained, and the matching is ended. The matching result includes index or all matching success information, so as to position the missing position of the cover.

[0067] In step S105, according to the matching result, the missing position of the waterproof cover of the to-be-tested chassis image is determined.

[0068] Specifically, according to the obtained matching result, when there is a waterproof cover missing, the missing position of the waterproof cover can be determined; when there is no waterproof cover missing, all matching success information is obtained.

[0069] Wherein, step S105 includes: obtaining an index according to the matching result, finding the position information of the corresponding template detection frame in the template detection frame information set according to the index; and obtaining the missing position of the waterproof cover in the to-be-tested chassis image according to the position information.

[0070] Specifically, when there is a waterproof cover missing, the recorded index is obtained according to the matching result, and the position information of the corresponding template detection frame in the template detection frame information set is found according to the index, as shown by the black solid line frame in Figure 7 The position information is the missing position of the waterproof cover, thereby realizing accurate positioning of the missing position of the waterproof cover, so as to accurately process the missing position.

[0071] In the embodiment, the sample image of the vehicle chassis is obtained, and the sample image is labeled to obtain the labeled cover quantity in the chassis image. The sample image is subjected to data enhancement processing to obtain an enhanced image, and the enhanced image and the sample image are made into an image data set together. The image data set is input into a yolov5s network for training to obtain a detection and recognition model. The detection and recognition model is used to detect the waterproof cover of the to-be-tested chassis image, and the recognized cover quantity is counted and compared with the labeled cover quantity to determine whether the to-be-tested chassis image detection has cover omission. When there is omission, a template image is obtained, and all template detection boxes in the template image are loaded and matched with the detection boxes of the to-be-tested chassis image to obtain a matching result. The matching result is used to determine the omission position of the waterproof cover. Through the detection of the waterproof cover, the influence of the chassis hole on the omission detection is avoided, and the detection accuracy of the cover omission is improved. At the same time, the template image is used to position the omitted cover, which can avoid the adverse effects of real-time vehicle deviation on cover positioning, thereby improving the cover positioning accuracy, realizing real-time detection and positioning of the waterproof cover omission, and improving the detection and positioning accuracy.

[0072] As shown in Figure 8 , a detection system 80 for detecting omission of a waterproof cover of an automobile chassis is provided for implementing the method for detecting omission of a waterproof cover of an automobile chassis as described above, comprising a waterproof cover labeling module 81, an identification model obtaining module 82, a cover omission judgment module 83, a detection box matching module 84, and an omission position determination module 85, wherein:

[0073] The waterproof cover labeling module 81 is used to obtain a sample image of a vehicle chassis, and label the waterproof cover in the sample image to count the labeled cover quantity.

[0074] The identification model obtaining module 82 is used to perform data enhancement processing on the sample image to obtain an enhanced image, and make the enhanced image and the sample image into an image data set together. The image data set is input into a yolov5s network for model training to obtain a detection and recognition model.

[0075] The cover omission judgment module 83 is used to detect the waterproof cover of the to-be-tested chassis image through the detection and recognition model, count the recognized cover quantity, and compare the recognized cover quantity with the labeled cover quantity to determine whether the to-be-tested chassis image detection has cover omission.

[0076] The detection box matching module 84 is used to obtain a template image when the to-be-tested chassis image detection has cover omission, load all template detection boxes in the template image, and match the template detection boxes with the detection boxes of the to-be-tested chassis image to obtain a matching result.

[0077] The omission position determination module 85 is used to determine the omission position of the waterproof cover in the to-be-tested chassis image according to the matching result.

[0078] In one embodiment, the identification model acquisition module 82 is specifically configured to: perform image symmetry operations on the sample image in three ways of x-axis, y-axis and center to obtain first enhanced data; adjust the chrominance, contrast and brightness of the sample image according to a preset ratio to obtain second enhanced data; add noise interference to the sample image to obtain third enhanced data; take the center of the sample image as a rotation axis to rotate the sample image, and fill the edges of the sample image with gray pixels to obtain fourth enhanced data; divide the first enhanced data, the second enhanced data, the third enhanced data, the fourth enhanced data and the sample image according to a preset division ratio to obtain a training set and a verification set; and input the training set and the verification set into a yolov5s network for model training to obtain a detection identification model.

[0079] In one embodiment, the plug cover omission judgment module 83 is further configured to: acquire a to-be-tested chassis image of a to-be-tested vehicle, and detect whether there is a vehicle body stud in the to-be-tested chassis image; if there is, the to-be-tested vehicle is determined as a valid detection vehicle; if there is not, the to-be-tested vehicle is determined as an invalid detection vehicle.

[0080] In one embodiment, the plug cover omission judgment module 83 is specifically configured to: perform identification detection on the to-be-tested chassis image by the detection identification model, output identified waterproof plugs, and count to obtain an identified plug quantity; compare the identified plug quantity with a labeled plug quantity to obtain a comparison result, and judge whether there is plug cover omission according to the comparison result; wherein, when the identified plug quantity and the labeled plug quantity are equal, it is determined that there is no plug cover omission in the to-be-tested chassis image detection; when the identified plug quantity and the labeled plug quantity are not equal, it is determined that there is plug cover omission in the to-be-tested chassis image detection.

[0081] In one embodiment, the omission position determination module 85 is specifically configured to: acquire an index according to the matching result, find position information of a corresponding template detection frame in the template detection frame information set according to the index, and obtain a missing position of the waterproof plug cover in the to-be-tested chassis image according to the position information.

[0082] In one embodiment, data augmentation is performed in the ways of flipping, color transformation, noise interference and rotation, and relevant data augmentation results are as shown in Figure 9 The results from top to bottom and from left to right are: an effect diagram (a) using flipping augmentation, an effect diagram (b) using color transformation augmentation, an effect diagram (c) using noise interference augmentation, and an effect diagram (d) using rotation augmentation.

[0083] After data augmentation, a total of more than 300 automobile plug cover data are obtained, and a ratio of 8:2 is used to divide the training set and the verification set, so as to complete the whole data set production and division process.

[0084] Among many target detection neural networks, the YOLO neural network is widely used due to its fast detection speed and high detection accuracy. In this embodiment, YOLOv5s is used as a basic detection model, and the model is fine-tuned on this basis. After 200 rounds of training, the precision reaches 99.7%, the recall reaches 99.5%, the value of mAP@0.5 reaches 0.996, and the value of mAP@0.5:0.95 is 0.788, and the recognition effect is very good, which can accurately identify the waterproof plug cover of the automobile chassis under various conditions and judge whether there is a plug cover missing.

[0085] The curve diagram of the specific experimental data is shown in Figure 10 . Among them, the table Box, Objectness, Classification respectively represents the object prediction box regression loss, object prediction confidence loss, and object prediction classification loss in the model training stage; the table val Box, val Objectness, val Classification respectively represents the object prediction box regression loss, object prediction confidence loss, and object prediction classification loss in the model verification stage; the table Precision, Recall respectively represents the precision and recall in the model verification stage; the table mAP@0.5 refers to the average precision mean value of all object categories after summing the average precision after setting the intersection over union threshold to 0.5; the table mAP@0.5:0.95 refers to the average precision mean value of all object categories after summing the average precision in the process of gradually increasing the intersection over union threshold from 0.5 to 0.95 with a step of 0.05.

[0086] In order to further illustrate the effect of model detection, this embodiment randomly selects an image of a waterproof plug cover for detection, and compares the labeled image with the detection result image, and the result is shown in Figure 11 . Subgraph (a) is a data labeling graph, and subgraph (b) is a model detection result graph. Through comparison, it can be found that the model has good detection effect on waterproof plug cover and vehicle body stud.

[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0088] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, and optionally, they can be realized by program codes executable by computing devices, so that they can be stored in computer storage media (ROM / RAM, magnetic disc, optical disc) and executed by computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be made into individual integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module. Therefore, the present application is not limited to any specific hardware and software combination.

[0089] The above is a further detailed description of the present application in combination with specific embodiments, and the specific implementation of the present application should not be limited to these descriptions. For those skilled in the art, some simple deductions or substitutions can be made without departing from the concept of the present application, which should be considered as falling within the protection scope of the present application.

Claims

1. A method for detecting missing of a waterproof plug cover for an automobile chassis, characterized by, The method comprises the following steps: obtaining a sample image of a vehicle chassis and labeling a waterproof plug cover in the sample image, and counting to obtain a labeled plug cover number, wherein the sample image is a sample vehicle chassis image after waterproof plug cover installation processing; performing data enhancement processing on the sample image to obtain an enhanced image, and making the sample image and the enhanced image into an image data set together, inputting the image data set into a yolov5s network for model training, and obtaining a detection and recognition model; obtaining a to-be-tested chassis image of a to-be-tested vehicle, and detecting whether a vehicle body stud exists in the to-be-tested chassis image; if the vehicle body stud exists, the to-be-tested vehicle is determined as a valid detection vehicle; if the vehicle body stud does not exist, the to-be-tested vehicle is determined as an invalid detection vehicle; detecting the waterproof plug cover of the to-be-tested chassis image of the valid detection vehicle through the detection and recognition model, counting to obtain a recognized plug cover number, and comparing the recognized plug cover number with the labeled plug cover number to determine whether there is plug cover omission in the to-be-tested chassis image detection; when there is plug cover omission in the to-be-tested chassis image detection, a template image is obtained, and all template detection boxes in the template image are loaded and matched with detection boxes of the to-be-tested chassis image to obtain a matching result, including: in initial detection, if there is plug cover omission, a locally reserved initial template image is read as a template image; in non-initial detection, if there is plug cover omission, a to-be-tested chassis image in which the labeled plug cover number is equal to the recognized plug cover number in the last detection is taken as a template image; all template detection boxes of the template image are obtained, a template detection box information set is constructed, the template detection box information set includes position information of template detection boxes of a plurality of plug covers; all detection boxes of the to-be-tested chassis image are obtained, a detection box information set of online detection of the to-be-tested chassis image is constructed, and the detection box information set includes position information of detection boxes of a plurality of plug covers; detection box matching is performed according to the template detection box information set and the detection box information set to obtain a matching result; determining a missing position of the waterproof plug cover in the to-be-tested chassis image according to the matching result.

2. The method for detecting missing water-blocking covers of an automobile chassis according to claim 1, characterized in that, performing data enhancement processing on the sample image to obtain an enhanced image, and making the sample image and the enhanced image into an image data set together, inputting the image data set into a yolov5s network for model training, and obtaining a detection and recognition model, comprising: performing image symmetry operation on the sample image in three ways of x-axis, y-axis and center to obtain first enhanced data; adjusting the chroma, contrast and brightness of the sample image according to a preset ratio to obtain second enhanced data; adding noise interference to the sample image to obtain third enhanced data; taking the center of the sample image as a rotation axis, rotating the sample image, and filling the edge with gray pixels to obtain fourth enhanced data; dividing the first enhanced data, the second enhanced data, the third enhanced data, the fourth enhanced data and the sample image according to a preset division ratio to obtain a training set and a verification set; inputting the training set and the verification set into the yolov5s network for model training to obtain the detection and recognition model.

3. The method for detecting missing water-blocking covers of an automobile chassis according to claim 1, characterized in that, The detection of the waterproof cover of the to-be-tested chassis image is performed by the detection identification model, and the number of identified covers is obtained. According to the number of identified covers and the number of labeled covers, it is judged whether there is cover omission in the detection of the to-be-tested chassis image, comprising: The detection identification model is used to identify and detect the to-be-tested chassis image, and the identified waterproof cover is output. The number of identified covers is obtained. Compare the number of identified covers with the number of labeled covers to obtain a comparison result, and determine whether there is cover omission according to the comparison result; When the number of identified covers and the number of labeled covers are equal, it is determined that there is no cover omission in the detection of the to-be-tested chassis image. When the number of identified covers and the number of labeled covers are not equal, it is determined that there is cover omission in the detection of the to-be-tested chassis image.

4. The method for detecting missing water plug covers for an automotive chassis according to claim 1, wherein, According to the template bounding box information set and the bounding box information set, the bounding box matching is performed to obtain a matching result, comprising: Select a target template bounding box from the template bounding box information set, and calculate the intersection over union of the target template bounding box and all bounding boxes in the bounding box information set; Get a pre-set matching threshold, compare the relationship between the matching threshold and the intersection over union; When the intersection over union is greater than the matching threshold, it is determined that the matching is successful, and there is no waterproof cover omission in the position corresponding to the target template bounding box; On the contrary, it is determined that the matching fails, and there is waterproof cover omission in the position corresponding to the target template bounding box, and the position index of the target template bounding box is recorded; Repeat the above steps until all template bounding boxes are matched, obtain the matching result and end the matching.

5. The method for detecting missing water-blocking covers of an automobile chassis according to claim 4, characterized in that, According to the matching result, the omission position of the waterproof cover in the to-be-tested chassis image is determined, comprising: According to the matching result, the index is obtained, and the position information of the corresponding template bounding box in the template bounding box information set is found according to the index; According to the position information, the omission position of the waterproof cover in the to-be-tested chassis image is obtained.

6. A detection system for leaking waterproof plugs on automobile chassis, characterized in that, A method for detecting the omission of a waterproof cover of a vehicle chassis is used to implement any one of claims 1-5, comprising: A waterproof cover labeling module is used to obtain a sample image of a vehicle chassis, and label the waterproof cover in the sample image. The number of labeled covers is obtained by statistics. The sample image is a sample vehicle chassis image after waterproof cover installation processing; An identification model acquisition module is used to perform data enhancement processing on the sample image to obtain an enhanced image, and to be used as an image data set together with the sample image. The image data set is input into a yolov5s network for model training to obtain a detection identification model; The plug cover missing judgment module is configured to acquire a to-be-tested chassis image of a to-be-tested vehicle, detect whether there is a vehicle body stud in the to-be-tested chassis image, determine that the to-be-tested vehicle is a valid detection vehicle if there is the vehicle body stud, determine that the to-be-tested vehicle is an invalid detection vehicle if there is not the vehicle body stud, detect a waterproof plug cover of the to-be-tested chassis image of the valid detection vehicle through the detection recognition model, count a recognized plug cover quantity, compare the recognized plug cover quantity with the labeled plug cover quantity, and judge whether there is plug cover missing in the to-be-tested chassis image detection; The detection box matching module is configured to acquire a template image when it is detected that there is plug cover missing in the to-be-tested chassis image detection, load all template detection boxes in the template image, match the detection boxes in the template image with detection boxes of the to-be-tested chassis image, and obtain a matching result. The matching result includes the following. When initial detection is performed, if there is plug cover missing, an initial template image reserved locally is read as a template image. When non-initial detection is performed, if there is plug cover missing, a to-be-tested chassis image in which the labeled plug cover quantity is equal to the recognized plug cover quantity in the last detection is taken as a template image. All template detection boxes of the template image are acquired, a template detection box information set is constructed, the template detection box information set includes position information of template detection boxes of multiple plug covers, all detection boxes of the to-be-tested chassis image are acquired, a detection box information set of online detection of the to-be-tested chassis image is constructed, the detection box information set includes position information of detection boxes of a plurality of plug covers, and detection box matching is performed according to the template detection box information set and the detection box information set to obtain a matching result. The missing position determination module is configured to determine a missing position of a waterproof plug cover in the to-be-tested chassis image according to the matching result.

Images