A method for detecting abnormalities under the vehicle body based on two-dimensional images
Through the EMU undercar abnormality detection method based on two-dimensional images, the histogram equalization and visual learning machine algorithm are used, combined with the Cascade Rcnn object recognition algorithm, the complex and inaccurate detection in the existing technology are solved, and efficient and accurate are achieved The detection of under-vehicle abnormality of the EMU improves the safety of EMU operation.
Patent Information
- Application Number
- CN202011259297.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-12
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-11-12
AI Technical Summary
In the prior art, the under-vehicle detection method is complicated and complicated, and requires pre-defined templates and samples training on the historical images of different trains. The images are easily affected by lighting factors, resulting in inaccurate detection results and errors and omissions.
Using two-dimensional image-based abnormality detection method for undercarriage, images are collected through high-deep cameras, histogram equalization processing and image coordinate positioning are carried out, and parts and foreign objects are identified by visual learning machine algorithms, and Cascade Rcnn object recognition algorithm and PyTorch are used. Carry out training, simplify the inspection process, and adapt to different models and shooting conditions.
It reduces the labor required for inspection, simplifies the inspection process, improves the detection accuracy and speed, enhances the safety of EMU operation, and can accurately identify common key parts and foreign object abnormalities.
Smart Images

Figure CN112330646B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of motor vehicle detection, and in particular to a motor vehicle underbody abnormality detection method based on two-dimensional images. Background Art
[0002] As the mileage of high-speed trains and EMUs increases, the speed of EMUs increases. In order to ensure safe operation, the frequency of vehicle body maintenance increases. In order to maintain the efficient and high-speed operation of EMUs, it is necessary to detect the potential safety risks under the EMUs in a timely manner and increase the frequency of maintenance.
[0003] Usually, the inspection items of the underbody of the train include loose parts, oil leakage, dirt, foreign matter, etc. In the prior art, there has been a method of acquiring the status of the parts by collecting the underbody images of the train with a camera and identifying them, matching the status of the identified parts with the status of the pre-defined parts, and determining whether the detected parts are abnormal. However, this method requires pre-defining templates and sample training for the historical images of different trains, and then comparing the status of the detected parts with the status of the pre-defined parts to determine whether they are abnormal. The implementation process is complicated and cumbersome. In addition, the images collected by the camera are easily affected by factors such as lighting, and often cannot accurately reflect the abnormal state, resulting in inaccurate detection results and errors. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention proposes a method for detecting abnormalities on the underbody of a train based on two-dimensional images, which can reduce the manpower required for detection, simplify the detection process, improve detection accuracy and speed, and improve the safety of train operation.
[0005] To achieve the above object, a method for detecting abnormalities under a motor vehicle based on a two-dimensional image of the present invention comprises the following steps:
[0006] S1: Collect images and use a high-depth camera to take pictures of the bottom of the train;
[0007] S2: Image preprocessing: According to statistical information, it is determined that the grayscale of the entire train is too high or too low, which is abnormal shooting. Histogram equalization is used to process overexposure or underexposure in the image, and the image bit depth is reduced to enhance the image quality.
[0008] S3: Image coordinate positioning, confirm the image world coordinate system position, identify key parts, reorganize the image, confirm the vehicle head direction and the image position on the vehicle, and pair it with the template vehicle;
[0009] S4: Parts and foreign objects recognition, using trained visual learning machine algorithms to identify parts and foreign objects and locate key parts of EMUs;
[0010] S5: Image and part matching, compare the image with the template vehicle to find out the anomalies and record the detection results.
[0011] Furthermore, in step S2, the specific steps of histogram equalization include:
[0012] S201: Count the number n of each pixel in the gray level in the entire image k ,k=0,...,L-1;
[0013] S202: Calculate the probability distribution P(r) of each gray level in the image r =n k / n;
[0014] S203: Calculate the cumulative distribution probability S k ∑P(r) r ,k=0,1,2,...,L-1l;
[0015] S204: Calculate the gray value cdf (S k );
[0016] S205: Mapping back to the pixel value of the original pixel coordinate.
[0017] Furthermore, in step S3, the curve of the front of the vehicle is fitted through a depth of field camera and a simple threshold segmentation and Huffman transform algorithm. Threshold segmentation and edge extraction are performed starting from the grayscale image and depth of field image of the first image, and the area and edge curve are calculated. When the area and the curve meet the conditions of the front of the vehicle, it is the front of the vehicle. At the same time, the same process is performed from the last image to determine the positions of the two fronts of the vehicle. By locating the positions of the two fronts of the vehicle, the position of the image on the vehicle is determined.
[0018] Furthermore, step S4 also includes the following sub-steps:
[0019] S401: Collect the motor vehicle images of the vehicle model to be tested, collect more than 100 images for each part to be tested, collect about 50 images for oil stains and foreign objects, and collect some images with abnormal exposure, and manually reduce or increase the brightness of the images;
[0020] S402: using a bounding box annotation tool to annotate the parts in the collected images, and annotate the location, size and type of the parts using a bounding box;
[0021] S403: Use the Cascade Rcnn object recognition algorithm, use cross entropy as the loss function for training, the evaluation index is the average precision mean, use Adam as the optimizer, and the multi-classification cross entropy formula is Among them, M is the number of categories, y icIndicator variable, if the class is the same as sample i, it is 1, otherwise it is 0, p ic The predicted probability that the observed sample i belongs to category c;
[0022] S404: The model recognizes the images of the vehicle to be detected and obtains the position information of the parts in each image.
[0023] Further, in step S402, the marked parts include hexagonal screw heads, hexagonal screw heads, plugs, nuts, brake pads, pins, tees, shock absorbers, oil observation ports, wire clamps, oil filling caps, and flat screw heads, and abnormal locations are also marked, including oil stains and foreign matter.
[0024] Furthermore, in step S5, the fault-free test vehicle model is manually marked as a template for the parts on the entire vehicle, and the anomaly is found by comparing it with the marked template vehicle.
[0025] The method for detecting abnormalities of the underbody of a motor vehicle based on two-dimensional images of the present invention can perform abnormality detection on images of the underbody of a motor vehicle taken by using an underbody detection trolley, can detect the missing of common key parts and abnormal foreign objects in key parts, can adapt to the underbody abnormality detection of motor vehicles of different models and different shooting conditions, reduce the manpower required for detection, simplify the detection process, improve the detection accuracy and speed, and improve the safety of the operation of the motor vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The present invention will be further described and explained below in conjunction with the accompanying drawings.
[0027] Figure 1 It is a flow chart of a method for detecting abnormalities under a motor vehicle based on two-dimensional images according to a preferred embodiment of the present invention;
[0028] Figure 2 is a schematic diagram of the detection model in step S4. DETAILED DESCRIPTION
[0029] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.
[0030] like Figure 1 As shown, a method for detecting abnormalities under a motor vehicle based on a two-dimensional image in a preferred embodiment of the present invention comprises the following steps:
[0031] S1: Collect images, use a high-depth camera to take pictures of the bottom of the train, and use a moving car to take pictures of the high-speed rail.
[0032] S2: Image preprocessing. Due to environmental restrictions, the camera is prone to overexposure or underexposure, and the image is too dark or too bright, which is not conducive to subsequent manual and program viewing and analysis. According to statistical information, if the grayscale of the entire train image is too high or too low, it is an abnormal shot. Since the image has no complex background, histogram equalization is used to process the overexposure or underexposure in the image, reduce the image bit depth and enhance the image quality. The specific steps of histogram equalization include:
[0033] S201: Count the number n of each pixel in the gray level in the entire image k ,k=0,...,L-1;
[0034] S202: Calculate the probability distribution P(r) of each gray level in the image r =n k / n;
[0035] S203: Calculate the cumulative distribution probability S k ∑P(r) r ,k=0,1,2,...,L-1l;
[0036] S204: Calculate the gray value cdf (S k );
[0037] S205: Mapping back to the pixel value of the original pixel coordinate.
[0038] S3: Image coordinate positioning. Since the position of the car is not fixed each time, it is necessary to know the specific position of the captured image, confirm the direction of the front of the car and the position of the image on the vehicle, identify key parts, reorganize the image, and pair it with the template car. Use the depth of field camera and simple threshold segmentation and Hoffman transform algorithm to fit the curve of the front of the car. Start with the grayscale image and depth of field image of the first image to perform threshold segmentation and edge extraction, calculate the area and edge curve, and when the area and curve meet the conditions of the front of the car, it is the front of the vehicle. At the same time, the same process is performed from the last image to determine the positions of the two fronts of the car. By locating the positions of the two fronts of the car, the position of the image on the vehicle is determined.
[0039] S4: Parts and foreign objects recognition, using trained visual learning machine algorithms to identify parts and foreign objects and locate key parts of EMUs. Different vehicles and models vary greatly, and there are many parts suppliers. The main parts and defects are selected to build object recognition algorithms. The program is written in PyTorch and accelerated by CUDA. The computers used for training and testing are equipped with 4 NVIDIA independent graphics cards.
[0040] S401: Collect the motor vehicle images of the vehicle model to be tested, collect more than 100 images for each part to be tested, collect about 50 images for oil stains and foreign objects, and collect some images with abnormal exposure, and manually reduce or increase the brightness of the images;
[0041] S402: Use the bounding box annotation tool to annotate the parts in the collected images, and use the bounding box to annotate the size and type of the parts. The annotated parts include hexagonal screw heads, hexagonal screw heads, plugs, nuts, brake pads, pins, tees, shock absorbers, oil viewing ports, wire clamps, oil filling caps, and slotted screw heads. Abnormal locations are also annotated, including oil stains and foreign objects. Annotate the parts in the collected images, and use the bounding box to annotate the size and type of the parts;
[0042] S403: Use the Cascade Rcnn object recognition algorithm, use cross entropy as the loss function for training, the evaluation indicator is the average precision mean, and use Adam as the optimizer. Figure 2 Figure 1 is a schematic diagram of the model. Compared with the more common Faster Rcnn, Cascade Rcnn achieves the purpose of continuously optimizing the prediction results by cascading several detection networks. The multiple detection networks of cascade rcnn are trained on positive and negative samples determined based on different IOU thresholds. Figure 2 B0 is the bounding box output by the candidate generation network RPN, H1 is the first-level detector, and the obtained B1 and C1 are the bounding box correction and category respectively. Then the input above a certain IOU threshold is input into the second-level detector to obtain B2 and C2, and then input into the third-level detector to finally obtain the result. Cascade Rcnn has achieved good results on the general large-scale object detection dataset COCO dataset, with a boxAP index of 44.5, and can also achieve ideal results in other specific tasks. We pre-trained on the large dataset COCO, initialized with well-trained weights, and then further trained and adjusted on the dataset. The multi-classification cross entropy formula is Among them, M is the number of categories, y ic Indicator variable, if the class is the same as sample i, it is 1, otherwise it is 0, p ic The predicted probability that the observed sample i belongs to category c. The mean average precision is mAP (mean average precision). In multi-category object detection, each category can draw a curve based on recall and precision. AP is the area under the curve, and mAP is the average of APs of multiple categories.
[0043] S404: The model recognizes the images of the vehicle to be detected and obtains the location information of the parts on each image. After the model training is completed, the vehicle parts can be recognized. A special C++ deployment program is developed by Libtorch, and the half-precision acceleration model is used. After the detection is completed, the location information of the parts on each image can be obtained.
[0044] S5: Match the image and parts, compare the image with the template vehicle to find out the anomaly, and record the test results. Manually mark the fault-free test vehicle model as a template for the parts on the whole vehicle, and find out the anomaly by comparing it with the marked template vehicle. The bounding box of the part in the template is appropriately expanded, and it is calculated whether the detected parts intersect to determine whether the part exists on the vehicle to be tested. The size information and position of the part and the feature value of the image are used to determine whether the image to be tested is too different from the template. If the difference is large, switch the template for detection. There are too many parts in the bottom photo that are similar to foreign objects and oil stains. By matching with the area in the template, only the oil stains and foreign objects in the key parts are output.
[0045] The method for detecting abnormalities of the underbody of a motor vehicle based on two-dimensional images of the present invention can perform abnormality detection on images of the underbody of a motor vehicle taken by using an underbody detection trolley, can detect the missing of common key parts and abnormal foreign objects in key parts, can adapt to the underbody abnormality detection of motor vehicles of different models and different shooting conditions, reduce the manpower required for detection, simplify the detection process, improve the detection accuracy and speed, and improve the safety of the operation of the motor vehicle.
[0046] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.
Claims
1. A method for detecting abnormalities under a motor vehicle based on two-dimensional images, characterized in that: The following steps are involved: S1: Collect images and use a high-depth camera to take pictures of the bottom of the train; S2: Image preprocessing: According to the statistical information, it is determined that the grayscale of the entire train image is too high or too low, which is abnormal shooting. Histogram equalization is used to process overexposure or underexposure in the image, reduce the image bit depth and enhance the image quality. The specific steps of the histogram equalization include: S201: Count the number n of each pixel in the gray level in the entire image k ,k=0,…,L-1; S202: Calculate the probability distribution P(r) of each gray level in the image r =n k / n; S203: Calculate the cumulative distribution probability S k ∑P(r) r ,k=0,1,2,…,L-1l; S204: Calculate the gray value cdf (S k ); S205: Mapping back to the pixel value of the original pixel coordinate; S3: Image coordinate positioning, confirm the position of the image in the world coordinate system, identify key parts, reorganize the image, confirm the direction of the vehicle head and the position of the image on the vehicle, match it with the template vehicle, fit the curve of the vehicle head through the depth of field camera and simple threshold segmentation and Huffman transform algorithm, start threshold segmentation and edge extraction from the grayscale image and depth of field image of the first image, calculate the area and edge curve, when the area and curve meet the vehicle head conditions, it is the vehicle head, and perform the same process from the last image to determine the positions of the two vehicle heads. By locating the positions of the two vehicle heads, determine the position of the image on the vehicle; S4: Parts and foreign objects recognition, using trained visual learning machine algorithms to identify parts and foreign objects and locate key parts of EMUs, including: S401: Collect the motor vehicle images of the vehicle model to be tested, collect more than 100 images for each part to be tested, collect about 50 images for oil stains and foreign objects, and collect some images with abnormal exposure, and manually reduce or increase the brightness of the images; S402: using a bounding box annotation tool to annotate the parts in the collected images, and annotate the location, size and type of the parts using a bounding box; S403: Use the Cascade Rcnn object recognition algorithm, use cross entropy as the loss function for training, the evaluation index is the average precision mean, use Adam as the optimizer, and the multi-classification cross entropy formula is Among them, M is the number of categories, y ic Indicator variable, if the class is the same as sample i, it is 1, otherwise it is 0, p ic The predicted probability that the observed sample i belongs to category c; S404: The model recognizes the image of the vehicle to be detected and obtains the position information of the parts on each image; S5: Image and part matching, compare the image with the template vehicle to find out the anomalies and record the detection results.
2. The method for detecting abnormalities under the motor vehicle based on two-dimensional images according to claim 1 is characterized in that: In step S402, the marked parts include hexagonal screw heads, hexagonal screw heads, plugs, nuts, brake pads, pins, tees, shock absorbers, oil observation ports, wire clamps, oil filling caps, and flat screw heads, and abnormal locations are also marked, including oil stains and foreign matter.
3. The method for detecting abnormalities under the vehicle based on two-dimensional images according to claim 1, characterized in that: In step S5, the fault-free test vehicle model is manually marked as a template for the parts on the entire vehicle, and the abnormalities are found by comparing it with the marked template vehicle.
