A method and system for train defect detection

Abstract

The present invention discloses a method for detecting train defects, comprising the following steps: obtaining a template of the train and the RGB images to be detected, the template and the depth images to be detected; preprocessing the RGB images to obtain the template and the grayscale images to be detected; performing feature matching on the template and the grayscale images to be detected to obtain a homography matrix; comparing the template grayscale image and the template depth image, and using the homography matrix to perform affine transformation on the grayscale image to be detected and the depth image to be detected to obtain the transformed grayscale image to be detected and the depth image to be detected; performing defect detection on the grayscale image and the depth image to obtain a grayscale defect position map and a depth defect position map; comparing the grayscale defect position map and the depth defect position map to determine the true defect points of the train. The technical solution provided by the present invention eliminates the influence of factors such as illumination and camera position transformation to a certain extent through double recognition and verification technology for RGB images and depth images, improves the detection accuracy, and achieves a better detection effect.

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and particularly to a method for detecting train defects. Background Art

[0002] There are various trains across the country, including bullet trains, high-speed trains, subways, etc. During the operation of trains, problems will inevitably occur gradually. For the safety of passengers, it is necessary to regularly inspect the trains. However, the inspection requires a large amount of manpower and material resources, and manual inspection can usually only be carried out when the train is out of service at night, which increases a lot of difficulties for the inspection work.

[0003] The average detection time of the manual detection method by technical personnel is relatively long, the efficiency is not high, and the inspection effect is easily affected by human factors, and it is easy to miss or misdetect.

[0004] At present, there are also some defect detection methods based on artificial intelligence. However, due to problems such as limitations in lighting, shooting environment, shooting equipment, and the algorithm itself, the detection effect is not good, it is difficult to be fully applicable to trains, and it is easy to have false positives and false negatives, and a large amount of manpower is still required to review the detection results. Summary of the Invention

[0005] In view of the above deficiencies in the current technology, the present invention provides a method for detecting train defects, which improves the detection accuracy through dual recognition and verification technologies and can achieve a better detection effect.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0007] A method for detecting train defects, comprising the following steps:

[0008] Obtain the template RGB image, the RGB image to be detected, the template depth image, and the depth image to be detected of the train respectively;

[0009] Preprocess the template RGB image and the RGB image to be detected to obtain the template grayscale image and the grayscale image to be detected;

[0010] Perform feature matching on the template grayscale image and the grayscale image to be detected to obtain the homography matrix;

[0011] Compare the template grayscale image and the template depth image, and use the homography matrix to perform affine transformation on the grayscale image to be detected and the depth image to be detected to obtain the transformed grayscale image to be detected and the depth image to be detected;

[0012] Perform defect detection on the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected to obtain the grayscale defect position map and the depth defect position map;

[0013] Compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train.

[0014] According to one aspect of the present invention, the RGB map and the depth map are obtained by shooting with a camera that supports shooting RGB maps and depth maps.

[0015] According to one aspect of the present invention, when the camera shoots the RGB map and the depth map, alignment processing is performed.

[0016] According to one aspect of the present invention, the preprocessing of the template RGB map and the RGB map to be detected includes the following steps: converting the RGB map to a grayscale map; performing median filtering on the grayscale map; performing non-limiting adaptive histogram equalization on the grayscale map.

[0017] According to one aspect of the present invention, the feature matching of the template grayscale map and the grayscale map to be detected to obtain the homography matrix includes the following steps:

[0018] Perform feature detection on the image using the ORB feature detector to obtain feature points;

[0019] Match the feature points with a brute-force matcher to obtain matching points;

[0020] Use cross-validation and the RANSAC algorithm to remove incorrect matching points;

[0021] Obtain the homography matrix according to the matching points.

[0022] According to one aspect of the present invention, the distance type used by the brute-force matcher is the Hamming distance.

[0023] According to one aspect of the present invention, the defect detection of the template grayscale map, the template depth map, the transformed grayscale map to be detected, and the depth map to be detected to obtain the grayscale defect position map and the depth defect position map is as follows:

[0024] Perform subtraction, binarization, and erosion on the template grayscale map and the transformed grayscale map to be detected to obtain the grayscale defect position map;

[0025] Perform subtraction, binarization, and erosion on the template depth map and the transformed depth map to be detected to obtain the depth defect position map.

[0026] According to one aspect of the present invention, the true defect points are: the positions where the pixel values on both the grayscale defect position map and the depth defect position map are 255.

[0027] A train defect detection system, based on the train defect detection method described above, includes:

[0028] An image acquisition module, which respectively acquires a template RGB image, a to-be-detected RGB image, a template depth image, and a to-be-detected depth image of the train;

[0029] An image preprocessing module, which preprocesses the template RGB image and the to-be-detected RGB image to obtain a template grayscale image and a to-be-detected grayscale image;

[0030] A feature matching module, which performs feature matching on the template grayscale image and the to-be-detected grayscale image to obtain a homography matrix;

[0031] An affine transformation module, which is used to compare the template grayscale image and the template depth image, and uses the homography matrix to perform affine transformation on the to-be-detected grayscale image and the to-be-detected depth image to obtain the transformed to-be-detected grayscale image and the to-be-detected depth image;

[0032] A defect detection module, which is used to perform defect detection on the template grayscale image, the template depth image, the transformed to-be-detected grayscale image, and the to-be-detected depth image to obtain a grayscale defect position map and a depth defect position map;

[0033] A defect confirmation module, which is used to compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train.

[0034] Advantages of the implementation of the present invention:

[0035] The present invention provides a train defect detection method, which improves the detection accuracy through a double recognition and verification technology for RGB images and depth images, and can achieve a better detection effect.

[0036] The double recognition and verification eliminates the influence of a series of environmental factors such as illumination and camera position transformation to a certain extent, and also eliminates the limitations brought by the insufficient accuracy of the depth image. Therefore, compared with the traditional algorithm, the detection effect has been greatly improved. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 It is a flowchart of a train defect detection method described in the present invention;

[0039] Figure 2 It is a template RGB image at the train detection location;

[0040] Figure 3 It is a to-be-detected RGB image at the train detection location;

[0041] Figure 4 It is the corresponding diagram of the matching points at the train detection location;

[0042] Figure 5 It is the diagram of the defect positions in the grayscale image at the train detection location. Specific implementation manners

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention.

[0044] Embodiment 1

[0045] As Figure 1 shown, a train defect detection method includes the following steps:

[0046] S1: Obtain the template RGB image, the RGB image to be detected, the template depth image, and the depth image to be detected of the train respectively.

[0047] The depth image is a grayscale image, where each pixel represents the distance between a point in the scene and the camera.

[0048] The RGB image and the depth image are obtained by shooting with a camera that supports shooting RGB images and depth images.

[0049] When the camera shoots the RGB image and the depth image, alignment processing is performed.

[0050] When shooting the train image, a binocular camera that can shoot RGB images and depth images simultaneously is used.

[0051] In practical applications, an MV-DB500S Hikvision RGB-D binocular intelligent stereo camera is used for shooting to obtain the template RGB image and the RGB image to be detected. For example: Figure 2 shown is the template RGB image of a certain detection location of the train shot by the camera, Figure 3 shown is the RGB image to be detected.

[0052] An MV-DB500S Hikvision RGB-D binocular intelligent stereo camera is used for shooting to obtain the depth images of the template image and the image to be detected. This camera has a function of setting the alignment of the RGB image and the depth image. When shooting, this setting is turned on, and the obtained RGB image and depth image are the aligned ones.

[0053] In practical applications, due to factors such as changes in lighting or camera position, the template image and the image to be detected cannot be completely aligned, or lighting is misjudged as a defect point. Therefore, using depth maps for dual recognition and verification to improve the detection accuracy can achieve better detection results.

[0054] S2: Preprocess the template RGB image and the RGB image to be detected to obtain the template grayscale image and the grayscale image to be detected.

[0055] The preprocessing of the template RGB image and the RGB image to be detected includes the following steps: converting the RGB image to a grayscale image; performing median filtering on the grayscale image; performing non - restricted adaptive histogram equalization on the grayscale image.

[0056] For operations such as feature matching of images, it is necessary to convert the RGB image to a grayscale image before further processing.

[0057] Median filtering is a non - linear signal processing technique based on the theory of sorting statistics that can effectively suppress noise. The basic principle of median filtering is to replace the value of a point in a digital image or digital sequence with the median of the values of the points in a neighborhood of that point, making the surrounding pixel values closer to the true values and thus eliminating isolated noise points.

[0058] Non - restricted adaptive histogram equalization enhances the contrast of an image by calculating the local histogram of the image and redistributing the brightness. Compared with the traditional histogram equalization method, adaptive histogram equalization is localized and equalizes according to the local regions of the image, thus avoiding the problems of over - enhancement or distortion that may be caused by global equalization.

[0059] The principle of non - restricted adaptive histogram equalization is as follows: The image is divided into many small regions or templates, and then histogram equalization is performed on the pixels within each template. This process is adaptive because it adjusts the equalization effect according to the local characteristics of the image. For example: if a certain region of the image is darker, then the brightness of this region will be increased to enhance the contrast and visual effect of the image; on the contrary, if a certain region of the image is brighter, then the brightness of this region will be reduced to prevent over - exposure or distortion of the image.

[0060] Image preprocessing can enhance the contrast of the image and reduce the impact of defect points in the image on the image.

[0061] S3: Perform feature matching on the template grayscale image and the grayscale image to be detected to obtain the homography matrix.

[0062] The steps of performing feature matching on the template grayscale image and the grayscale image to be detected in step S3 to obtain the homography matrix include the following steps:

[0063] S31: Use the ORB feature detector to perform feature detection on the image and obtain feature points.

[0064] The ORB (Oriented FAST and Rotated BRIEF) feature detector combines FAST (Features from Accelerated Segment Test) key point detection and BRIEF (Binary Robust Independent Elementary Features) descriptor, and at the same time adds the characteristics of rotation invariance and scale invariance.

[0065] The main principle of the ORB feature detector can be divided into the following three steps:

[0066] (1) Extract feature points: ORB feature points are the places where the boundaries or gray levels change significantly. The definition of ORB feature points is the points that are different from the surrounding points.

[0067] (2) Calculate descriptors: Once the feature points are detected, the ORB algorithm will calculate a descriptor for each feature point. ORB uses the BRIEF descriptor, which generates a binary string by comparing pixel pairs around the feature point. At the same time, ORB improves BRIEF to make it rotation invariant.

[0068] (3) Match feature points: After obtaining the descriptors of the feature points, by comparing the descriptors of the feature points in different images, the mutually matching feature points are found.

[0069] S32: Use a brute-force matcher to match the feature points and obtain matching points.

[0070] The distance type used by the brute-force matcher is the Hamming distance.

[0071] The brute-force matcher based on the Hamming distance is mainly used to handle the matching problems between binary feature descriptors (such as ORB, BRIEF, etc.). The Hamming distance is particularly useful in this scenario because it measures the number of different bits between two equal-length binary strings, which can directly reflect the differences between binary feature descriptors.

[0072] The working principle of the brute-force matcher is: (1) For each feature descriptor in the first image, the Hamming distance between it and all feature descriptors in the second image is calculated; (2) The descriptor pair with the smallest Hamming distance is selected as the matching item.

[0073] The brute-force matcher does not require any training or learning process, so it can be conveniently used in various application scenarios. In addition, since the brute-force matcher directly calculates the similarity between feature descriptors, it has good robustness to factors such as illumination changes, rotation, and scale changes. Due to the above advantages of the brute-force matcher, when it is used for train defect detection, it can, to a certain extent, eliminate the influence of a series of environmental factors such as illumination and camera position transformation on the detection.

[0074] S33: Use cross-validation and the RANSAC algorithm to remove incorrect matching points.

[0075] Cross-validation is used to perform cross-validation on the matching points: For each pair of matching points, one of the points is used as the query point, and its best matching point is searched for in the other image. If the best matching point found is the same as or very close to the original matching point, then this pair of matching points is considered reliable; otherwise, it is regarded as an incorrect matching point and removed. Swap the query and search images, transform the query point and the search matching point, and perform cross-validation.

[0076] The RANSAC algorithm can be used to remove incorrect matching points. The principle of the RANSAC algorithm is as follows: (1) Randomly select several pairs of matching points as sample points, estimate a transformation model using these sample points, and then calculate the distances from all matching points to this model; (2) Matching points with distances less than a certain threshold are considered inliers (i.e., correct matching points), while those with distances greater than the threshold are outliers (i.e., incorrect matching points); (3) Through multiple iterations, select the model with the largest number of inliers as the optimal model, and regard the matching points inconsistent with this model as incorrect matching points and remove them.

[0077] For example: After performing image preprocessing on the template RGB image of Figure 2 and the RGB image to be detected of Figure 3 feature matching is performed to obtain matching points, and the corresponding results of the matching points on the original RGB image are as shown in Figure 4 .

[0078] S34: Obtain the homography matrix according to the matching points.

[0079] The homography matrix is a 3x3 matrix that describes the projective transformation from one plane to another plane, including rotation, translation, scaling, and skew, etc. For example: The homography matrix is H in formula (1), and the points (x1, y1) on the first image and the points (x2, y2) on the second image have the mapping relationship of formula (2).

[0080]

[0081]

[0082] When calculating the homography matrix, the least squares method is usually used to solve a system of linear equations. Specifically, assume there are N pairs of matching points, and each pair of matching points consists of a source point coordinate (x_s, y_s) and a target point coordinate (x_t, y_t); two equations are constructed for each pair of matching points, and these equations describe the linear relationship between the source point coordinate and the target point coordinate; these equations are combined into a system of linear equations, and the least squares method is used to solve this system of equations to obtain the elements of the homography matrix.

[0083] S4: Compare the template grayscale image and the template depth image, and use the homography matrix to perform an affine transformation on the grayscale image to be detected and the depth image to be detected, obtaining the transformed grayscale image to be detected and the depth image to be detected.

[0084] The specific implementation manner of step S4 is as follows:

[0085] (1) First, convert the homography matrix into an affine transformation matrix: Since the homography matrix contains the information of perspective transformation, and affine transformation does not include perspective transformation, it is necessary to process the homography matrix to approximately implement affine transformation. Set the third row of the homography matrix to [0 0 1], thereby obtaining an approximate affine transformation matrix.

[0086] (2) Perform affine transformation: Perform matrix multiplication on the affine transformation matrix and the point coordinates in the source image, thereby mapping the points in one image to another image. Perform affine transformation on the template grayscale image, the grayscale image to be detected, the template depth image, and the depth image to be detected respectively, obtaining the transformed template grayscale image, the grayscale image to be detected, the template depth image, and the depth image to be detected.

[0087] S5: Perform defect detection on the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected, obtaining the grayscale defect position map and the depth defect position map.

[0088] The defect detection of the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected, obtaining the grayscale defect position map and the depth defect position map is as follows:

[0089] Perform subtraction, binarization, and erosion processing on the template grayscale image and the transformed grayscale image to be detected, obtaining the grayscale defect position map;

[0090] Perform subtraction, binarization, and erosion processing on the template depth image and the transformed depth image to be detected, obtaining the depth defect position map.

[0091] The principle of subtracting two images is: Subtract the corresponding pixel values of the two images to generate a new difference image. Subtraction is usually used to compare the differences between two images, especially for images at different times or under different conditions.

[0092] The principle of image binarization is as follows: convert a grayscale image into a binary image, that is, each pixel of the image has only two possible values, 0 and 255 (representing black and white respectively). During the binarization process, a suitable threshold needs to be selected. For all pixel points whose grayscale values are greater than or equal to the threshold, the point in the binary image is set to 255 (white); while for pixel points whose grayscale values are less than the threshold, the point in the binary image is set to 0 (black).

[0093] The principle of image erosion processing is as follows: (1) For a given image and a structuring element (a small-sized shape template), place the structuring element at a certain pixel position in the image; (2) If all the pixels within the structuring element match the pixels in the image (usually the pixel values are equal or meet a certain condition), then the pixel remains unchanged; otherwise, the pixel is set to the background value (usually 0 or black); (3) Perform the above operations on each pixel in the image in turn until the entire image is traversed. Erosion processing can eliminate small noise points and small objects in the image, or disconnect connected objects.

[0094] When performing erosion processing, the erosion thresholds for the grayscale image and the depth image are inconsistent, and need to be processed according to the actual situation.

[0095] For example: After processing the images of the train detection parts shown in Figure 2 and 3 through the above steps, finally, a grayscale defect position map as shown in Figure 5 can be obtained. The position of the white part in Figure 5 is the defect point.

[0096] S6: Compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train.

[0097] The so-called true defect points are: the positions where the pixel values on both the grayscale defect position map and the depth defect position map are 255. The positions with pixel value 255 are the white parts on the map, as shown in Figure 5 .

[0098] The beneficial effects of this embodiment are as follows: By performing double recognition and verification techniques on the RGB images and depth images of the template image and the image to be detected respectively, the detection accuracy can be improved, and a better detection effect can be achieved. Double recognition and verification can, to a certain extent, eliminate the influence of factors such as illumination and camera position transformation, and the limitations brought by the insufficient accuracy of the depth image. Therefore, compared with traditional algorithms, the detection effect of this method has been greatly improved.

[0099] Embodiment 2

[0100] A train defect detection system, based on the train defect detection method described in Embodiment 1, includes:

[0101] An image acquisition module that respectively acquires a template RGB image, a to-be-detected RGB image, a template depth image, and a to-be-detected depth image of a train;

[0102] An image preprocessing module that preprocesses the template RGB image and the to-be-detected RGB image to obtain a template grayscale image and a to-be-detected grayscale image;

[0103] A feature matching module that performs feature matching on the template grayscale image and the to-be-detected grayscale image to obtain a homography matrix;

[0104] An affine transformation module that is used to compare the template grayscale image and the template depth image, and uses the homography matrix to perform affine transformation on the to-be-detected grayscale image and the to-be-detected depth image to obtain the transformed to-be-detected grayscale image and the to-be-detected depth image;

[0105] A defect detection module that is used to perform defect detection on the template grayscale image, the template depth image, the transformed to-be-detected grayscale image, and the to-be-detected depth image to obtain a grayscale defect position map and a depth defect position map;

[0106] A defect confirmation module that is used to compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train.

[0107] Embodiment III

[0108] A computer program, when the computer program is executed, it implements the steps of the train defect detection method as described in Embodiment I.

[0109] Embodiment IV

[0110] A readable storage medium, on which the computer program as described in Embodiment III is stored, and when the computer program is executed, it implements the steps of the train defect detection method as described in Embodiment I.

[0111] As mentioned above, it is only the specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for detecting train defects, characterized in that, It includes the following steps: Obtain the template RGB image, the RGB image to be detected, the template depth image, and the depth image to be detected of the train respectively; Preprocess the template RGB image and the RGB image to be detected to obtain the template grayscale image and the grayscale image to be detected; Perform feature matching on the template grayscale image and the grayscale image to be detected to obtain a homography matrix; Compare the template grayscale image and the template depth image, and use the homography matrix to perform affine transformation on the grayscale image to be detected and the depth image to be detected to obtain the transformed grayscale image to be detected and the depth image to be detected; Perform defect detection on the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected to obtain the grayscale defect position map and the depth defect position map; Compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train; Among them, the step of performing defect detection on the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected to obtain the grayscale defect position map and the depth defect position map is as follows: Perform subtraction, binarization, and erosion processing on the template grayscale image and the transformed grayscale image to be detected to obtain the grayscale defect position map; Perform subtraction, binarization, and erosion processing on the template depth image and the transformed depth image to be detected to obtain the depth defect position map.

2. The train defect detection method according to claim 1, characterized in that, The RGB image and the depth image are obtained by a camera that supports shooting RGB images and depth images.

3. The train defect detection method according to claim 2, wherein When the camera shoots the RGB image and the depth image, alignment processing is performed.

4. The train defect detection method according to claim 1, characterized in that, The preprocessing of the template RGB image and the RGB image to be detected includes the following steps: convert the RGB image to a grayscale image; perform median filtering on the grayscale image; perform unrestricted adaptive histogram equalization on the grayscale image.

5. The train defect detection method according to claim 1, characterized in that, The step of performing feature matching on the template grayscale image and the grayscale image to be detected to obtain a homography matrix includes the following steps: Perform feature detection on the image using an ORB feature detector to obtain feature points; Match the feature points with a brute-force matcher to obtain matching points; Use cross-validation and the RANSAC algorithm to remove incorrect matching points; Obtain the homography matrix according to the matching points.

6. The train defect detection method according to claim 5, wherein, The distance type used by the brute-force matcher is the Hamming distance.

7. The train defect detection method according to claim 1, wherein The true defect points are: the positions where the pixel values on the grayscale defect position map and the depth defect position map are both 255.

8. A train defect detection system, characterized in that, Based on the train defect detection method according to any one of claims 1 to 7, it includes: An image acquisition module that respectively obtains the template RGB image, the RGB image to be detected, the template depth image, and the depth image to be detected of the train; An image preprocessing module that preprocesses the template RGB image and the RGB image to be detected to obtain the template grayscale image and the grayscale image to be detected; A feature matching module that performs feature matching on the template grayscale image and the grayscale image to be detected to obtain a homography matrix; An affine transformation module that is used to compare the template grayscale image and the template depth image, and use the homography matrix to perform affine transformation on the grayscale image to be detected and the depth image to be detected to obtain the transformed grayscale image to be detected and the depth image to be detected; A defect detection module that is used to perform defect detection on the template grayscale image, the template depth image, the transformed grayscale image to be detected, and the depth image to be detected to obtain the grayscale defect position map and the depth defect position map; A defect confirmation module is used to compare the grayscale defect position map and the depth defect position map to determine the true defect points of the train.

Images