A method and device for identifying defects on the tread surface of train wheelsets based on image fusion

Abstract

The present invention provides a train wheelset tread defect identification method and device based on image fusion, which can complete the accurate segmentation and extraction of wheel tread defects in complex on-site environments, and is of great significance to realizing intelligent operation and maintenance and ensuring the safety of train operations. The method comprises the following steps: collecting images by a visible light camera and an infrared camera to obtain an image of the train wheelset tread area; calibrating the visible light camera and the infrared camera; obtaining the transformation relationship between the visible light image and the infrared image based on the camera parameters obtained by calibration, and aligning the images taken by the visible light camera and the infrared camera; constructing a fusion model of the visible light image and the infrared image based on a neural network, training the model until the model converges, and inputting the corresponding visible light image and infrared image into the trained fusion model to obtain a fused image; adopting the region growing method to aggregate the pixel points in the fused image according to the similarity of the image grayscale values to obtain an image of the train wheelset tread defect area.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit safety detection, and in particular to a method and device for identifying train wheel tread defects based on image fusion. Background Art

[0002] During high-speed train operation, a good wheel-rail relationship is one of the key factors in ensuring safe and efficient train operation. Accurate detection and control of wheel tread defects directly determines the stability of the train. Therefore, achieving online wheel tread defect detection during high-speed train operation is crucial for ensuring safe train operation.

[0003] Existing detection methods, such as video, radar, and time-of-flight (TOF), are unable to meet the real-time and anti-interference requirements of wheelset tread defect detection. Visual sensors, with their non-contact, high-precision, and high-real-time characteristics, are widely used in dynamic target posture measurement. However, due to the high-speed operation of trains around the clock, outdoor stray light, complex backgrounds, and inclement weather severely restrict their detection accuracy. Image-based wheelset tread defect detection relies on the accurate detection of characteristic information such as scratches and peeling. However, the complex and changing outdoor environment and the strong reflective characteristics of the wheel tread itself make it extremely difficult to accurately extract wheelset tread defect information. Traditional wheelset defect detection methods based on single visible light images are no longer able to accurately extract wheelset tread defect information in complex outdoor environments. Summary of the Invention

[0004] In response to the above problems, the present invention provides a method and device for identifying train wheel tread defects based on image fusion. By collecting homologous image data through visible light and infrared cameras, and adopting a deep learning method to fuse the information of visible light and infrared images, the accurate segmentation and extraction of wheel tread defects in complex on-site environments are completed, which is of great significance for realizing intelligent operation and maintenance and ensuring the safety of train operations.

[0005] The technical solution is as follows: a train wheel set tread defect identification method based on image fusion, characterized by comprising the following steps: collecting images by a visible light camera and an infrared camera to obtain an image of the train wheel set tread area;

[0006] Calibrate visible light cameras and infrared cameras;

[0007] According to the camera parameters of the visible light camera and the infrared camera obtained through calibration, the transformation relationship between the visible light image and the infrared image is obtained, and the images taken by the visible light camera and the infrared camera are aligned;

[0008] A fusion model of visible light images and infrared images is constructed based on a neural network, and the model is trained until the model converges to obtain a trained fusion model. The corresponding visible light image and infrared image are input into the trained fusion model to obtain a fused image;

[0009] The region growing method is used to aggregate and fuse the pixels in the image according to the similarity of the image grayscale values to obtain the image of the defective area of the train wheel set tread.

[0010] Furthermore, the image acquisition by the visible light camera and the infrared camera includes:

[0011] Visible light cameras and infrared cameras are arranged under the rails at the position where the wheels pass. The visible light cameras and infrared cameras are arranged on two parallel optical axes. When the train passes, the wheels are photographed by the visible light cameras and infrared cameras to obtain images of the tread area of the train wheelset.

[0012] Furthermore, the image collection by the visible light camera and the infrared camera also includes: setting a magnetic steel trigger device at the position where the wheel passes under the rail, obtaining an induction signal based on the magnetic steel magnetic lines of force of the magnetic steel trigger device cut by the wheel, and triggering the visible light camera and the infrared camera to take pictures.

[0013] Furthermore, the calibration of the visible light camera and the infrared camera includes:

[0014] A checkerboard target is set up, and infrared LED lights are embedded in the checkerboard target;

[0015] Zhang Zhengyou's checkerboard calibration method is used to calibrate the visible light camera and infrared camera. The visible light camera and infrared camera synchronously image the checkerboard target. Feature points are extracted from the captured photos, and the intrinsic and extrinsic parameters under ideal conditions are calculated. The distortion parameters under actual radial distortion are estimated using the least squares method, and the obtained parameters are optimized using the maximum likelihood method.

[0016] Furthermore, the transformation relationship between the visible light image and the infrared image is obtained by the homography matrix H from the infrared light plane to the visible light image plane. tc To express, according to the homography matrix H from the infrared light plane to the visible light image plane tc , perform image alignment on the visible light image and the infrared image to obtain aligned visible light image and infrared image.

[0017] Furthermore, the homography matrix H from the infrared image plane to the visible light image plane is tc , calculated using the following formula:

[0018]

[0019] Among them, H cis the homography matrix from the visible light image plane to the checkerboard target plane, H t is the homography matrix from the infrared image plane to the checkerboard target; the homography matrix H is the homography matrix from the visible light image plane to the checkerboard target plane c And the homography matrix H from the infrared image plane to the checkerboard target t Obtained through calibration respectively.

[0020] Furthermore, the fusion model includes a sequentially arranged encoder, a spatial attention mechanism layer, a fusion layer, and a decoder.

[0021] The encoder consists of a convolutional layer and a dense block. The convolutional layer includes three 3*3 convolutional filters. The corresponding infrared image and visible light image are input into the encoder, and the rough features of the image are output.

[0022] The dense block includes 5 convolution filters. The size of the convolution filter is 3*3, and the stride of the convolution filter is 1. The dense block is used to extract deep features. The rough features output by the convolution layer are input into the dense block, and the multi-channel deep feature map is output.

[0023] The spatial attention mechanism layer adds weights to the deep feature map output by the encoder;

[0024] The fusion layer fuses the weighted depth feature map to obtain a fused feature map;

[0025] The decoder includes four 3*3 convolution filters, and the decoding converts the fused feature map into a fused image.

[0026] Furthermore, when training the fusion model, the loss function used is the pixel loss function L p And the structural similarity loss function L ssim The weighted expression is:

[0027]

[0028] Where O and I represent the output image and input image respectively, L p Yes O i and I i The Euclidean distance between i ,I i ) represents the structural similarity, λ and β are the pixel loss function L p And the structural similarity loss function L ssim The weight of , Loss represents the loss function.

[0029] Furthermore, a region growing method is used to aggregate pixels in the fused image based on the similarity of the image grayscale values to obtain an image of the defective area of the train wheel set tread, including the following steps:

[0030] Select the pixel whose gray value is not greater than the set threshold as the seed point;

[0031] Search for adjacent pixels with the seed point as the center. If the adjacent pixel satisfies the grayscale value of the pixel less than or equal to the grayscale of the current seed point, then merge the adjacent pixel into the area where the seed point is located, and iteratively search for the adjacent pixels of the newly merged pixel.

[0032] If there is no pixel point that meets the gray value condition among the adjacent pixels or it has exceeded the image area, the search is stopped and the area where the seed point is located at this time is regarded as the train wheel set tread defect area.

[0033] A train wheel set tread defect recognition device based on image fusion, characterized in that it comprises: a processor, a memory and a program;

[0034] The program is stored in the memory, and the processor calls the program stored in the memory to execute the above-mentioned train wheel set tread defect identification method based on image fusion.

[0035] A computer-readable storage medium, characterized in that: the computer-readable storage medium is used to store a program, and the program is used to execute the above-mentioned train wheelset tread defect identification method based on image fusion.

[0036] The present invention's train wheel tread defect identification method based on image fusion establishes a homologous imaging device for visible light and infrared images by providing a visible light camera and an infrared camera. This device can achieve on-site simultaneous acquisition of texture information and thermal imaging information from the train wheel tread image. By aligning the visible light image with the infrared image, effective alignment of multi-source wheel tread image information is achieved. Deep learning methods are used to fuse the visible light and infrared image information to obtain a fused image with multi-dimensional feature information. Defect information on the wheel tread can be obtained by performing threshold segmentation on the processed image. This method fully utilizes the characteristics of optical images that can enrich detail information and infrared images that are highly resistant to outdoor stray light interference to accurately segment and extract wheel tread defects in complex on-site environments. This method can effectively reduce false alarms and missed detections of wheel tread defects caused by factors such as outdoor stray light, complex backgrounds, and strong reflections, greatly improving the accurate detection of wheel tread defects and possessing important practical application value for achieving intelligent operation and maintenance and ensuring train operational safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the steps of a train wheelset tread defect recognition method based on image fusion in an embodiment;

[0038] Figure 2 Schematic diagram of collecting and fusing images in a train wheelset tread defect recognition method based on image fusion in an embodiment;

[0039] Figure 3 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] See Figure 1 The train wheel set tread defect recognition method based on image fusion of the present invention comprises at least the following steps:

[0041] Step 1: Collect images using a visible light camera and an infrared camera to obtain images of the train wheel tread area;

[0042] Step 2: Calibrate the visible light camera and infrared camera;

[0043] Step 3: Based on the intrinsic and extrinsic parameters of the visible light camera and infrared camera obtained through calibration, the transformation relationship between the visible light image and the infrared image is obtained, and the images taken by the visible light camera and the infrared camera are aligned;

[0044] Step 4: Construct a fusion model of visible light images and infrared images based on a neural network, train the model until the model converges, and obtain a trained fusion model. Input the corresponding visible light image and infrared image into the trained fusion model to obtain a fused image.

[0045] Step 5: Use the region growing method to aggregate the pixels in the fused image based on the similarity of the image grayscale values to obtain the image of the defective area of the train wheel set tread.

[0046] In one embodiment of the present invention, in step 1, the following steps are included:

[0047] Visible light cameras and infrared cameras are arranged under the rails at the position where the wheels pass. The visible light cameras and infrared cameras are arranged on two parallel optical axes. When the train passes, the wheels are photographed by the visible light cameras and infrared cameras to obtain images of the tread area of the train wheelset.

[0048] Specifically in this embodiment, a trigger device is also provided, which is used to trigger the visible light camera and the infrared camera to collect images and simultaneously shoot images of the tread area of the train wheelset. In this embodiment, the trigger device is a magnetic steel trigger device provided at the position where the wheel passes under the rail. The magnetic steel trigger device obtains an induction signal based on the magnetic lines of force of the magnetic steel trigger device cut by the wheel, thereby triggering the visible light camera and the infrared camera to shoot.

[0049] In step 1, the visible light camera and infrared camera arranged on two parallel optical axes establish a homologous imaging device for visible light and infrared images, which can realize the on-site synchronous acquisition and measurement of the texture information and thermal imaging information of the train wheel set tread image.

[0050] In step 2 of this embodiment, the visible light camera and the infrared camera are calibrated, including:

[0051] A checkerboard target is set. In this embodiment, the checkerboard target is improved by inlaying infrared LED lights in the checkerboard target. The coordinates of the corner points in the checkerboard target and the infrared LED lights are precisely calibrated and unified to form a unified target.

[0052] In this embodiment, Zhang Zhengyou's checkerboard calibration method is used to calibrate the visible light camera and the infrared camera. The visible light camera and the infrared camera synchronously image a checkerboard target and capture images of the checkerboard target from multiple angles. Feature points are extracted from the captured photos, and intrinsic and extrinsic parameters are calculated under ideal conditions. The distortion parameters under actual radial distortion are estimated using the least squares method, and the obtained parameters are optimized using the maximum likelihood method.

[0053] The imaging process of the checkerboard target is as follows: let the undistorted feature point image coordinates be p = [u, v, 1] T , the unit is pixel, and its normalized coordinate is p n =[x n ,y n ,1] T , in millimeters. q=[x,y,z,1] T is the coordinate of the feature point in the target coordinate system. According to the pinhole imaging model, the target point q and its corresponding image point p can be expressed as:

[0054]

[0055] Among them, ρ is a non-zero coefficient, is the camera internal parameter matrix. u0, v0 are the coordinates of the principal point on the image plane, f x , f y R = [r1 r2 r3] 3×3 , t are the rotation matrix and translation vector between the target coordinate system and the camera coordinate system respectively.

[0056] H is the homography matrix, let p d =[u d ,v d ,1] T is the homogeneous coordinate of the distorted image point, p n =[x n y n ,1] T is the homogeneous coordinate of the normalized image point. The lens distortion model can be expressed as:

[0057] u d =u+(u-u0)(k1r 2+k2r 4 )

[0058] v d =v+(v-v0)(k1r 2 +k2r 4 )

[0059] in, k1, k2 are the radial distortion coefficients of the lens. Before measurement, the internal parameters of the camera must be accurately calibrated, mainly including [f x ,f y ,γ,u0,v0,k1,k2], the above internal parameters are calibrated by a planar checkerboard target. The checkerboard target and a planar target inlaid with infrared light are used simultaneously to calibrate the external parameters of the visible light camera and the infrared camera.

[0060] In step 3 of this embodiment, the transformation relationship between the visible light image and the infrared image is obtained by the homography matrix H from the infrared light plane to the visible light image plane. tc To express, according to the homography matrix H from the infrared light plane to the visible light image plane tc , perform image alignment on the visible light image and the infrared image to obtain the aligned visible light image and infrared image, and the homography matrix H from the infrared image plane to the visible light image plane tc , calculated using the following formula:

[0061]

[0062] Among them, H c is the homography matrix from the visible light image plane to the checkerboard target plane, H t is the homography matrix from the infrared image plane to the checkerboard target; the homography matrix H is the homography matrix from the visible light image plane to the checkerboard target plane c And the homography matrix H from the infrared image plane to the checkerboard target t are obtained through the calibration in step 2 respectively.

[0063] In step 3, according to the homography matrix H from the infrared light plane to the visible light image plane c By aligning the visible light image and the infrared image, the resulting aligned visible light image and infrared image can unify the size of the spatial object in the infrared and visible light images. The infrared image is then mapped to the visible light image, creating an image in which each pixel in the visible light image incorporates information from both the visible light and infrared images. This creates a one-to-one correspondence between the visible light image and the infrared image.

[0064] In step 4 of this embodiment, a fusion model of visible light image and infrared image is constructed based on a neural network. The fusion model includes an encoder, a spatial domain attention mechanism layer, a fusion layer, and a decoder arranged in sequence.

[0065] The encoder consists of a convolutional layer and a dense block. The convolutional layer includes three 3*3 convolutional filters. The corresponding infrared image and visible light image are input into the encoder, and the rough features of the image are output.

[0066] The dense block includes 5 convolution filters. The size of the convolution filter is 3*3, and the stride of the convolution filter is 1. The dense block is used to extract deep features. The rough features output by the convolution layer are input into the dense block, and the multi-channel deep feature map is output.

[0067] The spatial attention mechanism layer adds weights to the deep feature map output by the encoder;

[0068] The fusion layer fuses the weighted depth feature map to obtain a fused feature map;

[0069] The decoder includes four 3*3 convolution filters, and the decoding converts the fused feature map into a fused image.

[0070] In this embodiment, the encoding layer of the constructed fusion model is a deep learning network structure combining convolutional layers and dense blocks. At the same time, the spatial domain attention mechanism layer is added to add weights. The data output of each layer is connected, and the fused image is decoded and reconstructed through weighted fusion of the fusion layer.

[0071] When training the fusion model, the loss function used is the pixel loss function L p And the structural similarity loss function L ssim The weighted expression is:

[0072]

[0073] Where O and I represent the output image and input image respectively, L p Yes O i and I i The Euclidean distance between i ,I i ) represents the structural similarity, λ and β are the pixel loss function L p And the structural similarity loss function L ssim The weight of , Loss represents the loss function, and the loss function is optimized until the model converges to obtain a trained fusion model.

[0074] After the training is completed, the one-to-one corresponding visible light image and infrared image are input into the fusion model, and the final fused image is output.

[0075] In step 5, the region growing method is used to aggregate pixels based on the similarity of image grayscale values to obtain an image of the train wheel tread defect area. Region growing starts with a set of seed pixels representing different growth regions. Next, pixels that meet the conditions in the neighborhood of the seed pixels are merged into the growth region represented by the seed pixels. The newly added pixels are used as new seed pixels and the merging process is continued until no new pixels that meet the conditions are found. The process includes the following steps:

[0076] Select the pixel whose gray value is not greater than the set threshold as the seed point;

[0077] Search for adjacent pixels with the seed point as the center. If the adjacent pixel satisfies the grayscale value of the pixel less than or equal to the grayscale of the current seed point, then merge the adjacent pixel into the area where the seed point is located. Iteratively search for the adjacent pixels of the newly merged pixel. Starting from the seed point, those neighboring pixels with similar grayscale values to the pre-defined seed are added to each seed to form a growth area.

[0078] If there is no pixel point that meets the grayscale value condition among the adjacent pixels or it has exceeded the image area, the search is stopped and the area where the seed point is located at this time is used as the train wheel tread defect area. The wheel tread defect area is specifically characterized by a small grayscale value and connectivity. The wheel tread defect area is easy to segment and extract through the region growing method. The extracted wheel tread defect area can be used for subsequent qualitative and quantitative analysis of defect information to ensure the safety of train operation.

[0079] During the wheel tread defect area identification process, the train wheels have been rubbing against the rails for a long time, and their surfaces have become very shiny. At the same time, the wheel surface morphology is complex, which mainly makes it impossible to ensure optimal full-frame imaging. It is also very easy to form strong reflections, resulting in the inability to observe the wheel image texture details, causing defects to be undetectable, missed detections, and false alarms. This is also the difficulty of wheel tread defect detection at present. At the same time, the current detection environment is mostly outdoor conditions. External factors such as stray light, sunlight, and complex backgrounds directly affect the camera imaging effect, resulting in low-quality wheel tread images and the inability to accurately detect tread defects.

[0080] To this end, the present invention establishes a homologous imaging device for visible light and infrared images by arranging visible light cameras and infrared cameras. This allows for the on-site simultaneous acquisition of texture information and thermal imaging information from train wheel tread images. The visible image captures reflected light, while the infrared image captures thermal radiation. The infrared image can distinguish the target from the background based on the radiation difference. This works well in all-weather and all-day / all-night conditions, unaffected by external stray light or strong reflections. Visible images, consistent with the human visual system, can provide texture details with high spatial resolution and clarity.

[0081] By aligning visible light images with infrared images, effective alignment of multi-source wheel tread image information is achieved. Deep learning methods are used to fuse the visible light and infrared image information to obtain a fused image with multi-dimensional feature information. Threshold segmentation is performed on the processed image to obtain wheel tread defect information. This fusion combines the advantages of thermal radiation information in infrared images and detailed texture information in visible images, fully capturing the rich information of wheel tread images and effectively improving the accuracy of wheel tread defect recognition.

[0082] This method fully utilizes the characteristics of rich detail information of optical images and strong resistance to outdoor stray light interference of infrared images to complete the accurate segmentation and extraction of wheel tread defects in complex on-site environments. It can effectively reduce the false alarms and missed alarms of wheel tread defects caused by factors such as outdoor stray light, complex background and strong reflection, greatly improving the accurate detection of wheel tread defects, and has important practical application value for realizing intelligent operation and maintenance and ensuring the safety of train operations.

[0083] In an embodiment of the present invention, a train wheelset tread defect recognition device based on image fusion is also provided, which specifically includes: a processor, a memory, and a program;

[0084] The program is stored in the memory, and the processor calls the program stored in the memory to execute the train wheelset tread defect recognition method based on image fusion in the above embodiment.

[0085] The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a train wheelset tread defect identification method based on image fusion is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0086] The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The memory is used to store programs, and the processor executes the programs after receiving execution instructions.

[0087] The processor can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0088] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0089] In an embodiment of the present invention, a computer-readable storage medium is further provided. The computer-readable storage medium is used to store a program, and the program is used to execute the above-mentioned train wheelset tread defect identification method based on image fusion.

[0090] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, computer devices, or computer program products. Thus, embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, computer devices, or computer program products according to the embodiments of the present invention. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, such that execution of the instructions by the processor of the computer or other programmable data processing terminal device produces means for implementing the functions specified in the flowcharts and / or block diagrams.

[0092] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in the flowchart.

[0093] The above is a detailed introduction to the application of the train wheelset tread defect identification method based on image fusion, computer device, and computer-readable storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A train wheel tread defect recognition method based on image fusion, characterized in that: The following steps are involved: The images of the train wheelset tread area are obtained by collecting images using a visible light camera and an infrared camera; Calibrate visible light cameras and infrared cameras; According to the camera parameters of the visible light camera and the infrared camera obtained through calibration, the transformation relationship between the visible light image and the infrared image is obtained, and the images taken by the visible light camera and the infrared camera are aligned; A fusion model of visible light images and infrared images is constructed based on a neural network, and the model is trained until the model converges to obtain a trained fusion model. The corresponding visible light image and infrared image are input into the trained fusion model to obtain a fused image; The region growing method is used to aggregate the pixels in the fused image based on the similarity of the image grayscale values to obtain the image of the defective area of the train wheel set tread. The fusion model consists of a sequentially arranged encoder, a spatial attention mechanism layer, a fusion layer, and a decoder. The encoder consists of a convolutional layer and a dense block. The convolutional layer includes three 3*3 convolutional filters. The corresponding infrared image and visible light image are input into the encoder, and the rough features of the image are output. The dense block includes 5 convolution filters. The size of the convolution filter is 3*3, and the stride of the convolution filter is 1. The dense block is used to extract deep features. The rough features output by the convolution layer are input into the dense block, and the multi-channel deep feature map is output. The spatial attention mechanism layer adds weights to the deep feature map output by the encoder; The fusion layer fuses the weighted depth feature map to obtain a fused feature map; The decoder includes four 3*3 convolution filters, and the decoding converts the fused feature map into a fused image.

2. The method for identifying train wheel tread defects based on image fusion according to claim 1, characterized in that: The image acquisition by the visible light camera and the infrared camera includes: A visible light camera and an infrared camera are placed under the rails at the position where the wheels pass. The visible light camera and the infrared camera are placed on two parallel optical axes. When the train passes, the visible light camera and the infrared camera capture the wheels to obtain images of the tread area of the train wheelset. A magnetic trigger device is set under the rail where the wheel passes. The induction signal is obtained when the wheel cuts the magnetic lines of force of the magnetic trigger device, triggering the visible light camera and infrared camera to take pictures.

3. The train wheel tread defect recognition method based on image fusion according to claim 2, characterized in that: The calibration of the visible light camera and the infrared camera includes: A checkerboard target is set up, and infrared LED lights are embedded in the checkerboard target; Zhang Zhengyou's checkerboard calibration method is used to calibrate the visible light camera and infrared camera. The visible light camera and infrared camera synchronously image the checkerboard target. Feature points are extracted from the captured photos, and the intrinsic and extrinsic parameters under ideal conditions are calculated. The distortion parameters under actual radial distortion are estimated using the least squares method, and the obtained parameters are optimized using the maximum likelihood method.

4. The method for identifying train wheel tread defects based on image fusion according to claim 1, characterized in that: The transformation relationship between the visible light image and the infrared image is obtained by the homography matrix H from the infrared light plane to the visible light image plane. tc To express, according to the homography matrix H from the infrared light plane to the visible light image plane tc , perform image alignment on the visible light image and the infrared image to obtain aligned visible light image and infrared image.

5. The method for identifying train wheel tread defects based on image fusion according to claim 4, characterized in that: The homography matrix H from the infrared image plane to the visible light image plane tc , calculated using the following formula: Among them, H c is the homography matrix from the visible light image plane to the checkerboard target plane, H t is the homography matrix from the infrared image plane to the checkerboard target; the homography matrix H is the homography matrix from the visible light image plane to the checkerboard target plane c And the homography matrix H from the infrared image plane to the checkerboard target t Obtained through calibration respectively.

6. The method for identifying train wheel tread defects based on image fusion according to claim 5, characterized in that: When training the fusion model, the loss function used is the pixel loss function L p And the structural similarity loss function L ssim The weighted expression is: Where O and I represent the output image and input image respectively, L p Yes O i and I i The Euclidean distance between i ,I i ) represents the structural similarity, λ and β are the structural similarity loss function L ssim And pixel loss function L p The weight of , Loss represents the loss function.

7. The method for identifying train wheel tread defects based on image fusion according to claim 6, characterized in that: The region growing method is used to aggregate and fuse the pixels in the image according to the similarity of the image grayscale values to obtain the image of the train wheel set tread defect area, including the following steps: Select the pixel whose gray value is not greater than the set threshold as the seed point; Search for adjacent pixels with the seed point as the center. If the adjacent pixel satisfies the grayscale value of the pixel less than or equal to the grayscale of the current seed point, then merge the adjacent pixel into the area where the seed point is located, and iteratively search for the adjacent pixels of the newly merged pixel. If there is no pixel point that meets the gray value condition among the adjacent pixels or it has exceeded the image area, the search is stopped and the area where the seed point is located at this time is regarded as the train wheel set tread defect area.

8. A train wheel tread defect recognition device based on image fusion, characterized in that: It includes: a processor, a memory and a program; The program is stored in the memory, and the processor calls the program stored in the memory to execute the train wheelset tread defect identification method based on image fusion according to claim 1.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a program, and the program is used to execute the train wheelset tread defect identification method based on image fusion according to claim 1.

Images