Rail fastener status detection method and system

Abstract

The application discloses a track fastener state detection method and system, relates to the track detection technical field, and collects three-dimensional depth images and two-dimensional texture images of a track, inputs the three-dimensional depth images to a detection model after preprocessing, and outputs a fastener rectangular region; verifies the output fastener rectangular region by using the two-dimensional texture images, excludes false positive fastener rectangular regions; performs secondary verification on the screened fastener rectangular regions based on prior spacing information between fasteners, performs repair processing if there is a missed fastener rectangular region; performs gray scale stretching operation on the fastener region and an adjacent region thereof, and obtains a local fastener region; extracts the position and depth features of a fastener component by using the local fastener region, corrects the component position by using prior data, and finally extracts the features of the fastener component; compares the extracted component features with standard component features, and outputs the fastener component state. The application can realize high-precision detection of the fastener state on a ballasted track or a non-ballasted track.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of track detection, and in particular to a track fastener state detection method and system. BACKGROUND

[0002] Fasteners are an important part of railway track structures, and their role is to fix the fasteners on the rail to the rail tie. In the long-term use, the fastener part will appear in the bad state or damage, and these abnormal states of the fastener will affect the dynamic response of the track structure, and in severe cases, will lead to train derailment and other safety accidents. Therefore, it is necessary to regularly check the state of the fastener and repair or replace the fastener in an abnormal state in time.

[0003] In recent years, with the rapid development of computer technology and image processing technology, the automatic detection technology of fastener state based on computer vision has become a research focus, especially the track rail clip defect recognition algorithm based on deep learning. Through the detection of the state of the rail clip, the interesting area of the fastener image is segmented from the collected track image, then the rail clip sub-image is extracted, and finally the rail clip detection is performed by taking the rail clip sub-image as the input of the deep convolutional neural network model.

[0004] However, the above method has the following defects: when positioning the fastener rectangle, only the fastener rectangle on the ballastless track can be processed according to the prior information; if there is a slight deviation in the production of the rail, the fastener rectangle will be offset according to the prior information; and when the rail clip does not exist, it will lead to missed detection. Therefore, there is an urgent need for a track fastener state detection method and system to solve the above problems. SUMMARY

[0005] Therefore, the present application provides a track fastener state detection method and system to solve the problems existing in the prior art.

[0006] The first aspect of the present application provides a track fastener state detection method, comprising:

[0007] acquiring a three-dimensional depth image and a two-dimensional texture image of a track to be inspected;

[0008] inputting the three-dimensional depth image after preprocessing into a preset detection model to obtain a plurality of first fastener rectangular regions;

[0009] verifying all the first fastener rectangular regions based on the two-dimensional texture image to screen a plurality of second fastener rectangular regions;

[0010] determine whether there is a missed second fastener rectangular region based on the prior spacing information between fasteners and the determined second fastener rectangular region, if yes, repair the missed second fastener rectangular region, and combine all the determined second fastener rectangular regions to obtain a reference fastener rectangular region, if no, record all the determined second fastener rectangular regions as the reference fastener rectangular region;

[0011] crop the fastener region and the adjacent region of the fastener region of all the reference fastener rectangular regions, and perform a gray stretch operation on the three-dimensional depth image of the corresponding region after the cropping to obtain a local fastener region;

[0012] input the three-dimensional depth image of the local fastener region into a deep learning segmentation model to obtain initial position data and corresponding depth data of the fastener assembly, input the three-dimensional depth image of the local fastener region into a deep learning classification model to obtain a classification result, select corresponding preset prior data based on the classification result, correct the initial position data of the fastener assembly by using the preset prior data, and extract actual feature data of the fastener assembly based on the corrected initial position data and the corresponding depth data;

[0013] construct a standard feature library of the fastener assembly, compare the extracted actual feature data of the fastener assembly with corresponding standard feature data in the standard feature library, and output a fastener state detection result.

[0014] In a possible implementation manner of the first aspect, the preprocessing of the three-dimensional depth image includes filtering and denoising processing and gray stretch.

[0015] In a possible implementation manner of the first aspect, the three-dimensional depth image is collected by using a line structured light camera, and the two-dimensional texture image is collected by using a line array camera.

[0016] In a possible implementation manner of the first aspect, the verification of the first fastener rectangular region based on the two-dimensional texture image includes:

[0017] mapping the first fastener rectangular region into the two-dimensional texture image, verifying whether there is a feature texture of a set component in the first fastener rectangular region by using a shape matching algorithm, if yes, taking no action, and if no, determining that the first fastener rectangular region is a false positive fastener rectangular region and performing filtering processing on the corresponding first fastener rectangular region.

[0018] In a possible implementation manner of the first aspect, determining whether there is a missed second fastener rectangular region based on the prior spacing information between fasteners and the determined second fastener rectangular region includes:

[0019] acquiring all the determined second fastener rectangular regions into a preset set;

[0020] extracting any second fastener rectangular region from the preset set, denoted as a second fastener rectangular verification region;

[0021] based on prior spacing information between fasteners, obtaining an adjacent second fastener rectangular region of the second fastener rectangular verification region;

[0022] matching the adjacent second fastener rectangular region with the second fastener rectangular region in the preset set, if the matching is successful, it is determined that there is no corresponding missed fastener rectangular region, and if the matching fails, it is determined that there is a corresponding missed fastener rectangular region.

[0023] In a possible implementation manner of the first aspect, the method further includes:

[0024] When it is determined that there is a corresponding missed fastener rectangular region, a repair processing is performed on the missed fastener rectangular region, specifically:

[0025] obtaining a three-dimensional depth image of the missed fastener rectangular region after preprocessing and inputting the three-dimensional depth image into the preset detection model, and reducing a detection threshold of the preset detection model to obtain a corresponding first fastener rectangular region, denoted as a first fastener repair region;

[0026] obtaining a two-dimensional texture image of the first fastener repair region and performing a texture enhancement operation, verifying all first fastener repair regions by using the two-dimensional texture image after the texture enhancement operation, and obtaining a corresponding second fastener rectangular region to complete the repair processing on the missed fastener rectangular region.

[0027] In a possible implementation manner of the first aspect, the gray scale stretching operation on the three-dimensional depth image of the cropped corresponding region includes:

[0028] based on the three-dimensional depth image of the cropped corresponding region, extracting a track bottom surface depth value of the cropped corresponding region and setting the track bottom surface depth value as a reference value, and extracting a fastener component depth value in the cropped corresponding region;

[0029] mapping a difference value between the fastener component depth value and the reference value as a gray scale difference value, and enlarging a gray scale interval corresponding to the gray scale difference value through gray scale stretching.

[0030] In a possible implementation manner of the first aspect, the method further includes:

[0031] determining whether the to-be-inspected track is a ballast track, if yes, inputting the three-dimensional depth image of the local fastener region into a deep learning classification model for screening, and inputting the three-dimensional depth image of the local fastener region after screening into a deep learning segmentation model;

[0032] if no, directly inputting the three-dimensional depth image of the local fastener region into the deep learning segmentation model.

[0033] In a possible implementation manner of the first aspect, the correcting the initial position data of the fastener assembly by using preset prior data comprises:

[0034] The preset prior data is fastener assembly arrangement prior data, and if the initial position data of the fastener assembly does not match the corresponding fastener assembly arrangement prior data, the position of the fastener assembly is corrected according to the fastener assembly arrangement prior data after excluding outliers based on the depth values of the corresponding three-dimensional depth image.

[0035] The second aspect of the present application provides a rail fastener state detection system, comprising:

[0036] A collection module is configured to collect three-dimensional depth images and two-dimensional texture images of a rail to be inspected.

[0037] A detection module is configured to input the three-dimensional depth images into a preset detection model after preprocessing to obtain a plurality of first fastener rectangular regions.

[0038] A verification module is configured to verify all the first fastener rectangular regions based on the two-dimensional texture images to screen a plurality of second fastener rectangular regions.

[0039] A determination module is configured to determine whether there is a missed fastener rectangular region based on prior spacing information between fasteners and the determined second fastener rectangular regions, and if so, to repair the missed fastener rectangular region and combine all the determined second fastener rectangular regions to obtain a reference fastener rectangular region, and if not, to record all the determined second fastener rectangular regions as the reference fastener rectangular region.

[0040] A cropping module is configured to crop fastener regions and adjacent regions of the fastener regions from all the reference fastener rectangular regions, and perform a gray scale stretching operation on the three-dimensional depth images of the cropped regions to obtain local fastener regions.

[0041] A feature extraction module is configured to input the three-dimensional depth images of the local fastener regions into a deep learning segmentation model to obtain initial position data and corresponding depth data of fastener assemblies, input the three-dimensional depth images of the local fastener regions into a deep learning classification model to obtain a classification result, select corresponding preset prior data based on the classification result, correct the initial position data of the fastener assembly by using the preset prior data, and extract actual feature data of the fastener assembly based on the corrected initial position data and the corresponding depth data.

[0042] A fastener detection module is configured to construct a standard feature library of fastener assemblies, compare the extracted actual feature data of the fastener assembly with corresponding standard feature data in the standard feature library, and output a fastener state detection result.

[0043] The beneficial effects are that the application provides a track fastener state detection method and system, three-dimensional depth images and two-dimensional texture images of the track are collected, the three-dimensional depth images are preprocessed and input into a target detection model, and a rectangular area containing the fastener is output; the collected two-dimensional texture images are used to verify the output fastener rectangular area, and the false positive fastener rectangular area is excluded; the filtered fastener rectangular area is verified again based on the prior spacing information between the fasteners, if there is a missed fastener rectangular area, repair processing is performed; after excluding the false positive fastener rectangular area and repairing the missed fastener rectangular area, the fastener area and its adjacent area are cropped and a gray stretch operation is performed to obtain a local fastener area; the position and depth features of the fastener assembly are extracted using the local fastener area, and the component position is corrected using prior data, and finally the actual features of the fastener assembly are extracted; the extracted component features are compared with the standard component features, and the fastener assembly state is output. The application can realize high-precision detection of the fastener state on the ballasted track and non-ballasted track, and is less affected by the installation position of the fastener and the elastic strip. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0045] Figure 1 It is a track fastener state detection method flowchart provided by the embodiment of the present application;

[0046] Figure 2 It is a track fastener state detection system composition schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0048] In this application, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0049] Embodiment one

[0050] In the prior art, a deep learning-based rail fastener defect is identified, and the scheme is to segment the fastener image of interest from the collected rail image, then extract the fastener sub-image, and finally take the fastener sub-image as the input of the deep convolutional neural network model to classify and identify the fastener. But there are the following defects: 1) The fastener position located according to prior information cannot adapt to ballast tracks. It mainly relies on the exclusive prior information of non-ballast tracks. The core feature of non-ballast tracks is "track slab prefabricated in factory, regular structure, no ballast interference". The core difference of ballast tracks is "no fixed track slab, loose accumulation of ballast, no uniformity of structure". Therefore, the existing scheme is difficult to effectively detect the state of the fastener assembly of the ballast track; 2) The fastener position determined according to the prior information deviates with the production deviation of the steel rail. The essence is to regard the prior information as an absolute fixed value, ignoring the physical transmission link that the fastener installation reference will deviate and the prior positioning will fail when the steel rail production deviates slightly. When the steel rail production deviates slightly, the state detection result of the fastener assembly of the existing scheme will also be distorted; 3) The existing scheme relies on the necessity of the fastener, does not construct a hierarchical judgment logic from the fastener as a whole to the individual fastener, and lacks exclusive identification of the missing fastener. Therefore, it is also impossible to effectively detect the fastener assembly lacking the fastener.

[0051] Therefore, the present application provides a rail fastener state detection method, as shown in Figure 1 , comprising:

[0052] Collecting the three-dimensional depth image and the two-dimensional texture image of the track to be inspected;

[0053] After preprocessing the three-dimensional depth image, input it into the preset detection model to obtain a plurality of first fastener rectangular regions;

[0054] Based on the two-dimensional texture image, all the first fastener rectangular regions are verified to obtain a plurality of second fastener rectangular regions;

[0055] Based on the prior spacing information between the fasteners and the determined second fastener rectangular region, it is judged whether there is a missed fastener rectangular region, if yes, the missed fastener rectangular region is repaired, and the reference fastener rectangular region is obtained by combining all the determined second fastener rectangular regions, if no, all the determined second fastener rectangular regions are recorded as the reference fastener rectangular region;

[0056] The fastener region and the adjacent region of the fastener region of all the reference fastener rectangular regions are cropped, and the three-dimensional depth image of the cropped corresponding region is subjected to a gray stretching operation to obtain a local fastener region;

[0057] The three-dimensional depth image of the local fastener region is input into a deep learning segmentation model to obtain initial position data and corresponding depth data of the fastener assembly; the three-dimensional depth image of the local fastener region is input into a deep learning classification model to obtain a classification result, and the corresponding preset prior data is selected based on the classification result; the initial position data of the fastener assembly is corrected using the preset prior data, and the actual feature data of the fastener assembly is extracted based on the corrected initial position data and the corresponding depth data;

[0058] A standard feature library about the fastener assembly is constructed, the extracted actual feature data of the fastener assembly is compared with the corresponding standard feature data in the standard feature library, and a fastener state detection result is output.

[0059] Among them, the three-dimensional depth image and the two-dimensional texture image of the track to be inspected are collected, the three-dimensional depth image is collected by a line structure light camera, its core function is to provide the spatial position reference of the fastener and the depth feature of the assembly, and the two-dimensional texture image is collected by a line array camera, its core function is to provide the visual feature details of the fastener assembly.

[0060] Among them, the collected three-dimensional depth image is preprocessed, specifically denoising and gray stretching processing, the three-dimensional depth image has abnormal values (such as local depth value mutation) due to environmental interference (such as light, dust), these noises are eliminated by filtering to avoid subsequent positioning interference, and then the depth gray value of the fastener region is stretched to a larger interval through gray stretching, so that the gray difference between the fastener (such as a spring strip protrusion) and the track plate is more significant, and the visual feature of the fastener is highlighted, providing clear input for the preset detection model.

[0061] The preprocessed three-dimensional depth image is input into a preset detection model, and in this embodiment, the preset detection model adopts a YOLOv8 detection model, and a plurality of first fastener rectangular regions are output. The detection logic of the YOLOv8 detection model is as follows: a large number of track three-dimensional depth images + samples of labeled fastener positions are trained to learn the general visual features of the fasteners. During inference, the model directly divides the grid on the preprocessed three-dimensional image, sets a detection threshold to predict whether there is a fastener in each grid, and then outputs the boundary coordinates of the fastener, that is, the first fastener position. Finally, all grid detection results of a single fastener are integrated to obtain the first fastener rectangular region.

[0062] When the YOLOv8 detection model outputs the first fastener rectangular region, there may be deviations, false positives and missed reports due to three-dimensional noise and insufficient model generalization. Therefore, this embodiment uses a two-dimensional texture image and a prior fastener spacing correction method to solve the false positives and missed reports of the fastener rough positioning.

[0063] The two-dimensional texture image is used to verify all the first fastener rectangular regions, and a plurality of second fastener rectangular regions are screened. The verification logic is as follows: the relative position of the fastener to the rail and the track plate is fixed, and the core components (bolts and elastic strips) have clear texture features. Therefore, all the first fastener rectangular region images of rough positioning are mapped into a two-dimensional texture image, and a shape matching algorithm is used to verify whether there are texture features such as bolts and elastic strips in the region. If there are, it is determined to be a real fastener, that is, a second fastener rectangular region. If not, it is determined to be a false positive fastener rectangular region, which needs to be filtered.

[0064] The prior fastener spacing correction method is used to solve the missed report problem. The processing logic is as follows: the track plate is prefabricated in the factory, and the spacing of the fasteners on the same track is a fixed value, that is, the prior fastener spacing. Based on the second fastener rectangular region that has been determined, the theoretical position of the adjacent fastener is calculated according to the prior fastener spacing, that is, the adjacent second fastener rectangular region, and it is determined whether there is already one in the second fastener rectangular region that has been determined. If there is, all the second fastener rectangular regions are directly taken as the reference fastener rectangular region. If not, it is determined that there is a missed report fastener rectangular region. For the missed report fastener rectangular region, this embodiment uses the method of reducing the detection threshold and enhancing the texture to repair it. Specifically, the three-dimensional depth image of the missed report fastener rectangular region after preprocessing is input into the preset detection model, and the detection threshold of the preset detection model is reduced to obtain the corresponding first fastener rectangular region, which is recorded as the first fastener repair region. The two-dimensional texture image of the first fastener repair region is obtained and a texture enhancement operation is performed. The two-dimensional texture image after the texture enhancement operation is used to verify all the first fastener repair regions to obtain the corresponding second fastener rectangular region. The repaired second fastener rectangular region and all the determined second fastener rectangular regions are recorded as the reference fastener rectangular region.

[0065] In order to improve the detection efficiency and avoid irrelevant background interference component detection, the reference fastener rectangular region is cropped in the embodiment, only the fastener region and its adjacent region (a small amount of surrounding background, such as the region extending outward by a certain number of pixels based on the outline of the fastener region) are retained, and the cropped region is subjected to deep gray scale stretching. The processing logic is as follows: the depth value of the cropped track bottom surface is extracted as a reference value, and the depth value of the fastener component is extracted. The difference between the component depth value and the reference value is mapped to a gray scale difference value, and the gray scale interval corresponding to the gray scale difference value is enlarged through gray scale stretching, so that the gray scale of components such as elastic strips and bolts is significantly higher than that of the track bottom surface (iron pad) background, highlighting the visual features of the components and preparing for subsequent component segmentation.

[0066] In order to improve the detection efficiency and avoid irrelevant background interference component detection, the reference fastener rectangular region is cropped in the embodiment, only the fastener region and its adjacent region (a small amount of surrounding background, such as the region extending outward by a certain number of pixels based on the outline of the fastener region) are retained, and the cropped region is subjected to deep gray scale stretching. The processing logic is as follows: the depth value of the cropped track bottom surface is extracted as a reference value, and the depth value of the fastener component is extracted. The difference between the component depth value and the reference value is mapped to a gray scale difference value, and the gray scale interval corresponding to the gray scale difference value is enlarged through gray scale stretching, so that the gray scale of components such as elastic strips and bolts is significantly higher than that of the track bottom surface (iron pad) background, highlighting the visual features of the components and preparing for subsequent component segmentation.

[0067] Further, in order to improve the accuracy of the state detection of the fastener on the ballast track in the present embodiment, the embodiment also sets a screening model. Before inputting the three-dimensional depth image of the local fastener region into the deep learning segmentation model, the screening model is used to screen the local fastener region. The screening logic of the screening model is as follows: by classifying the local fastener region of the ballast track, it is determined whether the degree of ballast covering is serious. If the covering is serious, the fastener is filtered out and not subjected to subsequent state detection. Otherwise, the fastener is retained and subjected to subsequent detection.

[0068] The extracted actual feature data of the fastener assembly is compared with corresponding standard feature data in the standard feature library, and the comparison logic is as follows: first, a standard feature library is established, and features of a standard assembly without defects are collected in advance, including geometric shape features, size features and depth features; then the extracted actual feature data of the fastener assembly is compared with the standard feature library, defects are judged through differences, including geometric shape feature differences, size feature differences and depth feature differences, and a fastener assembly state detection result is output.

[0069] Further, the embodiment also provides a detection method for part assemblies (different models but similar shapes, such as WJ-8 gauge baffle, different models only differ by 1mm in width, and only geometric shape features cannot be distinguished), specifically, key dimensions of the assembly are directly measured through coordinate data of the three-dimensional image, such as the width of the gauge baffle, matched with a standard model size library, to determine the specific model, and if the measured size is out of the standard range of the corresponding model, it is determined that the model is wrong or there is a size defect fault.

[0070] In some embodiments, the pre-processing of the three-dimensional depth image includes filtering and denoising processing and gray scale stretching of the three-dimensional depth image.

[0071] In some embodiments, the three-dimensional depth image is collected by a line structured light camera, and the two-dimensional texture image is collected by a line array camera.

[0072] In some embodiments, based on the two-dimensional texture image, verifying the first fastener rectangular region includes:

[0073] Mapping the first fastener rectangular region into the two-dimensional texture image, using a shape matching algorithm to verify whether there is a feature texture of a set component in the first fastener rectangular region, if yes, no action is taken, and if no, it is determined as a false positive fastener rectangular region and the corresponding first fastener rectangular region is filtered.

[0074] In some embodiments, based on the prior spacing information between fasteners and the determined second fastener rectangular region, whether there is a missed fastener rectangular region is judged.

[0075] All determined second fastener rectangular regions are obtained and placed in a preset set;

[0076] Any second fastener rectangular region is extracted from the preset set, denoted as a second fastener rectangular verification region;

[0077] Based on the prior spacing information between fasteners, a neighboring second fastener rectangular region of the second fastener rectangular verification region is obtained;

[0078] The second fastener rectangular region in the preset set is matched with the adjacent second fastener rectangular region, if the matching is successful, it is determined that there is no corresponding missed fastener rectangular region, and if the matching fails, it is determined that there is a corresponding missed fastener rectangular region.

[0079] In some embodiments, the method further comprises:

[0080] When it is determined that there is a corresponding missed fastener rectangular region, a repair process is performed on the missed fastener rectangular region, specifically:

[0081] The three-dimensional depth image of the missed fastener rectangular region after preprocessing is input into the preset detection model, and the detection threshold of the preset detection model is reduced to obtain a corresponding first fastener rectangular region, denoted as a first fastener repair region.

[0082] A two-dimensional texture image of the first fastener repair region is obtained and a texture enhancement operation is performed, and the two-dimensional texture image after the texture enhancement operation is used to verify all first fastener repair regions to obtain a corresponding second fastener rectangular region, thereby completing the repair process of the missed fastener rectangular region.

[0083] In some embodiments, the gray scale stretching operation on the three-dimensional depth image of the cropped corresponding region comprises:

[0084] Based on the three-dimensional depth image of the cropped corresponding region, the track bottom depth value of the cropped corresponding region is extracted and set as a reference value, and the fastener component depth value in the cropped corresponding region is extracted.

[0085] The difference between the fastener component depth value and the reference value is mapped to a gray scale difference value, and the gray scale interval corresponding to the gray scale difference value is enlarged through gray scale stretching.

[0086] In some embodiments, the method further comprises:

[0087] It is determined whether the track to be inspected is a ballast track, if yes, the three-dimensional depth image of the local fastener region is first input into a deep learning classification model for screening, and the three-dimensional depth image of the local fastener region after screening is input into a deep learning segmentation model.

[0088] If not, the three-dimensional depth image of the local fastener region is directly input into the deep learning segmentation model.

[0089] In some embodiments, the initial position data of the fastener component is corrected using preset prior data, comprising:

[0090] The preset prior data is fastener component arrangement prior data, if the initial position data of the fastener component does not match the corresponding fastener component arrangement prior data, then based on the depth value of the corresponding three-dimensional depth image after excluding outliers, the position of the fastener component is corrected according to the fastener component arrangement prior data.

[0091] Embodiment two

[0092] Based on the track fastener state detection method provided in Embodiment one of the present application, the corresponding track fastener state detection system provided in Embodiment two of the present application is also provided, as shown in Figure 2 , which comprises:

[0093] The acquisition module is configured to acquire three-dimensional depth images and two-dimensional texture images of the track to be inspected.

[0094] The detection module is configured to input the three-dimensional depth images after preprocessing into a preset detection model to obtain a plurality of first fastener rectangular regions.

[0095] The verification module is configured to verify all the first fastener rectangular regions based on the two-dimensional texture images to screen a plurality of second fastener rectangular regions.

[0096] The determination module is configured to determine whether there is a missed fastener rectangular region based on the prior spacing information between fasteners and the determined second fastener rectangular regions, if yes, repair the missed fastener rectangular region, and combine all the determined second fastener rectangular regions to obtain a reference fastener rectangular region, if no, record all the determined second fastener rectangular regions as the reference fastener rectangular region.

[0097] The cropping module is configured to crop the fastener region and the adjacent region of the fastener region of all the reference fastener rectangular regions, and perform a gray stretch operation on the three-dimensional depth images of the corresponding regions after cropping to obtain a local fastener region.

[0098] The feature extraction module is configured to input the three-dimensional depth images of the local fastener region into a deep learning segmentation model to obtain initial position data and corresponding depth data of the fastener component, input the three-dimensional depth images of the local fastener region into a deep learning classification model to obtain a classification result, select the corresponding preset prior data based on the classification result, correct the initial position data of the fastener component using the preset prior data, and extract actual feature data of the fastener component based on the corrected initial position data and the corresponding depth data.

[0099] The fastener detection module is configured to construct a standard feature library of the fastener component, compare the extracted actual feature data of the fastener component with the corresponding standard feature data in the standard feature library, and output a fastener state detection result.

[0100] The specific principles and execution processes of each unit in the track fastener state detection system disclosed in Embodiment Two of the present application are the same as the track fastener state detection method disclosed in Embodiment One of the present application. For details, refer to the corresponding part of the track fastener state detection method disclosed in Embodiment One of the present application, which will not be described here again.

[0101] Those skilled in the art will further appreciate that the individual steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, and to avoid obscuring the disclosure, various components have been described above generally in terms of their functionality, while realizing that such functionality can be implemented in various ways depending upon the particular application and design constraints. Those skilled in the art will recognize many modifications can be made to the examples described in this disclosure, consistent with the intent and principles of this disclosure. Accordingly, it is submitted that what is desired to be protected by letters patent is set forth and supported in the appended claims as follows.

[0102] Although preferred embodiments of the application have been described herein, those skilled in the art will appreciate that other changes and modifications can be made to the embodiments described without departing from the spirit and scope of the application. Accordingly, the appended claims are intended to encompass all such changes and modifications as falling within the scope of the application.

[0103] Obviously, various modifications and changes can be made to the present application without departing from the spirit and scope of the present application. Accordingly, it is intended that the present application embrace all such modifications and changes as fall within the scope of the appended claims and their equivalents.

Claims

1. A rail fastener condition detection method, characterized by, The method comprises the following steps: Collecting a three-dimensional depth image and a two-dimensional texture image of a track to be inspected; After preprocessing the three-dimensional depth image, input it into a preset detection model to obtain a plurality of first fastener rectangular regions; Based on the two-dimensional texture image, verify all first fastener rectangular regions to obtain a plurality of second fastener rectangular regions through screening; Based on the prior spacing information between fasteners and the determined second fastener rectangular regions, determine whether there is a missed fastener rectangular region, if yes, repair the missed fastener rectangular region, and combine all determined second fastener rectangular regions to obtain a reference fastener rectangular region, if no, record all determined second fastener rectangular regions as the reference fastener rectangular region; Crop the fastener region and the adjacent region of the fastener region of all reference fastener rectangular regions, and perform a gray stretch operation on the three-dimensional depth image of the corresponding region after cropping to obtain a local fastener region; Input the three-dimensional depth image of the local fastener region into a deep learning segmentation model to obtain initial position data and corresponding depth data of the fastener assembly; input the three-dimensional depth image of the local fastener region into a deep learning classification model to obtain a classification result, and select the corresponding preset prior data based on the classification result; Use the preset prior data to correct the initial position data of the fastener assembly, and extract the actual feature data of the fastener assembly based on the corrected initial position data and the corresponding depth data; Construct a standard feature library of the fastener assembly, compare the extracted actual feature data of the fastener assembly with the corresponding standard feature data in the standard feature library, and output a fastener state detection result; Based on the prior spacing information between fasteners and the determined second fastener rectangular regions, determine whether there is a missed fastener rectangular region, which comprises: Put all determined second fastener rectangular regions into a preset set; Extract any second fastener rectangular region from the preset set, and record it as a second fastener rectangular verification region; Based on the prior spacing information between fasteners, obtain the adjacent second fastener rectangular region of the second fastener rectangular verification region; Match the adjacent second fastener rectangular region with the second fastener rectangular region in the preset set, if the matching is successful, it is determined that there is no corresponding missed fastener rectangular region, if the matching fails, it is determined that there is a corresponding missed fastener rectangular region.

2. The track fastener status detection method of claim 1, wherein, The preprocessing of the three-dimensional depth image comprises filtering and denoising processing and gray stretch.

3. The track fastener status detection method of claim 1, wherein The three-dimensional depth image is collected by a line structured light camera, and the two-dimensional texture image is collected by a line array camera.

4. The track fastener status detection method of claim 1, wherein, Based on the two-dimensional texture image, the verification of the first fastener rectangular region comprises: Map the first fastener rectangular region to the two-dimensional texture image, use a shape matching algorithm to verify whether there is a feature texture of a set component in the first fastener rectangular region, if yes, do nothing, if no, determine it as a false positive fastener rectangular region and filter the corresponding first fastener rectangular region.

5. The method of claim 1, wherein, Further comprising: When it is determined that there is a corresponding missed fastener rectangular region, repair the missed fastener rectangular region, specifically: The three-dimensional depth image of the missed report fastener rectangular region after preprocessing is input into the preset detection model, and the detection threshold of the preset detection model is reduced to obtain a corresponding first fastener rectangular region, denoted as a first fastener repair region; A two-dimensional texture image of the first fastener repair region is obtained and a texture enhancement operation is performed, and the two-dimensional texture image after the texture enhancement operation is used to verify all the first fastener repair regions to obtain a corresponding second fastener rectangular region, thereby completing the repair processing of the missed report fastener rectangular region.

6. The track fastener status detection method of claim 1, wherein, The gray scale stretching operation on the three-dimensional depth image of the corresponding region after cropping includes: Based on the three-dimensional depth image of the corresponding region after cropping, the track bottom depth value of the corresponding region after cropping is extracted and set as a reference value, and the fastener component depth value in the corresponding region after cropping is extracted; The difference between the fastener component depth value and the reference value is mapped to a gray scale difference value, and the gray scale interval corresponding to the gray scale difference value is enlarged through gray scale stretching.

7. The method of claim 1, wherein, The method further includes: determining whether the track to be inspected is a ballast track, if so, the three-dimensional depth image of the local fastener region is first input into a deep learning classification model for screening, and the three-dimensional depth image of the local fastener region after screening is input into a deep learning segmentation model; if not, the three-dimensional depth image of the local fastener region is directly input into the deep learning segmentation model.

8. The method of claim 1, wherein, The initial position data of the fastener component is corrected using preset prior data, including: The preset prior data is fastener component arrangement prior data, if the initial position data of the fastener component does not match the corresponding fastener component arrangement prior data, then based on the depth value of the corresponding three-dimensional depth image after excluding abnormal values, the position of the fastener component is corrected according to the fastener component arrangement prior data.

9. A rail fastener condition detection system implemented by a rail fastener condition detection method according to claim 1, characterized by, including: The acquisition module is used to acquire the three-dimensional depth image and the two-dimensional texture image of the track to be inspected; The detection module is used to input the three-dimensional depth image after preprocessing into a preset detection model to obtain a plurality of first fastener rectangular regions; The verification module is used to verify all the first fastener rectangular regions based on the two-dimensional texture image to screen a plurality of second fastener rectangular regions; The determination module is used to determine whether there is a missed report fastener rectangular region based on the prior spacing information between fasteners and the determined second fastener rectangular region, if so, the missed report fastener rectangular region is repaired, and the reference fastener rectangular region is obtained by combining all the determined second fastener rectangular regions, if not, all the determined second fastener rectangular regions are recorded as the reference fastener rectangular region; The cropping module is used to crop the fastener region and the adjacent region of the fastener region of all the reference fastener rectangular regions, and perform gray scale stretching operation on the three-dimensional depth image of the corresponding region after cropping to obtain a local fastener region; The feature extraction module inputs the three-dimensional depth image of the local fastener region into a deep learning segmentation model to obtain the initial position data and the corresponding depth data of the fastener component, and inputs the three-dimensional depth image of the local fastener region into a deep learning classification model to obtain a classification result, and selects the corresponding preset prior data based on the classification result; The initial position data of the fastener assembly is corrected by using the preset prior data, and actual feature data of the fastener assembly is extracted based on the corrected initial position data and corresponding depth data; The fastener detection module is configured to construct a standard feature library about the fastener assembly, compare the extracted actual feature data of the fastener assembly with corresponding standard feature data in the standard feature library, and output a fastener state detection result.

Images