Railway fastener defect detection method and device

Abstract

The invention relates to a railway fastener defect detection method and device. The method comprises the following steps: acquiring a to-be-detected image; preprocessing the to-be-detected image to obtain a preprocessed image; inputting the preprocessed image into a pre-trained railway fastener defect detection model for defect detection to obtain a target railway fastener defect detection result; wherein the railway fastener defect detection model comprises an input layer, three feature extraction layers, a railway fastener defect detection layer and a result determination layer; the input layer is used for inputting the preprocessed image into each feature extraction layer, each feature extraction layer is used for carrying out feature extraction on the preprocessed image to obtain image features, and each railway fastener defect detection layer is used for processing the corresponding image features to obtain candidate railway fastener defect detection results; and the result determination layer is used for determining a target railway fastener defect detection result from the plurality of candidate railway fastener defect detection results, so that the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of railway detection technology, and in particular to a railway fastener defect detection method and device. Background Art

[0002] With the advancement of railway network layout and the rapid development of high-speed rail technology, railway operation, maintenance and inspection are taking on more and more important responsibilities. The safe operation of trains requires the safety of the railway system, and railway fasteners are important components that connect rails and sleepers to ensure track distance, and play an important role in the high-speed railway inspection system. However, due to the large distribution span of railway infrastructure, large environmental changes, and the influence of train vibration, railway fasteners may become loose, broken, or even missing, which will seriously affect the safety of railway operation and even cause railway derailment accidents. Therefore, railway fasteners need to be inspected regularly to ensure the safe operation of the railway.

[0003] At present, the types of fasteners on my country's railway lines vary, and traditional inspection methods mainly rely on visual and manual observation to evaluate the condition of railway fasteners. However, this method is subjective, labor-intensive, time-consuming, and occasionally disruptive and unreliable. At the same time, the large number of high-speed railway fasteners, coupled with their diverse types and complex service environments, increases the challenge of preventing omissions or inaccurate inspections during manual inspections. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a railway fastener defect detection method and device, which solves the technical problems existing in the prior art such as low detection efficiency caused by manual inspection methods.

[0006] (II) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, an embodiment of the present invention provides a railway fastener defect detection method, comprising: step S110, collecting an image to be detected; step S120, preprocessing the image to be detected to obtain a preprocessed image; step S130, inputting the preprocessed image into a pre-trained railway fastener defect detection model for defect detection to obtain a target railway fastener defect detection result; wherein the railway fastener defect detection model comprises an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each feature extraction layer, and a result determination layer connected to all railway fastener defect detection layers; the input layer is used to input the preprocessed image into each feature extraction layer respectively, and each feature extraction layer is used to extract features from the preprocessed image to obtain image features, and each railway fastener defect detection layer is used to process the corresponding image features to obtain candidate railway fastener defect detection results, and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

[0009] In a possible embodiment, the image to be detected is preprocessed to obtain a preprocessed image, including: adjusting the size of the image to be detected to a target size to obtain a size-adjusted image; filtering the size-adjusted image to obtain a filtered image; and sharpening the filtered image to obtain a preprocessed image.

[0010] In a possible embodiment, the three feature extraction layers include, from top to bottom, a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer; wherein the first feature extraction layer includes a first feature extraction sublayer, the first feature extraction sublayer is used to perform feature extraction on the preprocessed image to obtain a first image feature; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to perform feature extraction on the first enhanced image to obtain a second image feature; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and a feature fusion layer connected to both the third feature extraction layer and the fourth feature extraction layer, the third feature extraction layer is used to perform feature extraction on the preprocessed image to obtain a third image feature, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to perform feature extraction on the second enhanced image to obtain a fourth image feature, and the feature fusion layer is used to perform feature fusion on the third image feature and the fourth image feature to obtain an image fusion feature.

[0011] In a possible embodiment, the railway fastener defect detection layer includes a first railway fastener defect detection layer, a second railway fastener defect detection layer and a third railway fastener defect detection layer from top to bottom; wherein the first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include a convolution layer, an upsampling layer and a classification layer, the convolution layer is used to perform convolution processing on the corresponding image features to obtain convolution features, and the upsampling layer is used to perform upsampling processing on the corresponding convolution features to obtain upsampled features, and the classification layer performs classification processing on the corresponding upsampled features to obtain candidate railway fastener defect detection results.

[0012] In a possible embodiment, the result determination layer is used to determine whether the results of multiple candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; if at least two are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

[0013] In a second aspect, an embodiment of the present invention provides a railway fastener defect detection device, comprising: an acquisition module, used to acquire an image to be detected; a preprocessing module, used to preprocess the image to be detected to obtain a preprocessed image; a defect detection module, used to input the preprocessed image into a pre-trained railway fastener defect detection model for defect detection to obtain a target railway fastener defect detection result; wherein the railway fastener defect detection model includes an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each feature extraction layer, and a result determination layer connected to all railway fastener defect detection layers; the input layer is used to input the preprocessed image into each feature extraction layer respectively, and each feature extraction layer is used to extract features from the preprocessed image to obtain image features, and each railway fastener defect detection layer is used to process the corresponding image features to obtain candidate railway fastener defect detection results, and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

[0014] In a possible embodiment, the preprocessing module is specifically used to: adjust the size of the image to be detected to the target size to obtain a resized image; filter the resized image to obtain a filtered image; sharpen the filtered image to obtain a preprocessed image.

[0015] In a possible embodiment, the three feature extraction layers include, from top to bottom, a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer; wherein the first feature extraction layer includes a first feature extraction sublayer, the first feature extraction sublayer is used to perform feature extraction on the preprocessed image to obtain a first image feature; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to perform feature extraction on the first enhanced image to obtain a second image feature; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and a feature fusion layer connected to both the third feature extraction layer and the fourth feature extraction layer, the third feature extraction layer is used to perform feature extraction on the preprocessed image to obtain a third image feature, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to perform feature extraction on the second enhanced image to obtain a fourth image feature, and the feature fusion layer is used to perform feature fusion on the third image feature and the fourth image feature to obtain an image fusion feature.

[0016] In a possible embodiment, the railway fastener defect detection layer includes a first railway fastener defect detection layer, a second railway fastener defect detection layer and a third railway fastener defect detection layer from top to bottom; wherein the first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include a convolution layer, an upsampling layer and a classification layer, the convolution layer is used to perform convolution processing on the corresponding image features to obtain convolution features, and the upsampling layer is used to perform upsampling processing on the corresponding convolution features to obtain upsampled features, and the classification layer performs classification processing on the corresponding upsampled features to obtain candidate railway fastener defect detection results.

[0017] In a possible embodiment, the result determination layer is used to determine whether the results of multiple candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; if at least two are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

[0018] In a third aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to execute the method described in the first aspect or any optional implementation of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the method described in the first aspect or any optional implementation method of the first aspect is performed.

[0020] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0021] (III) Beneficial effects

[0022] The beneficial effects of the present invention are:

[0023] The embodiment of the present application acquires an image to be detected, preprocesses the image to be detected, obtains a preprocessed image, and inputs the preprocessed image into a pre-trained railway fastener defect detection model for defect detection to obtain a target railway fastener defect detection result. Compared with the prior art, it can not only improve the detection efficiency, but also improve the detection accuracy.

[0024] In order to make the above-mentioned objectives, features and advantages to be achieved by the embodiments of the present application more obvious and understandable, the following specifically cites preferred embodiments and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flow chart of a railway fastener defect detection method provided in an embodiment of the present application is shown;

[0027] Figure 2 A schematic diagram of the structure of a railway fastener defect detection model provided in an embodiment of the present application is shown;

[0028] Figure 3 A structural block diagram of a railway fastener defect detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0030] The embodiment of the present application acquires the image to be detected, pre-processes the image to be detected, obtains the pre-processed image, and inputs the pre-processed image into a pre-trained railway fastener defect detection model for defect detection to obtain the target railway fastener defect detection result. Compared with the prior art, it can not only improve the detection efficiency, but also improve the detection accuracy.

[0031] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0032] See also Figure 1 , Figure 1 A flow chart of a railway fastener defect detection method provided by an embodiment of the present application is shown. It should be understood that the railway fastener defect detection method can be executed by a railway fastener defect detection device, and the specific device of the railway fastener defect detection device can be set according to actual needs, and the embodiment of the present application is not limited thereto. For example, the railway fastener defect detection device can be a computer, or a server, etc. Specifically, the railway fastener defect detection method includes:

[0033] Step S110, collecting an image to be detected.

[0034] Specifically, high-speed railways are extensive belt-shaped engineering structures with a large number of different types of fasteners and complex use conditions. Manual inspections during maintenance are inefficient and pose safety risks. To address this problem, a high-speed camera and GPS locator can be installed on the inspection vehicle, so that image data of the railway fasteners to be inspected can be continuously collected while the vehicle is traveling along the track, and the shooting position coordinates of each image can be recorded at the same time when the image is taken, that is, each image to be inspected has a corresponding shooting position coordinate.

[0035] Step S120, preprocessing the image to be detected to obtain a preprocessed image.

[0036] It should be understood that the specific processing process included in the preprocessing can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0037] Optionally, the size of the image to be detected is adjusted to a target size to obtain a resized image; the resized image is filtered to obtain a filtered image; and the filtered image is sharpened to obtain a preprocessed image.

[0038] It should also be understood that the specific size of the target size can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0039] Step S130, inputting the preprocessed image into a pre-trained railway fastener defect detection model to perform defect detection, and obtaining a target railway fastener defect detection result.

[0040] It should be understood that the specific model of the railway fastener defect detection model can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0041] Optionally, see Figure 2 , Figure 2 FIG. 1 is a schematic diagram showing a structure of a railway fastener defect detection model provided in an embodiment of the present application. Figure 2 As shown, the railway fastener defect detection model includes an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each feature extraction layer, and a result determination layer connected to all railway fastener defect detection layers; wherein the input layer is used to input the preprocessed image into each feature extraction layer respectively; each feature extraction layer is used to extract features from the preprocessed image to obtain image features; each railway fastener defect detection layer is used to process corresponding image features to obtain candidate railway fastener defect detection results; and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

[0042] Further, see Figure 2The above three feature extraction layers include a first feature extraction layer, a second feature extraction layer and a third feature extraction layer from top to bottom. Among them, the first feature extraction layer includes a first feature extraction sublayer, and the first feature extraction sublayer is used to perform feature extraction on the preprocessed image to obtain a first image feature; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to perform feature extraction on the first enhanced image to obtain a second image feature; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and a feature fusion layer connected to the third feature extraction layer and the fourth feature extraction layer at the same time, the third feature extraction layer is used to perform feature extraction on the preprocessed image to obtain a third image feature, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to perform feature extraction on the second enhanced image to obtain a fourth image feature, and the feature fusion layer is used to perform feature fusion on the third image feature and the fourth image feature to obtain an image fusion feature.

[0043] It should be understood that the specific means of image enhancement transformation can be set according to actual needs, and the embodiments of the present application are not limited thereto.

[0044] Further, see Figure 2 The above-mentioned railway fastener defect detection layer includes a first railway fastener defect detection layer, a second railway fastener defect detection layer and a third railway fastener defect detection layer from top to bottom. The first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include a convolution layer, an upsampling layer and a classification layer. The convolution layer is used to perform convolution processing on the corresponding image features to obtain convolution features, and the upsampling layer is used to perform upsampling processing on the corresponding convolution features to obtain upsampling features, and the classification layer performs classification processing on the corresponding upsampling features to obtain candidate railway fastener defect detection results.

[0045] For example, the convolution layer in the third railway fastener defect detection layer is used to perform convolution processing on the image fusion features, and the upsampling layer in the third railway fastener defect detection layer is used to perform upsampling processing on the convolution features output by the convolution layer in the third railway fastener defect detection layer, and the classification layer in the third railway fastener defect detection layer is used to perform classification processing on the upsampling features output by the upsampling layer in the third railway fastener defect detection layer, to obtain the corresponding candidate railway fastener defect detection results. The candidate railway fastener defect detection results may be loose or broken.

[0046] Further, see Figure 2 The above result determination layer is used to determine whether the results of multiple candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; if at least two are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

[0047] In addition, when it is determined that the target railway fastener defect detection result is defective, the target shooting position coordinates corresponding to the target railway fastener defect detection result are determined, and the target shooting position coordinates and its target railway fastener defect detection result are output so that the staff can perform rapid maintenance based on the output information.

[0048] Therefore, with the help of the above-mentioned technical scheme, the embodiment of the present application obtains the preprocessed image by collecting the image to be detected and preprocessing the image to be detected, and inputs the preprocessed image into a pre-trained railway fastener defect detection model for defect detection to obtain the target railway fastener defect detection result. Compared with the existing technology, it can not only improve the detection efficiency, but also improve the detection accuracy.

[0049] In addition, for the railway fastener defect detection model of the present application, it can further improve the detection accuracy by identifying the features of the preprocessed image (i.e., the original image), the features of the first enhanced image after the original image is enhanced, and the image fusion features of the original image and the second enhanced image.

[0050] It should be understood that the above-mentioned railway fastener defect detection method is only exemplary, and those skilled in the art can make various modifications based on the above-mentioned method, and the modified scheme also falls within the protection scope of this application.

[0051] See also Figure 3 , Figure 3 The structural block diagram of a railway fastener defect detection device 300 provided in an embodiment of the present application is shown. It should be understood that the railway fastener defect detection device 300 is capable of executing each step in the above method embodiment. The specific functions of the railway fastener defect detection device 300 can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here. The railway fastener defect detection device 300 includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the railway fastener defect detection device 300. Specifically, the railway fastener defect detection device 300 includes:

[0052] The acquisition module 310 is used to acquire the image to be detected;

[0053] A preprocessing module 320 is used to preprocess the image to be detected to obtain a preprocessed image;

[0054] The defect detection module 330 is used to input the preprocessed image into a pre-trained railway fastener defect detection model to perform defect detection and obtain a target railway fastener defect detection result;

[0055] Among them, the railway fastener defect detection model includes an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each feature extraction layer, and a result determination layer connected to all railway fastener defect detection layers; the input layer is used to input the preprocessed image into each feature extraction layer respectively, and each feature extraction layer is used to extract features from the preprocessed image to obtain image features, and each railway fastener defect detection layer is used to process the corresponding image features to obtain candidate railway fastener defect detection results, and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

[0056] In a possible embodiment, the preprocessing module 320 is specifically used to: adjust the size of the image to be detected to the target size to obtain a resized image; filter the resized image to obtain a filtered image; sharpen the filtered image to obtain a preprocessed image.

[0057] In a possible embodiment, the three feature extraction layers include, from top to bottom, a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer;

[0058] Among them, the first feature extraction layer includes a first feature extraction sublayer, and the first feature extraction sublayer is used to extract features from the preprocessed image to obtain first image features; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to extract features from the first enhanced image to obtain second image features; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and a feature fusion layer connected to the third feature extraction layer and the fourth feature extraction layer at the same time, the third feature extraction layer is used to extract features from the preprocessed image to obtain third image features, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to extract features from the second enhanced image to obtain fourth image features, and the feature fusion layer is used to perform feature fusion on the third image features and the fourth image features to obtain image fusion features.

[0059] In a possible embodiment, the railway fastener defect detection layer includes, from top to bottom, a first railway fastener defect detection layer, a second railway fastener defect detection layer, and a third railway fastener defect detection layer;

[0060] Among them, the first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include convolution layers, upsampling layers and classification layers. The convolution layers are used to perform convolution processing on corresponding image features to obtain convolution features, and the upsampling layers are used to perform upsampling processing on corresponding convolution features to obtain upsampled features, and the classification layers perform classification processing on corresponding upsampled features to obtain candidate railway fastener defect detection results.

[0061] In a possible embodiment, the result determination layer is used to determine whether the results of multiple candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; if at least two are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

[0062] Since the device described in the above embodiment of the present invention is a device used to implement the method of the above embodiment of the present invention, based on the method described in the above embodiment of the present invention, a person skilled in the art can understand the specific structure and deformation of the device, so it is not described here. All devices used in the method of the above embodiment of the present invention belong to the scope of protection of the present invention.

[0063] The present application provides a storage medium having a computer program stored thereon. The computer program is executed by a processor to execute the method described in the embodiment.

[0064] The present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.

[0065] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0067] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0068] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0069] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0070] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A railway fastener defect detection method, characterized in that: include: Step S110, collecting an image to be detected; Step S120, preprocessing the image to be detected to obtain a preprocessed image; Step S130, inputting the preprocessed image into a pre-trained railway fastener defect detection model to perform defect detection, and obtaining a target railway fastener defect detection result; Among them, the railway fastener defect detection model includes an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each of the feature extraction layers, and a result determination layer connected to all of the railway fastener defect detection layers; the input layer is used to input the preprocessed image into each of the feature extraction layers respectively, and each of the feature extraction layers is used to extract features from the preprocessed image to obtain image features, and each of the railway fastener defect detection layers is used to process corresponding image features to obtain candidate railway fastener defect detection results, and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

2. The railway fastener defect detection method according to claim 1, characterized in that: The preprocessing of the image to be detected to obtain a preprocessed image includes: Adjusting the size of the image to be detected to a target size to obtain an image after the size adjustment; Performing filtering on the resized image to obtain a filtered image; The filtered image is sharpened to obtain a preprocessed image.

3. The railway fastener defect detection method according to claim 1, characterized in that: The three feature extraction layers include, from top to bottom, a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer; Among them, the first feature extraction layer includes a first feature extraction sublayer, and the first feature extraction sublayer is used to perform feature extraction on the preprocessed image to obtain a first image feature; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to perform feature extraction on the first enhanced image to obtain a second image feature; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and the feature fusion layer connected to both the third feature extraction layer and the fourth feature extraction layer, the third feature extraction layer is used to perform feature extraction on the preprocessed image to obtain a third image feature, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to perform feature extraction on the second enhanced image to obtain a fourth image feature, and the feature fusion layer is used to perform feature fusion on the third image feature and the fourth image feature to obtain an image fusion feature.

4. The railway fastener defect detection method according to claim 3, characterized in that: The railway fastener defect detection layer includes, from top to bottom, a first railway fastener defect detection layer, a second railway fastener defect detection layer and a third railway fastener defect detection layer; Among them, the first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include a convolution layer, an upsampling layer and a classification layer, the convolution layer is used to perform convolution processing on the corresponding image features to obtain convolution features, and the upsampling layer is used to perform upsampling processing on the corresponding convolution features to obtain upsampled features, and the classification layer performs classification processing on the corresponding upsampled features to obtain candidate railway fastener defect detection results.

5. The railway fastener defect detection method according to claim 4, characterized in that: The result determination layer is used to determine whether the results of the plurality of candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; If at least two of them are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

6. A railway fastener defect detection device, characterized in that: include: An acquisition module, used for acquiring images to be detected; A preprocessing module, used for preprocessing the image to be detected to obtain a preprocessed image; A defect detection module, used for inputting the preprocessed image into a pre-trained railway fastener defect detection model to perform defect detection and obtain a target railway fastener defect detection result; Among them, the railway fastener defect detection model includes an input layer, three feature extraction layers connected to the input layer, a railway fastener defect detection layer connected to each of the feature extraction layers, and a result determination layer connected to all of the railway fastener defect detection layers; the input layer is used to input the preprocessed image into each of the feature extraction layers respectively, and each of the feature extraction layers is used to extract features from the preprocessed image to obtain image features, and each of the railway fastener defect detection layers is used to process corresponding image features to obtain candidate railway fastener defect detection results, and the result determination layer is used to determine the target railway fastener defect detection result from multiple candidate railway fastener defect detection results.

7. The railway fastener defect detection device according to claim 6, characterized in that: The preprocessing module is specifically used to: adjust the size of the image to be detected to the target size to obtain a resized image; perform filtering on the resized image to obtain a filtered image; and perform sharpening on the filtered image to obtain a preprocessed image.

8. The railway fastener defect detection device according to claim 6, characterized in that: The three feature extraction layers include, from top to bottom, a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer; Among them, the first feature extraction layer includes a first feature extraction sublayer, and the first feature extraction sublayer is used to perform feature extraction on the preprocessed image to obtain a first image feature; the second feature extraction layer includes a first data enhancement layer and a second feature extraction sublayer connected to the first data enhancement layer, the first data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a first enhanced image, and the second feature extraction sublayer is used to perform feature extraction on the first enhanced image to obtain a second image feature; the third feature extraction layer includes a third feature extraction layer, a second data enhancement layer, a fourth feature extraction layer connected to the second data enhancement layer, and the feature fusion layer connected to both the third feature extraction layer and the fourth feature extraction layer, the third feature extraction layer is used to perform feature extraction on the preprocessed image to obtain a third image feature, and the second data enhancement layer is used to perform data enhancement transformation on the preprocessed image to obtain a second enhanced image, and the fourth feature extraction layer is used to perform feature extraction on the second enhanced image to obtain a fourth image feature, and the feature fusion layer is used to perform feature fusion on the third image feature and the fourth image feature to obtain an image fusion feature.

9. The railway fastener defect detection device according to claim 8, characterized in that: The railway fastener defect detection layer includes, from top to bottom, a first railway fastener defect detection layer, a second railway fastener defect detection layer and a third railway fastener defect detection layer; Among them, the first railway fastener defect detection layer, the second railway fastener defect detection layer and the third railway fastener defect detection layer all include a convolution layer, an upsampling layer and a classification layer, the convolution layer is used to perform convolution processing on the corresponding image features to obtain convolution features, and the upsampling layer is used to perform upsampling processing on the corresponding convolution features to obtain upsampled features, and the classification layer performs classification processing on the corresponding upsampled features to obtain candidate railway fastener defect detection results.

10. The railway fastener defect detection device according to claim 9, characterized in that: The result determination layer is used to determine whether the results of the plurality of candidate railway fastener defect detection results are all the same. If all are the same, any one of the candidate railway fastener defect detection results is used as the target railway fastener defect detection result; If at least two of them are the same, one of the at least two identical candidate railway fastener defect detection results is used as the target railway fastener defect detection result.

Images