Rail joint damage state intelligent identification method and system based on vehicle-mounted CCD camera

Abstract

The application discloses a rail joint damage state intelligent identification method and system based on a vehicle-mounted CCD camera, and comprises the following steps: acquiring rail top surface image data scanned by a vehicle-mounted linear array CCD monochrome camera; establishing a ring buffer zone to store fixed row image data; inputting the ring buffer zone data into a joint discriminator to determine whether the data contains a rail joint and record the mileage of an encoder; inputting the ring buffer zone data containing the rail joint into a rail extractor to extract a rail top surface area; inputting the extracted rail top surface area into a rail joint measurer and a damage diagnostic device respectively to obtain a rail joint measurement value and a joint damage state; inputting the rail joint measurement value and the joint damage state into a marking controller to output whether the rail joint needs to be marked; and writing the diagnostic result into a hard disk, so that the method can diagnose the rail joint state in real time during a track laying machine construction operation, provides guarantee for subsequent operations, and has the advantages of high efficiency and intelligence.

Description

Technical Field

[0001] This invention relates to the field of railway track condition detection technology, specifically to a method and system for intelligent identification of track joint damage conditions based on an onboard CCD camera. Background Technology

[0002] Currently, the most common implementation method for laying tracks in newly built railways in my country is "rail panel group laying + seamless track welding." This involves laying the rail panels after they are nailed together, followed by on-site welding of long rails. The main procedures are: rail panel nailing (25m new rails) → laying rail panels (25m unit rail panel length) with a track laying machine → welding short rails into long rails → stress release and locking. This technology is simple, economical, and can utilize existing rail panel laying equipment, saving a significant amount of tool rails.

[0003] However, due to limitations in process transitions and construction space during the aforementioned construction process, rail welding can only be carried out 1-2 km after the track panels have been laid. This necessitates multiple passes by the track-laying equipment and engineering trains, causing impact vibrations to the unwelded rail ends and easily damaging them. When welding rail joints, if the rail ends are not sawed, the numerous damages will affect the quality of the seamless track; if the rail ends are continuously sawed off, frequent rail misalignment will result in a significant additional workload, impacting efficiency and increasing construction costs. Therefore, there is an urgent need to propose an intelligent identification method and system for track panel joint damage based on an onboard CCD camera, specifically tailored to the operational characteristics of the "track panel group laying + seamless track welding" construction process. This system would intelligently diagnose the damage status of track panel joints during construction, providing a basis for subsequent temporary reinforcement measures. Summary of the Invention

[0004] The technical problem this invention aims to solve is that the damage status of rail panel joints is unknown during the construction process of "rail panel group laying + seamless track welding," making it impossible to determine whether temporary reinforcement is needed, thus increasing construction costs when welding rail joints. The purpose of this invention is to achieve intelligent diagnosis of rail panel joint damage status based on vehicle-mounted CCD cameras and artificial intelligence technology, providing a basis for subsequent temporary reinforcement measures. First, vehicle-mounted linear CCD monochrome cameras continuously capture images of the top surface of the rails. Then, an image measurement system composed of CCD cameras is used to measure the rail gap width. Further, a deep learning model is used to determine the joint damage status. This invention can achieve intelligent diagnosis of rail panel joint damage status during construction, improve the construction process, and enhance the level of intelligence in railway track laying.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides an intelligent identification method for track joint damage status based on an onboard CCD camera, comprising:

[0007] Acquire image data of the top surface of the rail scanned by an onboard linear CCD monochrome camera;

[0008] A circular buffer is established to store scan data from a linear CCD camera with a fixed number of rows. The circular buffer is established in memory (RAM).

[0009] The data from the annular buffer is input into the joint discriminator to determine whether the annular buffer contains a track panel joint image and to record the encoder mileage.

[0010] Copy the annular buffer data containing the rail joint to a new, separate memory area and input it into the rail extractor to extract the image data of the top surface area of ​​the rail.

[0011] The extracted image data of the top surface area of ​​the rail is input into the rail gap measuring device and the damage diagnostic device respectively to obtain the rail gap measurement value and the joint damage status.

[0012] Input the rail gap measurement value and joint damage status into the marking controller, and output whether the rail panel joint needs to be marked;

[0013] The analyzed image data of the top surface area of ​​the rail, encoder mileage, rail gap measurement value, damage diagnosis results and marking results are written to the hard disk for permanent storage.

[0014] Optionally, a circular buffer can be established, specifically including:

[0015] Based on the encoder's vertical resolution and the space window length, determine the number of rows stored in the circular buffer:

[0016] (1)

[0017] In the formula, S y This refers to the encoder's vertical resolution, measured in mm per line. L The length of the spatial window is in meters (m).

[0018] (2)

[0019] In the formula, PPR The number of pulses per revolution of the rotary encoder connected to the axle. D The circumference of the wheel;

[0020] The scanning data from the vehicle-mounted linear CCD monochrome camera is continuously input into the circular buffer. When the joint discriminator outputs that the circular buffer contains a rail panel joint, the data in the current circular buffer is copied to a new address in memory.

[0021] When the joint discriminator outputs an annular buffer that does not contain rail panel joints (i.e., a normal rail top surface image), the data in the current annular buffer is overwritten by the data from the next time step.

[0022] Optional, a connector discriminator, specifically including:

[0023] The joint discriminator is mainly used to determine whether the current circular buffer contains track panel joint data;

[0024] For image data within the circular buffer, along its width direction ( X Perform grayscale projection on the axis and calculate the grayscale value of each row. Y Axis position i The average grayscale value of each pixel is used to generate a one-dimensional vertical grayscale projection curve. P ( i ):

[0025] (3)

[0026] In the formula, I ( i,j ) is a circular buffer in the th i line, number j The pixel grayscale value of the column;

[0027] For projection curves P ( i Using a sliding window average filter, calculate the values ​​prior to the current position. M The average of the points:

[0028] (4)

[0029] In the formula, P baseline This represents the average brightness of a normal rail surface;

[0030] Determine the detection threshold T=s* P baseline ( s (The sensitivity coefficient) is shown in the projection curve. P ( i Search for whether a "joint trough region" exists that simultaneously meets the following conditions:

[0031] The lowest point of the trough P min It must be significantly lower than the detection threshold. T And the width of the trough (i.e. P ( i ) continuously below the threshold T (Number of points) W valley It must be within the preset range;

[0032] If and only if in the projection curve P ( i When a region that simultaneously meets the above conditions is found, the output determines that a track joint exists within the annular buffer and records the row number corresponding to the center position of the trough. y center The spatial mileage of the joint is:

[0033] (5)

[0034] Optional, rail extractor, specifically including:

[0035] The rail extractor is a deep neural network model based on YOLO v8, consisting of a backbone network for feature extraction, a neck network for feature fusion, and a head network for prediction.

[0036] We collected a dataset of linear CCD scan images of the top surface of rails under different conditions (light, oil, ballast, etc.). The dataset includes real rail image samples and pseudo rail image samples synthesized by generative adversarial networks.

[0037] By integrating a Spatial-to-Depth (SPD) module into the YOLO v8 backbone network structure and replacing the original loss function of the YOLOv8 framework with the WIoUv3 loss function, an improved YOLO v8 model is obtained.

[0038] An improved YOLO v8 model was trained on a dataset of rail top surface images to obtain a well-trained rail extractor.

[0039] Optional, rail gap measuring device, specifically including:

[0040] The rail gap measuring instrument is an image measurement system consisting of a CCD camera and an LED light source;

[0041] Calibrate the parameters of a CCD camera-based measurement system, including the camera intrinsic parameter matrix, distortion coefficients, and... X Axis (along the cross-section of the rail) pixel equivalent S x ;

[0042] Using the calibrated camera intrinsic parameter matrix and distortion coefficients, distortion correction is performed on the annular buffer data containing the track joint to eliminate geometric distortion introduced by lens optical characteristics and generate a distortion-free image. I ;

[0043] At the center of the connector y centerWithin the neighborhood of the image, scan column by column along the horizontal axis (X-axis), for each column... x j The longitudinal grayscale distribution is analyzed, and a sub-pixel edge detection algorithm is used to calculate the longitudinal coordinates of the upper and lower edges of the rail seam in this column, thus forming the upper edge point set. E upper ={( x j , y uj )} and the lower edge point set E lower ={( x j , y lj )};

[0044] To eliminate noise artifacts caused by localized reflections or contamination, a random sampling consensus algorithm is used to process the upper and lower edge point sets separately. E upper and E lower By performing a straight line fitting, two robust straight line equations representing the actual rail gap edge are obtained. y u_fit and y = y l_fit ;

[0045] Using the calibrated vertical resolution S y Convert pixel distance to physical width W mm The final calculation formula is:

[0046] (6)

[0047] In the formula, W mm This is the measured value of the rail gap.

[0048] Optional, injury diagnostic equipment, specifically including:

[0049] The network structure of the damage diagnostic tool adopts a multi-task learning architecture with a shared backbone, including a shared feature extraction backbone network composed of convolutional layers, residual layers, pooling layers, etc., used to learn and extract general deep feature representations that can characterize the geometric position and surface damage of the joint from the input rail joint image; and two parallel task-specific prediction heads connected to the output of the feature extraction backbone network. Each prediction head consists of several fully connected layers and classification output layers, which are used to receive shared features and independently perform a three-class classification task of joint geometric position status and a five-class classification task of joint surface damage status.

[0050] The three-category task of determining the geometric position status of the rail joint is based on the top surface image of the rail joint, and the labels are "normal", "misaligned", and "relative". The five-category task of determining the surface damage status of the rail joint is based on the top surface image of the rail joint, and the labels are "normal", "crack", "chipped", "dent", and "corrugation".

[0051] A dataset of linear CCD scan images of the top surface of rail joints under different damage conditions was collected. The dataset includes real rail joint image samples and pseudo-samples of rail joint images generated by 3D modeling and rendering technology. The 3D modeling and rendering technology refers to the creation of a high-fidelity rail joint model in 3D modeling software, the simulation of various damage conditions of the model through parametric methods (including setting different degrees of misalignment height difference, misalignment distance, and simulating the generation of different forms of cracks, chips, dents and corrugated textures), and the realistic rendering of the simulated model.

[0052] The model training process employs an adjustable, combinatorial multi-task loss function. L total The loss function is a weighted sum of the classification losses from the two tasks, designed to optimize the overall learning performance of the model by adjusting the weights. Its calculation formula is:

[0053] (7)

[0054] In the formula, L p It is the cross-entropy loss function used for the "joint geometric position state" three-class classification task. L d It is the cross-entropy loss function used for the five-class classification task of "joint surface damage status". α and β These are preset hyperparameter weights used to balance the importance and learning rate of the two tasks during model training, preventing the model from being overly biased towards one task and neglecting the other.

[0055] Optional, the identifier controller, specifically including:

[0056] Set up rail gap warning signs T f Location warning signs T p and damage warning signs T d ;

[0057] When the rail gap measurement value W mm If the rail gap warning value is exceeded, a rail gap warning indicator will be output. T fIf TRUE is selected, a position warning indicator will be output when the predicted geometric position of the joint is "misaligned" or "intersecting". T p If the condition is TRUE, a damage warning indicator will be output when the predicted surface damage condition of the joint is "crack", "chipping", "dent", or "corrugation". T d If countered, output FALSE;

[0058] The marking controller is a binary automatic control system that receives a rail gap warning marking. T f When the setting is TRUE, the control marking spraying device sprays red dots of paint onto the rail fasteners; when the receiving location warning marking is TRUE... T p When TRUE is selected, the control marking spraying device sprays yellow dot paint onto the rail fasteners; when a damage warning sign is received... T d When TRUE is selected, the marking spraying device is controlled to spray white dot paint onto the rail fasteners.

[0059] Secondly, the present invention provides an intelligent identification system for track joint damage status based on an on-board CCD camera, comprising:

[0060] Image acquisition module: A linear CCD monochrome camera, a strip LED light source, and a spray marking device are installed on each side of the track-laying vehicle's running section. The industrial linear CCD monochrome camera is placed perpendicular to the top surface of the rail and is used to continuously capture images of the top surface of the rail.

[0061] Position encoding module: Rotary encoders are installed on the axles of the wheels of the track-laying vehicle. The rotary encoders are used to control the CCD camera to sample according to spatial distance and thus determine the position. Y Axis (longitudinal) pixel equivalent S y ;

[0062] The servo control module is used to control the spray marking device to spray different colored paint markings onto the rail fasteners based on the system diagnostic results.

[0063] Data storage module: Used to establish a circular buffer for temporary storage of CCD camera scan data and permanent storage of diagnostic results data.

[0064] Data processing module: Used to input data from the circular buffer into the joint discriminator, rail extractor, rail gap measuring device, damage diagnostic device, and marking controller to perform joint status diagnosis.

[0065] Visualization module: Used to visualize surface images and diagnostic results data of rail joints.

[0066] Thirdly, the present invention provides a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the intelligent identification method for track joint damage status based on an on-board CCD camera as described in any one of claims 1-7.

[0067] Fourthly, the present invention provides an electronic device, characterized in that it includes: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the above-mentioned intelligent identification method for track joint damage status based on an on-board CCD camera.

[0068] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention directly utilizes the measurement system established by the vehicle-mounted CCD camera to realize the identification, measurement and diagnosis of track panel joints, avoiding the disadvantages of traditional manual visual inspection such as low efficiency, high manpower requirements and strong subjectivity. In addition, the diagnostic system can be directly deployed on the running gear of the track laying vehicle, thereby realizing the operation mode of diagnosis while construction. The diagnostic results are fed back to the on-site construction personnel in real time to guide the next construction plan, which has the advantages of high efficiency and intelligence.

[0069] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 A flowchart of a method for intelligent identification of track joint damage status based on an on-board CCD camera provided in an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of a circular buffer operation provided in an embodiment of the present invention.

[0073] Figure 3 This is a schematic diagram of an image recognition method for a rail panel joint, provided as an embodiment of the present invention.

[0074] Figure 4 This is a schematic diagram of a rail extractor model architecture provided in an embodiment of the present invention.

[0075] Figure 5This is a schematic diagram illustrating the generation of pseudo-sample data according to an embodiment of the present invention.

[0076] Figure 6 This is a schematic diagram of rail gap width measurement provided in an embodiment of the present invention.

[0077] Figure 7 This is a schematic diagram of a damage diagnostic device model architecture provided in an embodiment of the present invention.

[0078] Figure 8 A flowchart for enhancing joint damage image data is provided in an embodiment of the present invention.

[0079] Figure 9 This is a schematic diagram of a CCD camera installation provided in an embodiment of the present invention.

[0080] Figure 10 This is a schematic diagram of the installation of a marking spraying device provided in an embodiment of the present invention. Detailed Implementation

[0081] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0082] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0083] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0084] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0085] Example 1

[0086] This invention provides a technical solution: an intelligent identification method for track joint damage status based on an onboard CCD camera, such as... Figure 1 As shown;

[0087] Step 1: Acquire the image data of the top surface of the rail. The image data of the top surface of the rail is scanned by a linear CCD monochrome camera installed on the front axle box of the first bogie of the track laying vehicle.

[0088] Step 2: Establish a circular buffer to store scan data from a linear CCD camera with a fixed number of rows. The circular buffer is established in memory (RAM), such as... Figure 2 As shown;

[0089] Step 3: Input the data from the annular buffer into the joint discriminator to determine whether the annular buffer contains a track panel joint image and record the encoder mileage.

[0090] In this example, establishing a circular buffer specifically includes:

[0091] Based on the encoder's vertical resolution and the space window length, determine the number of rows stored in the circular buffer:

[0092] (1)

[0093] In the formula, S y This refers to the encoder's vertical resolution, measured in mm per line. L The length of the spatial window is in meters (m).

[0094] (2)

[0095] In the formula, PPR The number of pulses per revolution of the rotary encoder connected to the axle. D The circumference of the wheel;

[0096] The scanning data from the vehicle-mounted linear CCD monochrome camera is continuously input into the circular buffer. When the joint discriminator outputs that the circular buffer contains a rail panel joint, the data in the current circular buffer is copied to a new address in memory.

[0097] When the joint discriminator outputs an annular buffer that does not contain rail panel joints (i.e., a normal rail top surface image), the data in the current annular buffer is overwritten by the data from the next time step.

[0098] In this example, the joint discriminator is mainly used to determine whether the current circular buffer contains track panel joint data, such as... Figure 3 As shown, the details are as follows:

[0099] For image data within the circular buffer, along its width direction ( X Perform grayscale projection on the axis and calculate the grayscale value of each row. Y Axis position i The average grayscale value of each pixel is used to generate a one-dimensional vertical grayscale projection curve. P ( i ):

[0100] (3)

[0101] In the formula, I ( i,j ) is a circular buffer in the th i line, number j The pixel grayscale value of the column;

[0102] For projection curves P ( i Using a sliding window average filter, calculate the values ​​prior to the current position. M The average of the points:

[0103] (4)

[0104] In the formula, P baseline This represents the average brightness of a normal rail surface;

[0105] Determine the detection threshold T=s* P baseline ( s (The sensitivity coefficient) is shown in the projection curve. P ( i Search for whether a "joint trough region" exists that simultaneously meets the following conditions:

[0106] The lowest point of the trough P min It must be significantly lower than the detection threshold. T And the width of the trough (i.e. P ( i ) continuously below the threshold T (Number of points) W valley It must be within the preset range;

[0107] If and only if in the projection curve P ( iWhen a region that simultaneously meets the above conditions is found, the output determines that a track joint exists within the annular buffer and records the row number corresponding to the center position of the trough. y center The spatial mileage of the joint is:

[0108] (5)

[0109] Step 4: Copy the annular buffer data containing the rail joint to a new, independent memory area and input it into the rail extractor to extract the image data of the top surface area of ​​the rail.

[0110] In this example, the rail extractor is a deep neural network model based on YOLO v8, consisting of a backbone network for feature extraction, a neck network for feature fusion, and a head network for prediction, as follows: Figure 4 As shown;

[0111] Using YOLO v8-n, a Spatial-to-Depth (SPD) module is integrated into the backbone network structure, and the WIoUv3 loss function is used to replace the original loss function of the YOLOv8 framework, as shown in the following formula, to obtain the improved YOLO v8 model.

[0112] (6)

[0113] In the formula, IoU The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. R This is a distance / shape penalty term (such as center point distance, aspect ratio, etc., similar to CIoU / DIoU). The adaptive weights (related to IoU value and sample difficulty) are determined by the following formula: λ and γ These are weight hyperparameters;

[0114] (7)

[0115] In the formula, α, β, μ These are hyperparameters, taken as 2, 4, and 0.5 respectively;

[0116] We collected a dataset of linear CCD scan images of the top surface of rails under different conditions (light, oil, ballast, etc.). The dataset includes real rail image samples and pseudo rail image samples synthesized by generative adversarial networks.

[0117] In this example, the generative adversarial network consists of a generator and a discriminator. The generator generates samples with a similar distribution to the rail image, and the discriminator distinguishes between real and fake samples. Figure 5 As shown;

[0118] The synthesized rail image dataset was manually annotated with anchor boxes and divided into training, validation, and test sets in a 7:2:1 ratio. The improved YOLO v8 model was then trained on the rail top surface image dataset to obtain a trained rail extractor.

[0119] Step 5: Input the extracted image data of the top surface area of ​​the rail into the rail gap measuring device and the damage diagnostic device respectively to obtain the rail gap measurement value and the joint damage status;

[0120] In this example, the rail gap measuring device is an image measurement system consisting of a CCD camera and an LED light source;

[0121] Calibrate the parameters of a CCD camera-based measurement system, including the camera intrinsic parameter matrix, distortion coefficients, and... X Axis (along the cross-section of the rail) pixel equivalent S x ;

[0122] (8)

[0123] In the formula, f x and f y Focal length (in pixels); c x and c y Principal point coordinates (usually the image center);

[0124] (9)

[0125] In the formula, ( u , v Let be the pixel coordinates of a point on the image, and let its corresponding normalized camera coordinates be ( ). x , y ,1);

[0126] (10)

[0127] In the formula, ( x c ,y c , z c ) represents the camera coordinates, ( X w ,Y w , Z w () represents world coordinates;

[0128] (11)

[0129] In the formula, D r To determine the actual distance between two points on the calibration board, D p The corresponding pixel distance on the image;

[0130] Using the calibrated camera intrinsic parameter matrix and distortion coefficients, distortion correction is performed on the annular buffer data containing the track joint to eliminate geometric distortion introduced by lens optical characteristics and generate a distortion-free image. I ;

[0131] At the center of the connector y center Within the neighborhood of the image, scan column by column along the horizontal axis (X-axis), for each column... x j The longitudinal grayscale distribution is analyzed, and a sub-pixel edge detection algorithm is used to calculate the longitudinal coordinates of the upper and lower edges of the rail seam in this column, thus forming the upper edge point set. E upper ={( x j , y uj )} and the lower edge point set E lower ={( x j , y lj )},like Figure 6 As shown;

[0132] To eliminate noise artifacts caused by localized reflections or contamination, a random sampling consensus algorithm is used to process the upper and lower edge point sets separately. E upper and E lower By performing a straight line fitting, two robust straight line equations representing the actual rail gap edge are obtained. y u_fit and y = y l_fit ;

[0133] Using the calibrated vertical resolution S y Convert pixel distance to physical width W mm The final calculation formula is:

[0134] (12)

[0135] In the formula, W mm This is the measured value of the rail gap.

[0136] In this example, the network structure of the damage diagnostic tool adopts a multi-task learning architecture with a shared backbone, such as... Figure 7 As shown, it includes a shared feature extraction backbone network composed of convolutional layers, residual layers, pooling layers, etc., used to learn and extract general deep feature representations that can characterize the geometric position and surface damage of the joint from the input rail joint image; and two parallel task-specific prediction heads connected to the output of the feature extraction backbone network. Each prediction head consists of several fully connected layers and a classification output layer, which are used to receive shared features and independently perform a three-class classification task of joint geometric position state and a five-class classification task of joint surface damage state, respectively.

[0137] The three-category task of determining the geometric position status of the rail joint is based on the top surface image of the rail joint, and the labels are "normal", "misaligned", and "relative". The five-category task of determining the surface damage status of the rail joint is based on the top surface image of the rail joint, and the labels are "normal", "crack", "chipped", "dent", and "corrugation".

[0138] A dataset of linear CCD scan images of the top surface of rail joints under different damage conditions was collected. The dataset includes real rail joint image samples and pseudo-samples of rail joint images generated by 3D modeling and rendering techniques. The 3D modeling and rendering techniques refer to the creation of high-fidelity rail joint models in 3D modeling software, such as... Figure 8 As shown, various damage states of the model are simulated using parametric methods (including setting different degrees of malocclusion height difference, misalignment distance, and simulating the generation of different shapes of cracks, chips, depressions and ripple textures), and the simulated model is rendered realistically.

[0139] The model training process employs an adjustable, combinatorial multi-task loss function. L total The loss function is a weighted sum of the classification losses from the two tasks, designed to optimize the overall learning performance of the model by adjusting the weights. Its calculation formula is:

[0140] (13)

[0141] In the formula, L p It is the cross-entropy loss function used for the "joint geometric position state" three-class classification task. L d It is the cross-entropy loss function used for the five-class classification task of "joint surface damage status". α and βThese are preset hyperparameter weights used to balance the importance and learning rate of the two tasks during model training, preventing the model from being overly biased towards one task and neglecting the other.

[0142] Step 6: Input the rail gap measurement value and joint damage status into the marking controller, and output whether the rail panel joint needs to be marked;

[0143] In this example, the controller is identified, specifically including:

[0144] Set up rail gap warning signs T f Location warning signs T p and damage warning signs T d ;

[0145] When the rail gap measurement value W mm If the rail gap warning value is exceeded, a rail gap warning indicator will be output. T f If TRUE is selected, a position warning indicator will be output when the predicted geometric position of the joint is "misaligned" or "intersecting". T p If the condition is TRUE, a damage warning indicator will be output when the predicted surface damage condition of the joint is "crack", "chipping", "dent", or "corrugation". T d If countered, output FALSE;

[0146] The marking controller is a binary automatic control system that receives a rail gap warning marking. T f When the setting is TRUE, the control marking spraying device sprays red dots of paint onto the rail fasteners; when the receiving location warning marking is TRUE... T p When TRUE is selected, the control marking spraying device sprays yellow dot paint onto the rail fasteners; when a damage warning sign is received... T d When TRUE is selected, the marking spraying device is controlled to spray white dot paint onto the rail fasteners.

[0147] Step 7: Write the analyzed image data of the top surface area of ​​the rail, encoder mileage, rail gap measurement value, damage diagnosis result and marking result to the hard disk for permanent storage.

[0148] Example 2

[0149] Based on Embodiment 1, Embodiment 2 provides an intelligent recognition system for track joint damage status based on an onboard CCD camera. This intelligent recognition system for track joint damage status based on an onboard CCD camera corresponds to the aforementioned intelligent recognition method for track joint damage status based on an onboard CCD camera, and specifically includes:

[0150] Image acquisition module: A linear CCD monochrome camera, a strip LED light source, and a spray marking device are installed on each side of the track-laying vehicle's running gear. The industrial linear CCD monochrome camera is placed perpendicular to the top surface of the rail for continuous image capture of the rail's top surface, such as... Figure 9 , Figure 10 As shown;

[0151] Position encoding module: Rotary encoders are installed on the axles of the wheels of the track-laying vehicle. The rotary encoders are used to control the CCD camera to sample according to spatial distance and thus determine the position. Y Axis (longitudinal) pixel equivalent S y ;

[0152] The servo control module is used to control the spray marking device to spray different colored paint markings onto the rail fasteners based on the system diagnostic results.

[0153] Data storage module: Used to establish a circular buffer for temporary storage of CCD camera scan data and permanent storage of diagnostic results data.

[0154] Data processing module: Used to input data from the circular buffer into the joint discriminator, rail extractor, rail gap measuring device, damage diagnostic device, and marking controller to perform joint status diagnosis.

[0155] Visualization module: Used to visualize surface images and diagnostic results data of rail joints.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent identification of track joint damage status based on vehicle-mounted CCD camera, characterized in that, include: Acquire image data of the top surface of the rail scanned by an onboard linear CCD monochrome camera; A circular buffer is established to store scan data from a linear CCD camera with a fixed number of rows. The circular buffer is established in memory. Establishing a circular buffer includes: Based on the encoder's vertical resolution and the space window length, determine the number of rows stored in the circular buffer: ， In the formula, S y This refers to the encoder's vertical resolution, measured in mm per line. L The length of the spatial window is in meters (m). ， In the formula, PPR The number of pulses per revolution of the rotary encoder connected to the axle. D The circumference of the wheel; The scanning data from the vehicle-mounted linear CCD monochrome camera is continuously input into the circular buffer. When the joint discriminator outputs that the circular buffer contains a rail panel joint, the data in the current circular buffer is copied to a new address in memory. When the output of the joint discriminator does not contain a rail panel joint in the annular buffer, the data in the current annular buffer will be overwritten by the data in the next time step. The data from the annular buffer is input into the joint discriminator to determine whether the annular buffer contains a track panel joint image and to record the encoder mileage. Copy the annular buffer data containing the rail joint to a new, separate memory area and input it into the rail extractor to extract the image data of the top surface area of ​​the rail. The extracted image data of the top surface area of ​​the rail is input into the rail gap measuring device and the damage diagnostic device respectively to obtain the rail gap measurement value and the joint damage status. Input the rail gap measurement value and joint damage status into the marking controller, and output whether the rail panel joint needs to be marked; The analyzed image data of the top surface area of ​​the rail, encoder mileage, rail gap measurement value, damage diagnosis results and marking results are written to the hard disk for permanent storage.

2. The intelligent identification method for track joint damage status based on an on-board CCD camera according to claim 1, characterized in that, Connector discriminator, specifically including: The joint discriminator is mainly used to determine whether the current circular buffer contains track panel joint data; For the image data within the circular buffer, grayscale projection is performed along its width, and the average grayscale value of each row of pixels is calculated, thereby generating a one-dimensional vertical grayscale projection curve. P ( i ): ， In the formula, I ( i,j ) is a circular buffer in the th i line, number j The pixel grayscale value of the column; Vertical grayscale projection curve P ( i Using a sliding window average filter, calculate the values ​​prior to the current position. M The average of the points: ， In the formula, P baseline This represents the average brightness of a normal rail surface; Determine the detection threshold T=s* P baseline ,in s The sensitivity coefficient is shown in the vertical grayscale projection curve. P ( i Search for whether there exists a joint trough region that simultaneously meets the following conditions: The lowest point of the trough P min It must be significantly lower than the detection threshold. T And the width of the trough W valley It must be within the preset range; If and only if in the vertical grayscale projection curve P ( i When a region that simultaneously meets the above conditions is found, the output determines that a track joint exists within the annular buffer and records the row number corresponding to the center position of the trough. y center The spatial mileage of the joint is .

3. The intelligent identification method for track joint damage status based on an on-board CCD camera according to claim 1, characterized in that, Rail extractor, specifically including: The rail extractor is a deep neural network model based on YOLO v8, consisting of a backbone network for feature extraction, a neck network for feature fusion, and a head network for prediction. A dataset of linear CCD scanned images of the top surface of rails under different conditions was collected. The dataset includes real rail image samples and pseudo rail image samples synthesized by generative adversarial networks. The Spatial-to-Depth (SPD) module is integrated into the YOLO v8 backbone network structure, and the WIoUv3 loss function is used to replace the original loss function of the YOLOv8 framework, resulting in an improved YOLO v8 model. An improved YOLO v8 model was trained on a dataset of rail top surface images to obtain a well-trained rail extractor.

4. The intelligent identification method for track joint damage status based on an on-board CCD camera according to claim 1, characterized in that, The rail gap measuring instrument specifically includes: The rail gap measuring instrument is an image measurement system consisting of a CCD camera and an LED light source; Calibrate the parameters of a CCD camera-based measurement system, including the camera intrinsic parameter matrix, distortion coefficients, and... X Axis pixel equivalent S x ; Using the calibrated camera intrinsic parameter matrix and distortion coefficients, distortion correction is performed on the annular buffer data containing the track joint to eliminate geometric distortion introduced by lens optical characteristics and generate a distortion-free image. I ; At the center of the connector y center Within the neighborhood of the image, scan column by column horizontally, for each column... x j The longitudinal grayscale distribution is analyzed, and the longitudinal coordinates of the upper and lower edges of the rail seam are calculated using a sub-pixel edge detection algorithm, thus forming the upper edge point set. E upper ={( x j , y uj )} and the lower edge point set E lower ={( x j , y lj )}; To eliminate noise artifacts caused by localized reflections or contamination, a random sampling consensus algorithm is used to process the upper and lower edge point sets separately. E upper and E lower By performing a straight line fitting, two robust straight line equations representing the actual rail gap edge are obtained. y u_fit and y = y l_fit ; Using the calibrated vertical resolution S y Convert pixel distance to physical width W mm The final calculation formula is: ， In the formula, W mm This is the measured value of the rail gap.

5. The intelligent identification method for track joint damage status based on an on-board CCD camera according to claim 1, characterized in that, Damage diagnostic equipment, specifically including: The network structure of the damage diagnostic tool adopts a multi-task learning architecture with a shared backbone, including a shared feature extraction backbone network composed of convolutional layers, residual layers, and pooling layers, which is used to learn and extract general deep feature representations that can characterize the geometric position and surface damage of the joint from the input rail joint image; and two parallel task-specific prediction heads connected to the output of the feature extraction backbone network. Each prediction head consists of several fully connected layers and a classification output layer, which are used to receive shared features and independently perform a three-class classification task of joint geometric position status and a five-class classification task of joint surface damage status. The three-classification task for the geometric position status of the joint is to determine the position status of the joint based on the top surface image of the rail joint, and the labels are normal, misaligned, and relative. The five-classification task for the surface damage status of the joint is to determine the damage status of the joint based on the top surface image of the rail joint, and the labels are normal, crack, chip, dent, and corrugation. A dataset of linear CCD scan images of the top surface of rail joints under different damage conditions was collected. The dataset includes real rail joint image samples and pseudo-samples of rail joint images generated by 3D modeling and rendering technology. 3D modeling and rendering technology refers to building a high-fidelity rail joint model in 3D modeling software, simulating various damage conditions of the model through parametric methods, and performing realistic rendering on the simulated model. The model training process employs an adjustable, combinatorial multi-task loss function. L total The loss function is a weighted sum of the classification losses from the two tasks. The overall learning performance of the model is optimized by adjusting the weights. Its calculation formula is as follows: ， In the formula, L p It is the cross-entropy loss function used for the three-class classification task of joint geometric position state. L d It is the cross-entropy loss function used for a five-class classification task of joint surface damage status. α and β These are preset hyperparameter weights used to balance the importance and learning rate of the two tasks during model training, preventing the model from being overly biased towards one task and neglecting the other.

6. The intelligent identification method for track joint damage status based on an on-board CCD camera according to claim 1, characterized in that, Identification controller, specifically including: Set up rail gap warning signs T f Location warning signs T p and damage warning signs T d ; When the rail gap measurement value W mm If the rail gap warning value is exceeded, a rail gap warning indicator will be output. T f If TRUE is selected, a position warning indicator will be output when the predicted geometric position of the connector is misaligned or relative. T p TRUE indicates a damage warning indicator will be output when the predicted surface damage condition of the joint is crack, chipping, dent, or corrugation. T d Conversely, output FALSE. The marking controller is a binary automatic control system that receives a rail gap warning marking. T f When the setting is TRUE, the control marking spraying device sprays red dots of paint onto the rail fasteners; when the receiving location warning marking is TRUE... T p When TRUE is selected, the control marking spraying device sprays yellow dot paint onto the rail fasteners; when a damage warning sign is received... T d When TRUE is selected, the marking spraying device is controlled to spray white dot paint onto the rail fasteners.

7. An intelligent identification system for track joint damage based on an onboard CCD camera, characterized in that, include: Image acquisition module: A linear CCD monochrome camera, a strip LED light source, and a spray marking device are installed on each side of the track-laying vehicle's running section. The CCD monochrome camera is placed perpendicular to the top surface of the rail and is used to continuously capture images of the top surface of the rail. Position encoding module: Rotary encoders are installed on the axles of the wheels of the track-laying vehicle. The rotary encoders are used to control the CCD camera to sample according to spatial distance and thus determine the position. Y Axis pixel equivalent S y ; The servo control module is used to control the spray marking device to spray different colored paint markings onto the rail fasteners based on the system diagnostic results. Data storage module: used to establish a circular buffer for temporarily storing CCD camera scan data and permanently saving diagnostic results data; the establishment of the circular buffer specifically includes: Based on the encoder's vertical resolution and the space window length, determine the number of rows stored in the circular buffer: ， In the formula, S y This refers to the encoder's vertical resolution, measured in mm per line. L The length of the spatial window is in meters (m). ， In the formula, PPR The number of pulses per revolution of the rotary encoder connected to the axle. D The circumference of the wheel; The scanning data from the vehicle-mounted linear CCD monochrome camera is continuously input into the circular buffer. When the joint discriminator outputs that the circular buffer contains a rail panel joint, the data in the current circular buffer is copied to a new address in memory. When the output of the joint discriminator does not contain a rail panel joint in the annular buffer, the data in the current annular buffer will be overwritten by the data in the next time step. The data from the annular buffer is input into the joint discriminator to determine whether the annular buffer contains a track panel joint image and to record the encoder mileage. Copy the annular buffer data containing the rail joint to a new, separate memory area and input it into the rail extractor to extract the image data of the top surface area of ​​the rail. The extracted image data of the top surface area of ​​the rail is input into the rail gap measuring device and the damage diagnostic device respectively to obtain the rail gap measurement value and the joint damage status. Input the rail gap measurement value and joint damage status into the marking controller, and output whether the rail panel joint needs to be marked; The analyzed image data of the top surface area of ​​the rail, encoder mileage, rail gap measurement value, damage diagnosis result and identification result are written to the hard disk for permanent storage; Data processing module: used to input data from the circular buffer into the joint discriminator, rail extractor, rail gap measuring device, damage diagnostic device, and marking controller to perform joint status diagnosis; Visualization module: Used to visualize surface images and diagnostic results data of rail joints.

8. A non-transitory computer-readable storage medium, characterized in that, A non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the intelligent identification method for track joint damage status based on an on-board CCD camera as described in any one of claims 1-6.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the intelligent identification method for track joint damage status based on an on-board CCD camera as described in any one of claims 1-6.

Images