A railway cargo inspection intelligent detection method, system, device and medium

Abstract

This invention provides an intelligent inspection method, system, equipment, and medium for railway freight inspection, belonging to the field of railway freight inspection technology. The method includes: acquiring images of the freight cars using a line-scan camera array on both sides and the top of the track; performing spatiotemporal synchronization and adaptive parameter adjustment based on an encoder and a precise clock protocol; performing geometric correction, flat-field correction, motion compensation, and stitching on the images to generate a two-dimensional unfolded image of the freight car; using a deep learning-based freight car segmentation model to segment and number freight car instances, and extracting images of individual freight cars; finally, using a damage detection model integrating severity assessment to identify damage, outputting a structured result including damage category, location, confidence level, and severity score. This invention achieves non-contact, full-coverage, and high-precision intelligent inspection of freight cars, effectively improving inspection efficiency and automation levels.

Description

Technical Field

[0001] This invention belongs to the field of railway freight inspection technology, and more specifically relates to an intelligent inspection method, system, equipment and medium for railway freight inspection. Background Technology

[0002] With the continuous increase in railway freight volume, the daily inspection tasks of freight cars are becoming increasingly heavy. Currently, railway freight inspection in my country still mainly relies on manual visual inspection and manual operation. This method is not only inefficient and has limited inspection coverage, but also highly dependent on the inspector's personal experience and subjective judgment, making it difficult to standardize inspection standards. In addition, manual inspection needs to be carried out when the train is stopped or running at low speed, which is labor-intensive and poses high safety hazards. It is no longer suitable for the development needs of high-efficiency, high-density, and all-weather operation of railway freight, and has become a bottleneck restricting the improvement of freight efficiency and safety levels.

[0003] To improve inspection efficiency, the industry has attempted to introduce vision-based automated inspection technologies. However, existing solutions generally use fixed-position area scan cameras for image acquisition, which has significant technical limitations. First, limited by a fixed viewing angle and a limited field of view, a single camera cannot achieve full-coverage imaging of the surface of a train carriage passing at high speed (especially side walls, tops, bottoms, and other parts), resulting in blind spots. Second, when a train passes at high speed, the area scan camera is prone to image blurring and motion blur due to the mismatch between exposure time and movement speed, severely affecting the accuracy of feature recognition and defect analysis.

[0004] Furthermore, the railway freight yard environment is complex and variable, with day-night lighting differences, strong glare from metal surfaces, rain, snow, fog, haze, and dust pollution posing severe challenges to imaging quality. Existing detection systems generally lack robust optical design and adaptive image processing mechanisms for such complex and harsh conditions, resulting in insufficient system stability and high false alarm and false negative rates. In addition, existing technologies mainly produce two-dimensional images or simple alarm information, and the detection results are unstructured, making it difficult to deeply integrate with back-end management systems such as carriage digital models and maintenance work orders, and failing to directly form a structured data chain that can drive precise operation and maintenance decisions.

[0005] In summary, both traditional manual inspection methods and existing automated inspection technologies based on area scan cameras have significant shortcomings in terms of coverage, high-speed adaptability, environmental robustness, and result availability. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide an intelligent inspection method, system, equipment, and medium for railway freight inspection. By deploying a multi-view line scan camera array and adaptive synchronous acquisition technology, combined with an image preprocessing process that includes environmental robustness design, and based on a deep learning model to achieve automatic segmentation of carriages and multi-dimensional damage identification, the invention ultimately outputs structured inspection results. This achieves full coverage, high precision, and all-weather intelligent inspection of railway freight carriages, significantly improving inspection efficiency, accuracy, and the business usability of the results.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide an intelligent detection method for railway freight inspection, including: In response to the detection of a train entering the detection area, continuous image data of the train carriages are collected by an array of line-scanning cameras deployed on both sides and the top of the track. Based on encoder signals and hardware trigger timestamps, the continuous image data is subjected to spatiotemporal synchronization processing. The continuous image data after the spatiotemporal synchronization process is preprocessed and stitched together to generate a two-dimensional unfolded image of the carriage. The two-dimensional unfolded image of the carriage is segmented based on the carriage segmentation model to obtain the segmentation mask and number association of each carriage, and the segmented image corresponding to each carriage is extracted from the two-dimensional unfolded image of the carriage according to the segmentation mask. The segmented image is subjected to damage identification based on the damage detection model, and a structured detection result containing the damage location, damage type and confidence level is output.

[0008] In an optional implementation, in response to detecting a train entering the detection area, continuous image data of the train carriages is acquired using an array of line-scan cameras deployed on both sides and the top of the track, including: The image acquisition process is initiated by triggering a signal, controlling the line scan camera array to start image acquisition at the set line frequency, and controlling the matching high color index LED light source to light up; During image acquisition, each camera in the control line scan camera array acquires continuous image data streams from the left, right, and top sides of the carriage according to its spatial position. When acquiring the continuous image data streams, the light incident on the camera sensor is polarized and filtered by a cross polarizer configured for the high color rendering index LED light source and the camera lens to suppress specular reflection light components from the metal surface of the carriage.

[0009] In an optional implementation, the spatiotemporal synchronization processing of the continuous image data based on encoder signals and hardware trigger timestamps includes: It receives pulse signals generated in real time by the trackside encoder and calculates the instantaneous linear velocity of the train based on the pulse signals. and cumulative displacement; Based on a precise clock protocol, a unified timestamp is configured for each row of the continuously acquired image data stream to generate a timestamped image data stream; Based on instantaneous linear velocity and target spatial resolution Through formula Dynamically calculate and adjust the line acquisition frequency of each line scan camera. This keeps the spatial sampling rate of the image constant.

[0010] In an optional implementation, the preprocessing and stitching of the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage includes: Obtain the pre-calibrated camera intrinsic parameter matrix and distortion coefficient vector and pre-acquired calibration plate bright field images Dark field images under light-blocking conditions ; For each row of the timestamped image data stream, the intrinsic parameter matrix is ​​applied sequentially. and distortion coefficient vector Perform geometric transformations to correct geometric distortions caused by lens optical characteristics and generate geometrically corrected image data; For the geometrically corrected image data, apply the brightness field image... and the dark field image The calculation is performed using the formula. Calculate the gain and bias matrices to perform flat-field and dark-field corrections for sensor pixel response non-uniformity and dark current noise, obtaining the corrected image. ,in The original input image; Based on the displacement reference provided by the encoder signal and the correction image The optical flow field for each row of image data is calculated using the Lucas-Kanade optical flow algorithm. ,in For pixel coordinates, The optical flow vector at this coordinate represents the local motion between adjacent image rows; According to the optical flow field By performing subpixel-level inverse deformation and interpolation on the image rows, motion compensation is performed for the inter-row misalignment caused by train speed fluctuations, resulting in a compensated image. ; For the compensated image For consecutive image rows, the sub-pixel offset is calculated using a phase correlation method based on Fourier transform. And use the random sampling consensus algorithm to filter the correct set of matching point pairs. Multiple image lines are stitched together and merged into a complete two-dimensional unfolded image of the carriage. ;in, and These represent the sub-pixel offsets between rows in the horizontal and vertical directions, respectively.

[0011] In an optional implementation, the step of performing instance segmentation on the two-dimensional unfolded image of the carriage based on the carriage segmentation model to obtain the segmentation mask and number association for each carriage includes: The two-dimensional unfolded image of the carriage The input is fed into a preset carriage segmentation model for forward inference; the carriage segmentation model is based on an encoder-decoder structure, and its encoder part uses a cross-stage local network structure to extract multi-scale feature maps. ,in The total number of layers in the feature map. For the first The feature map of the layer is used; the decoder part employs a path aggregation network structure to fuse and upsample feature maps of different scales to generate the final feature map. The carriage segmentation model uses its segmentation head to analyze feature maps. At each pixel location, semantic category prediction and instance mask coefficient prediction are performed simultaneously, and a predicted mask containing multiple candidate instances is output. and their corresponding category confidence scores ,in Index for candidate instances; Prediction mask for multiple candidate instances output by the forward inference of the carriage segmentation model and confidence score Post-processing is performed, using cluster analysis and non-maximum suppression based on the spatial location, contour topology, and confidence score of the predicted mask to form connected regions corresponding to each individual carriage. ,in The total number of carriages detected at the end. Indicates the first The pixel regions corresponding to each carriage are assigned a unique logical number based on the timing logic of the train entering the detection area. This allows us to obtain the final segmentation mask for each carriage. With number The related pairs.

[0012] In an optional implementation, the damage identification of the segmented image based on the damage detection model, and the output of structured detection results including damage location, damage type, and confidence level, include: For each carriage, based on its associated final segmentation mask Two-dimensional unfolded image of the carriage The corresponding segmented image is cropped from the middle. ; By the segmented image The input is fed into a pre-trained damage detection model for forward inference to generate the original detection result; wherein, the damage detection model is based on a single-stage target detection framework, including a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a detection head including a classification head, a regression head, and a severity regression head; The specific process of forward inference includes: The input image is processed through the backbone network. Feature extraction is performed to generate an initial feature map; the initial feature map is then fused and enhanced using the neck network at multiple scales to generate a fused feature map suitable for detection; the fused feature map is then processed by the regression head of the detection head to generate a set of predicted damage bounding boxes. ;in Indicates the first One prediction box, For its center coordinates, Its width and height; The fused feature map is processed by the classification head of the detection head to generate a corresponding set of damage category probability distributions. ,in Indicates the first Each prediction box belongs to The probability of each predefined damage category is calculated; the fused feature map is processed by the severity regression head of the detection head to generate a corresponding set of damage severity scores. ,in Indicates the first Damage severity prediction score for each prediction box; After forward inference is completed, the set of predicted bounding boxes in the original detection results is... Set of category probability distributions and severity score set Non-maximum suppression (NMS) algorithm is applied to eliminate overlapping redundant detection boxes; the original detection results after NMS processing are then analyzed according to a preset confidence threshold. Filter the boxes and retain those with a confidence level higher than the threshold to obtain the final set of valid boxes. Integrate the car numbers The system generates a structured damage list for each train car by taking the bounding box coordinates, damage category label, confidence score, and severity score of each valid detection box, according to a predefined data structure. .

[0013] In an optional implementation, the step of preprocessing and stitching the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage further includes: During the image acquisition phase, based on preset ambient light sensor data, the driving current of the LED light source is dynamically adjusted via a negative temperature coefficient thermistor temperature compensation circuit. To compensate for light output fluctuations caused by changes in ambient temperature, and to automatically switch between bright and dark illumination modes to adapt to different surface materials, thereby optimizing the signal-to-noise ratio and contrast of the continuous image data stream; In the image preprocessing stage, the corrected image or the compensated image Dynamic range analysis is performed, and for detected overexposed or underexposed areas, a high dynamic range image reconstruction algorithm based on multi-exposure fusion or tone mapping is activated to generate detailed image data. This image data is then used in subsequent stitching to generate the two-dimensional unfolded image of the carriage. ; The two-dimensional unfolded image of the carriage Input to the carriage segmentation model or the segmented image Before inputting the damage detection model, the two-dimensional unfolded image of the carriage is processed. or the segmented image Style normalization based on domain adaptive algorithm is performed to reduce the impact of differences in imaging distribution caused by different weather, seasons or line environments on the stability of model inference.

[0014] Secondly, embodiments of this application also provide an intelligent inspection system for railway freight, comprising: The image acquisition module is used to acquire continuous image data of the train carriages in response to the detection of a train entering the detection area by using an array of line-scan cameras deployed on both sides and the top of the track. The synchronization module is used to perform spatiotemporal synchronization processing on the continuous image data based on encoder signals and hardware trigger timestamps; The preprocessing module is used to preprocess and stitch together the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage. The image segmentation module is used to perform instance segmentation on the two-dimensional unfolded image of the carriage based on the carriage segmentation model, obtain the segmentation mask and number association of each carriage, and extract the segmented image corresponding to each carriage from the two-dimensional unfolded image of the carriage according to the segmentation mask; The detection module is used to identify damage in the segmented image based on the damage detection model and output a structured detection result containing the damage location, damage type and confidence level.

[0015] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the intelligent detection method for railway freight inspection as described in any of the above.

[0016] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the intelligent detection method for railway freight inspection as described in any of the above claims.

[0017] As can be seen from the above technical solutions, the present invention has the following advantages: The intelligent inspection method for railway freight inspection provided in this application achieves comprehensive, high-definition image coverage of the side walls, tops, and other surfaces of high-speed train carriages by deploying a multi-view, high-resolution line-scan camera array and adaptive synchronous acquisition technology. Utilizing an advanced preprocessing workflow including geometric correction, flat-field correction, motion compensation, and robust stitching algorithms, it effectively overcomes the effects of lens distortion, sensor noise, speed fluctuations, and complex lighting conditions (such as strong reflections and low illumination), generating high-quality, uniform-scale two-dimensional unfolded images of the carriages. Furthermore, based on a proprietary instance segmentation model and a multi-task target detection model, this method can automatically and accurately separate each carriage and locate, classify, and assess the severity of various types of damage on their surfaces, ultimately outputting structured inspection results that can directly drive operational and maintenance decisions. Overall, this method achieves full automation from image acquisition to defect identification, significantly improving the efficiency, accuracy, environmental adaptability, and usability of inspection results, providing a reliable and efficient intelligent inspection method for railway freight safety.

[0018] This application employs multiple high-resolution line-scan camera arrays on both sides and the top of the track for synchronous acquisition, combined with adaptive synchronous control technology, to achieve seamless and continuous coverage of the side walls, top, and other surfaces of the carriage during high-speed movement. By dynamically adjusting the line frequency and exposure based on encoder signals, a constant high spatial resolution is ensured, fundamentally solving the problems of image blurring and uneven sampling in high-speed scenes, and providing a high-quality raw data foundation for subsequent analysis.

[0019] To address the complex and challenging working conditions at railway freight inspection sites, including day and night lighting, metallic reflections, and the effects of rain, snow, and dust, this application integrates multiple active and passive optimization measures. Strong reflections are suppressed through cross-polarized illumination and programmable light source control; weather interference is addressed using image enhancement algorithms based on physical models or deep learning (such as HDR reconstruction, defogging, and deraining); and domain adaptation technology is introduced to reduce cross-scene differences. These designs ensure that the system can stably acquire clear images suitable for analysis even in variable environments, achieving reliable all-weather operation.

[0020] This application constructs an end-to-end automated processing chain from raw data to structural results. Through a series of automated preprocessing steps, including spatiotemporal synchronization, geometric and flat-field correction, motion compensation, and high-precision stitching, the raw line scan data is efficiently transformed into complete two-dimensional unfolded images of the carriage. Subsequently, the carriage instance segmentation and damage identification process based on a deep learning model is fully automated, replacing the traditional manual inspection and judgment, significantly reducing labor intensity and improving overall detection efficiency.

[0021] This application employs a deep neural network architecture optimized for complex scenarios. The carriage segmentation model can accurately separate each carriage and associate it with a number; the damage detection model can not only locate and classify various types of damage (such as dents, cracks, and corrosion), but also provide a quantitative damage severity score through an added severity regression branch. The advanced loss functions such as CIoU and Focal Loss used in model training effectively improve the ability to detect small targets and handle class imbalance problems, thereby enabling the accuracy and granularity of damage identification to reach the level required for operation and maintenance decision-making.

[0022] The final output of this application is not a simple image or alarm, but a structured data list (such as JSON format) containing the compartment number, damage type, precise location, confidence level, and severity score. This highly structured result can be directly interfaced with a vehicle maintenance management information system (MIS) or digital twin platform, seamlessly driving advanced applications such as maintenance work order generation, health status assessment, and asset life prediction, greatly enhancing the business value and system usability of the inspection results. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating the intelligent inspection method for railway freight provided in this application.

[0025] Figure 2 This is a schematic diagram of the intelligent inspection system for railway freight provided in this application.

[0026] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0027] The various embodiments of this disclosure will be described more fully in the following detailed description of the specific steps of the intelligent inspection method for railway freight. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0028] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The diagram shows a flowchart of a smart inspection method for railway freight in a specific embodiment. The method includes: S1: In response to the detection of a train entering the detection area, continuous image data of the train carriages are collected by an array of line-scan cameras deployed on both sides and the top of the track.

[0031] In a specific implementation, the image acquisition process is initiated in response to a trigger signal emitted by a lidar or infrared light curtain sensor installed at the entrance of the detection area. Specifically, acquisition commands are sent to the line scan camera array deployed on both sides of the track (left and right camera groups) and on the gantry above the track (top camera group). All cameras are high-resolution line scan cameras (e.g., DALSA Piranha 4 series) with 4096 pixels, a bit depth of 14 bits, and a maximum line frequency of 67kHz. Simultaneously, high color rendering index (CRI>90) LED linear light sources, strictly synchronized with the cameras, are illuminated.

[0032] To address the strong specular reflection from the metal surfaces of the train carriages, a cross-polarization scheme was implemented: linear polarizers were installed in front of the LED light sources, and analyzers were added in front of each camera lens. The polarization directions of the two systems were perpendicular to each other, effectively filtering out most of the reflected glare and enhancing the visibility of surface textures and defects. As the train passed at a constant or varying speed, each camera group continuously acquired data, generating continuous line scan image data streams corresponding to the left, right, and top surfaces of the carriages.

[0033] S2: Based on the encoder signal and hardware trigger timestamp, perform spatiotemporal synchronization processing on the continuous image data.

[0034] In this specific implementation, the aim is to establish a unified temporal and spatial reference for all acquired image data. The ABZ three-phase pulse signal generated by a trackside incremental rotary encoder (e.g., the OMRONE6C3 series) is received in real-time via an RS-422 interface. Based on this pulse frequency, the instantaneous linear velocity of the train is calculated in real-time using the built-in system software. (Unit: meters per second). Simultaneously, all cameras, encoders, and industrial control computers are networked through industrial switches supporting the IEEE 1588 Precision Time Protocol (PTP) to achieve microsecond-level cross-device clock synchronization, assigning a precise and unified timestamp to each line of acquired image data. Based on the calculated instantaneous velocity and the preset target spatial resolution (For example, setting) Dynamically adjust the actual line frequency of each line scan camera. The formula is adjusted as follows:

[0035] For example, when the train speed At a speed of 5 m / s, the line frequency The speed is automatically adjusted to 10,000 lines per second to ensure that the spatial sampling rate of the image remains constant in the direction of train travel, regardless of the train speed, thus laying a geometric consistency foundation for subsequent image stitching.

[0036] S3: Preprocess and stitch the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage.

[0037] In a specific implementation, this step involves a series of corrections and fusions of the synchronized original image stream to generate a high-quality, distortion-free two-dimensional unfolded image of the carriage.

[0038] First, pixel-level correction is performed. This involves calling the intrinsic parameter matrix for each camera, which was pre-obtained using a checkerboard calibration method. and distortion coefficient vector (Including radial distortion) and tangential distortion For each row of raw data, the `undistort` function from the OpenCV library is applied to eliminate barrel or pincushion distortion of the lens through inverse mapping and bilinear interpolation. Next, flat-field and dark-field corrections are performed. A pre-acquired uniform white board image is then loaded. (Bright field) and images when the lens is completely covered (Dark field). For each frame of the corrected image According to the formula Pixel-by-pixel calculations are performed. This operation eliminates photodynamic non-uniformity (PRNU) and dark current noise (DSNU) in each pixel of the sensor, outputting a corrected image with uniform grayscale response. .

[0039] Then, motion compensation is performed. The system combines the macroscopic displacement information provided by the encoder with... The sequence was analyzed using the Lucas-Kanade pyramid optical flow algorithm to calculate the local optical flow field between adjacent image rows. Based on the optical flow field, subpixel-level inverse deformation (using bicubic interpolation) is performed on the image rows to compensate for the inter-row misalignment and stretching caused by minute fluctuations in train speed, generating a precisely aligned compensated image. .

[0040] Finally, perform strip splicing. For consecutive image rows belonging to the same physical plane (such as the entire left side image), the sub-pixel-level offset between rows is calculated using a phase correlation method based on Fourier transform. The RANSAC (Random Sample Consensus) algorithm is used to select the correct set of matching point pairs. To eliminate mismatches caused by local feature similarity, a robust global transformation model is obtained. Applying this model, all image rows are mapped to a unified coordinate system, and a multi-band fusion algorithm is used for seam fusion to eliminate brightness differences, ultimately generating a seamless and complete two-dimensional unfolded image of the entire carriage's side or top surface. .

[0041] S4: Based on the carriage segmentation model, perform instance segmentation on the two-dimensional unfolded image of the carriage, obtain the segmentation mask and number association of each carriage, and extract the segmented image corresponding to each carriage from the two-dimensional unfolded image of the carriage according to the segmentation mask.

[0042] In a specific implementation, this step involves displaying an image of the entire unfolded train. It is broken down into independent single-car units.

[0043] First, The input is fed into a pre-trained carriage segmentation model. This model is based on the YOLOv11-seg instance segmentation architecture, with its backbone network using CSPDarknet and its neck network using a Bidirectional Feature Pyramid Network (BiFPN) to enhance multi-scale feature fusion capabilities. The model performs forward inference on the input and outputs multiple candidate instances, each containing a predicted binary mask. ( (as candidate indexes) and a confidence score indicating that the region is a carriage. During the model training phase, a combined loss function is used:

[0044] Among them, classification loss Use focus loss (FocalLoss) To address foreground / background pixel imbalance; mask regression loss Dice loss is used to optimize the prediction mask and the ground truth labeled mask. The degree of overlap between them.

[0045] For example, the model's forward inference process includes: extracting multi-scale feature maps using a cross-stage local network structure with its encoder part. ,in The total number of layers in the feature map. For the first The feature map of the layer is used; the decoder part employs a path aggregation network structure to fuse and upsample feature maps of different scales to generate the final feature map. The carriage segmentation model uses its segmentation head to analyze feature maps. At each pixel location, semantic category prediction and instance mask coefficient prediction are performed simultaneously, and a predicted mask containing multiple candidate instances is output. and their corresponding category confidence scores ,in Index for candidate instances.

[0046] Then, post-processing is performed on the multiple candidate predictions output by the model. First, based on the confidence level... Initial filtering is performed. Next, non-maximum suppression (NMS) is applied to the remaining prediction masks, merging highly overlapping predictions based on an IoU (Intersection over Union) threshold (e.g., 0.5). Finally, connected component analysis is performed on the processed masks, with each independent connected component... This is considered as a precise pixel area of ​​a train carriage. Based on the sequence of train passage through the sensors (timing logic), these connected components are assigned consecutive logical numbers. (e.g., 1, 2, 3…), thus establishing the final segmentation mask for each carriage. Its logical number The related pairs.

[0047] S5: Based on the damage detection model, perform damage identification on the segmented image and output a structured detection result containing the damage location, damage type, and confidence level.

[0048] In a specific implementation, this step involves detailed defect detection for each carriage.

[0049] First, based on the segmentation mask obtained in the previous step... Images unfolded from two dimensions of the carriage Crop out the corresponding single carriage image from the middle .

[0050] Then, The input is fed into a pre-trained defect detection model. This model is based on the YOLOv11-det object detection architecture and has been customized and extended. Its detection head contains three parallel branches: (1) Regression branch: outputs predicted bounding boxes. (2) Classification branch: Output the probability distribution of the box belonging to each damage category (such as dent, crack, corrosion, breakage, deformation, etc.). (3) Severity regression branch: Output a scalar between 0 and 1 This indicates the severity of the damage. During model training, a weighted multi-task loss is used:

[0051] in, CIoU loss is used to improve the positioning accuracy of the frame; Employing a class-balanced focus loss, weighted by an inverse class frequency factor. To alleviate sample imbalance; Smoothed L1 loss was used to regress the severity score.

[0052] Model pair Perform forward inference to generate a set containing multiple original bounding boxes. Apply non-maximum suppression (NMS) to this set and set a confidence threshold. (For example, 0.7) is used for filtering to obtain the final list of valid damage detections.

[0053] Finally, the results are integrated and output. For each valid detection result, its bounding box coordinates, class (the one with the highest probability), confidence score, and predicted severity score are extracted. This information is then linked to the logical number of the train car. The data, along with the detection timestamp, is encapsulated in a predefined JSON format to generate a structured damage report for that train car. .For example: {"carriage_id": "3","timestamp": "2023-10-27T08:30:15.123Z","damages": [{"id": "1","type": "Rust","bbox": [1250, 560, 85, 120],"confidence": 0.92,"severity": 0.75}]} After all the reports from the carriages are compiled, they can be pushed to the operation and maintenance management platform in real time via a network interface (such as RESTful API), directly driving the generation and dispatch of maintenance work orders.

[0054] In this embodiment, by deploying a multi-view line-scan camera array and an adaptive synchronous acquisition mechanism, high-definition, all-around image coverage of high-speed moving carriages is achieved. A robust preprocessing workflow integrating geometric correction, flat-field correction, and optical flow motion compensation effectively overcomes the effects of lens distortion, sensor noise, and speed fluctuations, generating high-quality, uniform-scale two-dimensional unfolded images of the carriages. Through a deep learning-based carriage instance segmentation model and a multi-task damage detection model, automatic carriage separation and damage localization, classification, and severity assessment are achieved, significantly improving recognition accuracy and automation. Finally, a structured report containing damage type, location, confidence level, and severity score is output, enabling seamless integration of detection results with the operation and maintenance management system. This method significantly improves the efficiency, accuracy, environmental adaptability, and usability of railway freight inspection, providing a complete intelligent solution for railway freight safety.

[0055] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0056] The preprocessing and stitching of the image data after the spatiotemporal synchronization process also includes the following environmental adaptation process: During the image acquisition phase, based on data from the ambient light sensor, the driving current of the LED light source is dynamically adjusted using a negative temperature coefficient thermistor temperature compensation circuit. This compensates for light output fluctuations caused by changes in ambient temperature and automatically switches between bright and dark illumination modes to adapt to different surface materials, thereby optimizing the signal-to-noise ratio and contrast of the continuous image data stream.

[0057] In the image preprocessing stage, the corrected image or the compensated image Dynamic range analysis is performed, and for detected overexposed or underexposed areas, a high dynamic range image reconstruction algorithm based on multi-exposure fusion or tone mapping is activated to generate detailed image data. This image data is then used in subsequent stitching to generate the two-dimensional unfolded image of the carriage.

[0058] The two-dimensional unfolded image of the carriage Input to the carriage segmentation model or the segmented image Before inputting the damage detection model, the two-dimensional unfolded image of the carriage is processed. or the segmented image Style normalization based on domain adaptive algorithm is performed to reduce the impact of differences in imaging distribution caused by different weather, seasons or line environments on the stability of model inference.

[0059] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0060] During the training phase of the carriage segmentation model, the overall loss function used is... pixel-level classification loss and instance mask regression loss The weighted composition is expressed as: ,in and To balance the contributions of the two losses; the pixel-level classification loss A focus loss function is used to calculate and address the pixel imbalance between the foreground (carriage) and background. The formula is as follows:

[0061] in, This represents the total number of pixels in a single image. For the model to the first The predicted probability that each pixel belongs to its true class. The focus parameter used to adjust the weights of easy and difficult samples; the instance mask regression loss. Predict the mask by calculation Compared to real masks The optimization is achieved by using the Dice coefficient loss between them, and the formula is as follows:

[0062] in, For the binarization mask predicted by the model, A manually annotated, true binary mask. This indicates the number of foreground pixels in the calculation mask.

[0063] In one embodiment of the present invention, based on step S5, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0064] The damage detection model is trained through the following steps: Construct a bounding box with labeled damage. Damage Category and severity score of injury The training dataset; by inputting training images into the damage detection model for forward inference, a prediction result is generated, and the loss between the prediction result and the annotation is calculated; the training loss of the regression head. Using full intersection and union ratio loss The calculation formula is as follows:

[0065] in, The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. This indicates the calculation of the Euclidean distance between the center points of two bounding boxes. Let be the diagonal length of the smallest bounding rectangle containing the predicted bounding box and the ground truth bounding box. It is a parameter that measures the consistency of the aspect ratio of two bounding boxes. It is used for balance The weighting coefficients of each contribution; the training loss of the classification head. The focus loss with a weighted factor is calculated as follows:

[0066] in, For the set of all damage categories, For categories The weighting factor is calculated by inverse frequency weighting based on the frequency of that category in the training data. For the model to determine if a sample belongs to its true category The predicted probability, To focus on the parameters; the training loss of the severity regression head. Using smoothed L1 loss, the formula is as follows:

[0067] in To control the smoothing parameter of the loss function as it transitions from squared loss to linear loss; the overall training loss of the defect detection model. It is the weighted sum of the above losses, expressed as:

[0068] in , , These are the weight coefficients for the bounding box regression loss, classification loss, and severity regression loss, respectively; the overall training loss is minimized using the gradient descent algorithm. The parameters of the damage detection model are iteratively optimized until the model converges.

[0069] like Figure 2 As shown, the following are embodiments of the intelligent railway freight inspection system provided in this disclosure. This system and the intelligent railway freight inspection methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the intelligent railway freight inspection system, please refer to the embodiments of the intelligent railway freight inspection methods described above.

[0070] A railway freight inspection intelligent detection system includes: an image acquisition module, a synchronization module, a preprocessing module, an image segmentation module, and a detection module.

[0071] The image acquisition module is used to acquire continuous image data of the train carriages in response to the detection of a train entering the detection area by using an array of line-scan cameras deployed on both sides and the top of the track.

[0072] In a specific implementation, the image acquisition module is configured to acquire continuous image data of the train carriages through an array of line-scan cameras deployed on both sides and the top of the track.

[0073] The image acquisition module includes a left camera group, a right camera group, and a top camera group. Each camera group uses a high-resolution line scan camera with more than 4096 pixels, supports 12-bit or 14-bit bit depth, and has a line frequency of more than 60,000 lines per second.

[0074] The image acquisition module may also include a light source assembly. The light source assembly may include a high color rendering index (CRI) LED strip light source and a surface light source, configured with a zoned programmable constant current drive controller, operating at a frequency greater than 20 kHz to eliminate flicker. The light source assembly may be configured with a cross-polarization scheme, including a polarized light source and lens polarizers, to suppress strong reflections from metal surfaces.

[0075] For example, the image acquisition module uses a Scheimpflug tilt mounting scheme to improve edge sharpness in oblique scenes. Telecentric or semi-telecentric lenses can be selected to reduce the impact of magnification changes on recognition, with distortion controlled within 0.3%. The synchronization module is used to perform spatiotemporal synchronization processing on the continuous image data based on encoder signals and hardware trigger timestamps.

[0076] In a specific implementation, the synchronization module includes a trackside encoder and a clock synchronization unit. The trackside encoder outputs an ABZ pulse signal or an RS-422 signal to detect train speed and provide a trigger signal. The clock synchronization unit uses the IEEE 1588 Precision Time Protocol (PTP) to achieve cross-device clock alignment, ensuring timing consistency of data acquired by multiple cameras.

[0077] The synchronization module can calculate the train's real-time speed based on encoder pulses and adaptively adjust the camera's line frequency and exposure time accordingly to maintain a constant spatial sampling rate. When train speed fluctuates, the synchronization module can maintain consistent image quality through exposure-speed closed-loop control. The preprocessing module is used to preprocess and stitch together the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage.

[0078] In a specific implementation, the preprocessing module is used to perform the following operations: Geometric correction: including intrinsic parameter correction and distortion correction, to eliminate geometric distortion introduced by the lens.

[0079] Flat field correction: including PRNU (photoresponse non-uniformity) correction, DSNU (dark signal non-uniformity) correction and dark current correction, to improve image signal-to-noise ratio and grayscale consistency.

[0080] Lens shadow correction: Perform FFC (flat field correction) processing to eliminate light attenuation at the lens edges.

[0081] Motion compensation: Motion compensation is performed based on optical flow information and inertial navigation information to suppress geometric distortion caused by velocity fluctuations.

[0082] Strip stitching: A phase-correlated subpixel registration method and ECC (enhanced correlation coefficient) optimization are used for strip stitching. Combined with RANSAC geometric constraints and photometric consistency constraints, a uniform-scale unfolded image of the carriage is constructed.

[0083] Special working condition processing: including bad frame removal, line segment resampling, local HDR processing and glare removal, to ensure the usable screen rate under extreme working conditions.

[0084] The image segmentation module is used to perform instance segmentation on the two-dimensional unfolded image of the carriage based on the carriage segmentation model, obtain the segmentation mask and number association of each carriage, and extract the segmented image corresponding to each carriage from the two-dimensional unfolded image of the carriage according to the segmentation mask.

[0085] In a specific implementation, the carriage segmentation model is constructed using an instance segmentation network based on the YOLOv11-seg architecture. The image segmentation module outputs instance masks and their associated numbers for each carriage, ensuring accurate representation of contours and edge details.

[0086] The detection module is used to identify damage in the segmented image based on the damage detection model and output a structured detection result containing the damage location, damage type and confidence level.

[0087] In this specific implementation, the damage detection model is constructed using a target detection network based on the YOLOv11-det architecture. The model includes a severity regression branch to output a damage severity score normalized to 0 to 1. The model employs an IoU-aware classification mechanism to improve localization quality.

[0088] In some embodiments, the damage detection model is trained using CIoU and Focal loss functions to improve the detection capability of small targets. Class-Balanced Reweighting and OHEM (Online Hard Example Mining) strategies are employed to improve the detection stability of damage categories with few samples.

[0089] In a specific implementation, the system may also include an environment adaptive module.

[0090] The environment adaptive module is used to automatically adjust system parameters and processing strategies according to environmental conditions, specifically including: In response to the detection of nighttime or low-light environments, the environment adaptation module can enable adaptive exposure control and HDR imaging mode. A TDI (Time Delay Integration) linear scan camera can be selected to improve the signal-to-noise ratio in low-light conditions through multiple exposure integrations.

[0091] In response to the detection of strong metallic reflection, the environment adaptive module can enable a cross-polarization suppression strategy, which eliminates specular reflection by cooperating with the polarization light source and the lens polarizer.

[0092] In response to the detection of rain, snow, or dust, the environment adaptation module can enable prior defogging, deraining, or haze removal based on the physical model, or employ learning-based perturbation methods (Retinex, CLAHE, adaptive gain, etc.).

[0093] The environment adaptation module is also used to perform feature alignment for cross-line or cross-seasonal scenes based on domain adaptation algorithms (TTA, perturbation consistency, adversarial adaptation, etc.), and introduces style transfer (Style Transfer, Color Constancy) to reduce the impact of imaging differences on detection.

[0094] The intelligent railway freight inspection system provided in this embodiment achieves a comprehensive improvement in inspection capabilities through the collaborative work of multiple modules, including image acquisition, synchronous control, preprocessing, intelligent segmentation, and recognition. The system utilizes a multi-view line-scan camera array and an adaptive synchronization mechanism to ensure high-speed, full-coverage, and high-definition image acquisition. Its integrated robust preprocessing workflow effectively suppresses interference from complex environments, ensuring the stability and usability of image quality. The carriage segmentation and damage recognition modules based on deep learning models achieve a high degree of automation and high precision in the processing workflow. Finally, the structured damage report output by the system can be directly connected to the backend management system, significantly improving the business-driven capability of the inspection results and the efficiency of operation and maintenance decisions. Overall, this system realizes a closed loop from data perception to intelligent diagnosis, providing a reliable, efficient, and engineerably deployable intelligent solution for railway freight safety.

[0095] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0096] The intelligent inspection method for railway freight provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0097] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0098] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0099] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0100] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0101] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0102] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0103] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0104] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0105] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0106] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0107] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0108] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0109] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0110] The aforementioned electronic equipment realizes the intelligent inspection method for railway freight inspection proposed in this application. By deploying a multi-view line scan camera array and implementing adaptive synchronous acquisition, combined with an image preprocessing process that includes environmental robustness design and automated segmentation and recognition based on a deep learning model, structured inspection results are finally generated. This achieves the beneficial effects of full coverage, high precision, and all-weather intelligent inspection of railway freight cars, and significantly improves inspection efficiency, reliability, and business availability of results.

[0111] The storage medium provided in this application stores a program product capable of implementing an intelligent detection method for railway freight inspection.

[0112] Intelligent inspection methods for railway freight include: In response to the detection of a train entering the detection area, continuous image data of the train carriages are collected by an array of line-scanning cameras deployed on both sides and the top of the track. Based on encoder signals and hardware trigger timestamps, the continuous image data is subjected to spatiotemporal synchronization processing. The continuous image data after the spatiotemporal synchronization process is preprocessed and stitched together to generate a two-dimensional unfolded image of the carriage. The two-dimensional unfolded image of the carriage is segmented based on the carriage segmentation model to obtain the segmentation mask and number association of each carriage, and the segmented image corresponding to each carriage is extracted from the two-dimensional unfolded image of the carriage according to the segmentation mask. The segmented image is subjected to damage identification based on the damage detection model, and a structured detection result containing the damage location, damage type and confidence level is output.

[0113] In some possible implementations, the intelligent detection method for railway freight inspection disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0114] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0115] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A smart inspection method for railway freight, characterized in that, include: In response to the detection of a train entering the detection area, continuous image data of the train carriages are collected by an array of line-scanning cameras deployed on both sides and the top of the track. Based on encoder signals and hardware trigger timestamps, the continuous image data is subjected to spatiotemporal synchronization processing. The continuous image data after the spatiotemporal synchronization process is preprocessed and stitched together to generate a two-dimensional unfolded image of the carriage. The two-dimensional unfolded image of the carriage is segmented based on the carriage segmentation model to obtain the segmentation mask and number association of each carriage, and the segmented image corresponding to each carriage is extracted from the two-dimensional unfolded image of the carriage according to the segmentation mask. The segmented image is subjected to damage identification based on the damage detection model, and a structured detection result containing the damage location, damage type and confidence level is output.

2. The intelligent detection method for railway freight inspection according to claim 1, characterized in that, In response to the detection of a train entering the detection area, continuous image data of the train carriages is acquired by an array of line-scan cameras deployed on both sides and the top of the track, including: The image acquisition process is initiated by triggering a signal, controlling the line scan camera array to start image acquisition at the set line frequency, and controlling the matching high color index LED light source to light up; During image acquisition, each camera in the control line scan camera array acquires continuous image data streams from the left, right, and top sides of the carriage according to its spatial position. When acquiring the continuous image data streams, the light incident on the camera sensor is polarized and filtered by a cross polarizer configured for the high color rendering index LED light source and the camera lens to suppress specular reflection light components from the metal surface of the carriage.

3. The intelligent detection method for railway freight inspection according to claim 2, characterized in that, The spatiotemporal synchronization processing of the continuous image data based on encoder signals and hardware trigger timestamps includes: It receives pulse signals generated in real time by the trackside encoder and calculates the instantaneous linear velocity of the train based on the pulse signals. and cumulative displacement; Based on a precise clock protocol, a unified timestamp is configured for each row of the continuously acquired image data stream to generate a timestamped image data stream; Based on instantaneous linear velocity and target spatial resolution Through formula Dynamically calculate and adjust the line acquisition frequency of each line scan camera. This keeps the spatial sampling rate of the image constant.

4. The intelligent detection method for railway freight inspection according to claim 3, characterized in that, The step of preprocessing and stitching the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage includes: Obtain the pre-calibrated camera intrinsic parameter matrix and distortion coefficient vector and pre-acquired calibration plate bright field images Dark field images under light-blocking conditions ; For each row of the timestamped image data stream, the intrinsic parameter matrix is ​​applied sequentially. and distortion coefficient vector Perform geometric transformations to correct geometric distortions caused by lens optical characteristics and generate geometrically corrected image data; For the geometrically corrected image data, apply the brightness field image... and the dark field image The calculation is performed using the formula. Calculate the gain and bias matrices to perform flat-field and dark-field corrections for sensor pixel response non-uniformity and dark current noise, obtaining the corrected image. ,in The original input image; Based on the displacement reference provided by the encoder signal and the correction image The optical flow field for each row of image data is calculated using the Lucas-Kanade optical flow algorithm. ,in For pixel coordinates, The optical flow vector at this coordinate represents the local motion between adjacent image rows; According to the optical flow field By performing subpixel-level inverse deformation and interpolation on the image rows, motion compensation is performed for the inter-row misalignment caused by train speed fluctuations, resulting in a compensated image. ; For the compensated image For consecutive image rows, the sub-pixel offset is calculated using a phase correlation method based on Fourier transform. And use the random sampling consensus algorithm to filter the correct set of matching point pairs. Multiple image lines are stitched together and merged into a complete two-dimensional unfolded image of the carriage. ;in, and These represent the sub-pixel offsets between rows in the horizontal and vertical directions, respectively.

5. The intelligent detection method for railway freight inspection according to claim 4, characterized in that, The step of performing instance segmentation on the two-dimensional unfolded image of the carriage based on the carriage segmentation model, and obtaining the segmentation mask and number association for each carriage, includes: The two-dimensional unfolded image of the carriage The input is fed into a preset carriage segmentation model for forward inference; the carriage segmentation model is based on an encoder-decoder structure, and its encoder part uses a cross-stage local network structure to extract multi-scale feature maps. ,in The total number of layers in the feature map. For the first The feature map of the layer is used; the decoder part employs a path aggregation network structure to fuse and upsample feature maps of different scales to generate the final feature map. The carriage segmentation model uses its segmentation head to analyze feature maps. At each pixel location, semantic category prediction and instance mask coefficient prediction are performed simultaneously, and a predicted mask containing multiple candidate instances is output. and their corresponding category confidence scores ,in Index for candidate instances; Prediction mask for multiple candidate instances output by the forward inference of the carriage segmentation model and confidence score Post-processing is performed, using cluster analysis and non-maximum suppression based on the spatial location, contour topology, and confidence score of the predicted mask to form connected regions corresponding to each individual carriage. ,in The total number of carriages detected at the end. Indicates the first The pixel regions corresponding to each carriage are assigned a unique logical number based on the timing logic of the train entering the detection area. This allows us to obtain the final segmentation mask for each carriage. With number The related pairs.

6. The intelligent detection method for railway freight inspection according to claim 5, characterized in that, The segmented image is subjected to damage identification based on a damage detection model, and the structured detection results, including damage location, damage type, and confidence level, are output, including: For each carriage, based on its associated final segmentation mask Two-dimensional unfolded image of the carriage The corresponding segmented image is cropped from the middle. ; By the segmented image The input is fed into a pre-trained damage detection model for forward inference to generate the original detection result; wherein, the damage detection model is based on a single-stage target detection framework, including a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a detection head including a classification head, a regression head, and a severity regression head; The specific process of forward inference includes: The input image is processed through the backbone network. Feature extraction is performed to generate an initial feature map; the initial feature map is then fused and enhanced using the neck network at multiple scales to generate a fused feature map suitable for detection; the fused feature map is then processed by the regression head of the detection head to generate a set of predicted damage bounding boxes. ;in Indicates the first One prediction box, For its center coordinates, Its width and height; The fused feature map is processed by the classification head of the detection head to generate a corresponding set of damage category probability distributions. ,in Indicates the first Each prediction box belongs to The probability of each predefined damage category is calculated; the fused feature map is processed by the severity regression head of the detection head to generate a corresponding set of damage severity scores. ,in Indicates the first Damage severity prediction score for each prediction box; After forward inference is completed, the set of predicted bounding boxes in the original detection results is... Set of category probability distributions and severity score set Non-maximum suppression (NMS) algorithm is applied to eliminate overlapping redundant detection boxes; the original detection results after NMS processing are then analyzed according to a preset confidence threshold. Filter the boxes and retain those with a confidence level higher than the threshold to obtain the final set of valid boxes. Integrate the car numbers The system generates a structured damage list for each train car by taking the bounding box coordinates, damage category label, confidence score, and severity score of each valid detection box, according to a predefined data structure. .

7. The intelligent detection method for railway freight inspection according to claim 4, characterized in that, The step of preprocessing and stitching the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage also includes: During the image acquisition phase, based on preset ambient light sensor data, the driving current of the LED light source is dynamically adjusted via a negative temperature coefficient thermistor temperature compensation circuit. To compensate for light output fluctuations caused by changes in ambient temperature, and to automatically switch between bright and dark illumination modes to adapt to different surface materials, thereby optimizing the signal-to-noise ratio and contrast of the continuous image data stream; In the image preprocessing stage, the corrected image or the compensated image Dynamic range analysis is performed, and for detected overexposed or underexposed areas, a high dynamic range image reconstruction algorithm based on multi-exposure fusion or tone mapping is activated to generate detailed image data. This image data is then used in subsequent stitching to generate the two-dimensional unfolded image of the carriage. ; The two-dimensional unfolded image of the carriage Input to the carriage segmentation model or the segmented image Before inputting the damage detection model, the two-dimensional unfolded image of the carriage is processed. or the segmented image Style normalization based on domain adaptive algorithm is performed to reduce the impact of differences in imaging distribution caused by different weather, seasons or line environments on the stability of model inference.

8. A railway freight inspection intelligent detection system, characterized in that, The system employs the intelligent detection method for railway freight inspection as described in any one of claims 1 to 7; The system includes: The image acquisition module is used to acquire continuous image data of the train carriages in response to the detection of a train entering the detection area by using an array of line-scan cameras deployed on both sides and the top of the track. The synchronization module is used to perform spatiotemporal synchronization processing on the continuous image data based on encoder signals and hardware trigger timestamps; The preprocessing module is used to preprocess and stitch together the continuous image data after the spatiotemporal synchronization processing to generate a two-dimensional unfolded image of the carriage. The image segmentation module is used to perform instance segmentation on the two-dimensional unfolded image of the carriage based on the carriage segmentation model, obtain the segmentation mask and number association of each carriage, and extract the segmented image corresponding to each carriage from the two-dimensional unfolded image of the carriage according to the segmentation mask; The detection module is used to identify damage in the segmented image based on the damage detection model and output a structured detection result containing the damage location, damage type and confidence level.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the intelligent detection method for railway freight inspection as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent detection method for railway freight inspection as described in any one of claims 1 to 7.

Images