Railway wagon loading state detection method based on image recognition

Abstract

The invention discloses a railway freight car loading state detection method based on image recognition, and particularly relates to the field of railway freight car loading state detection.The method comprises the steps that in the process that a railway freight car passes through an image collection area, carriage image frames are collected at a constant frame rate through linear array imaging equipment; an image sequence is constructed based on the wheel axle displacement, the acquisition timestamp and the frame number, the image sequence comprises a main image frame, a relay interpolation frame and an interpolation pseudo frame, and the main image frame, the relay interpolation frame and the interpolation pseudo frame are arranged to support time domain modeling of subsequent state response changes. The dynamic capture and accurate identification of the instantaneous abnormal state of the loading structure are realized by constructing an anchor point tracking graph and combining a time sequence rhythm modeling and similarity discrimination mechanism of a disturbance behavior. The problems that in the prior art, an identification system lacks the space-time continuous modeling capacity for the short-time change state, and an abnormal judgment blind area and misjudgment distortion exist are solved.

Description

Technical Field

[0001] This invention relates to the field of railway freight car loading status detection technology, and more specifically, to a railway freight car loading status detection method based on image recognition. Background Technology

[0002] In current railway freight car loading status image recognition systems, high-speed linear array scanning equipment is used to acquire single-frame images of the train, and a static target detection model is used to identify the image content. However, in actual operating environments, many critical loading anomalies (such as loose container doors, slippage of tarpaulin straps, slight opening of tank car top covers, and window latches coming off track) are not continuous static anomalies, but rather appear as instantaneous changes in state. These changes usually occur when the vehicle vibrates or passes through track nodes such as joints, curves, and slopes. Their duration is extremely short, often covering only a few frames of acquired images, and in some cases, even appearing only in a single frame. Because the imaging system and the recognition system lack joint modeling capabilities in the time dimension, this "spatiotemporal drift of abnormal states" is easily misjudged by the system as normal transitional behavior or directly missed due to incomplete information. Further analysis shows that such transient anomalies may be early signs of real risks or just slight shaking caused by structural loosening. The distinction requires combining multiple key dimensions: the trajectory continuity of the target behavior in the time series, the spatial offset trend between adjacent frames, and the inertial response pattern of the structural components themselves. However, most of the current mainstream recognition mechanisms are limited to the level of single image frames or single targets, lacking the ability to construct "micro motion patterns" based on continuous frames. This makes it impossible for the system to determine whether a certain state has the potential to develop into a high-risk fault. This not only brings serious risks of missed detection, but also allows some potential anomalies that require early intervention to accumulate during train operation, eventually turning into serious accidents such as cargo loss, jamming, and collision. In summary, the problem with existing technologies is that existing image recognition systems lack the ability to dynamically perceive short-term changes in the state of the loading structure and cannot construct a spatiotemporal continuity representation model of the target state. This results in significant blind spots and discrimination distortion in the identification of instantaneous abnormal states, and this problem is particularly prominent in railway freight scenarios with high-speed operation and severe environmental disturbances. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a railway freight car loading status detection method based on image recognition. By constructing an anchor point tracking map and combining it with temporal rhythm modeling of disturbance behavior and a similarity discrimination mechanism, the method achieves dynamic capture and accurate identification of instantaneous abnormal states of the loading structure. This solves the problems mentioned in the background art, such as the lack of spatiotemporal continuity modeling capability for short-term changing states, the existence of anomaly discrimination blind spots, and misjudgment distortion.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting the loading status of railway freight cars based on image recognition, comprising: S1. During the process of railway freight cars passing through the image acquisition area, a linear array imaging device is used to acquire car image frames at a constant frame rate. An image sequence is constructed based on wheel axle displacement, acquisition timestamp, and frame number. The image sequence includes the main image frame, relay interpolation frames, and interpolation pseudo-frames. S2. Perform loading structure region segmentation operation on each image frame in the image sequence. Based on the structural segmentation model of the segmentation operation, identify the loading structure regions corresponding to the vehicle door, box door, can lid, and tarpaulin belt. Construct an anchor point set based on the boundary lines segmented from the loading structure regions. Perform trajectory matching between frames of the anchor point set in the temporal tensor to generate an anchor point tracking map. S3. Perform trajectory extraction operation on the anchor point tracking map, calculate the perturbation amount sequence based on the spatial coordinate change and edge contour change of the anchor point between image frames, and extract structural segments that meet the perturbation conditions from the perturbation amount sequence based on the preset perturbation amount threshold to form a perturbation candidate group. S4. For each structural segment in the perturbation candidate group, apply an encoder to extract the rhythm frequency, deformation amplitude and duration, and construct a perturbation feature set. Combine the image texture changes corresponding to the structural segment to solve the perturbation behavior expression vector, and summarize it into a perturbation vector set. S5. Input the perturbation vector set into the abnormal trend discrimination network, calculate the Euclidean distance with the stored abnormal behavior cluster centers, and generate similarity score results. If the score result is greater than the upper limit of the preset threshold, the corresponding structural segment is marked as an abnormal trajectory; if the score result is within the discrimination range formed by the upper and lower limits of the preset threshold, the disturbed trajectory is included in the set of paths to be confirmed. S6. Perform anomaly annotation on structural segments identified as abnormal trajectories and generate corresponding image frame position identifiers, disturbance behavior expression vectors, and similarity scores to form structured anomaly records; for structural segments in the set of paths to be confirmed, perform image frame backhaul and update the boundary range in the anchor point tracking map.

[0005] In a preferred embodiment, in step S1, during the process of a railway freight car passing through the image acquisition area, wheel axle displacement data, image acquisition timestamp data, and image frame number corresponding to each image acquisition moment are acquired, and a time positioning index is established based on the relationship between wheel axle displacement and timestamp. Based on the time positioning index, image frames are selected at fixed acquisition intervals during the image acquisition process to form a main image frame set, which constitutes the time axis of the image sequence; Between every two adjacent main image frames, image frames whose acquisition time falls between the two frames are selected to construct a relay interpolation frame set, which is used to supplement the transition state information between the main image frame sets. For acquisition time points not covered by the main image frame set and the relay interpolation frame set, the image interpolation results between adjacent image frames are calculated using an interpolation algorithm to generate an interpolation pseudo-frame set. The interpolation pseudo-frame set is used to construct the missing frame positions in the image time series. The interpolation algorithm includes using a linear interpolation method based on pixel intensity. The main image frame set, the relay interpolation frame set, and the interpolation pseudo-frame set are merged in chronological order to generate an image sequence.

[0006] In a preferred embodiment, in S2, a structural segmentation operation is performed on each image frame in the image sequence, the structural segmentation model is called, and the corresponding structural region map is output in each image frame based on the edge features of the car door, box door, can lid and tarpaulin belt. Based on the boundary contours of the structural regions described in the structural region diagram, perform a boundary line extraction operation to extract the closed boundary lines of the structural regions and construct a set of boundary lines. Based on the curvature change points and corner points of the boundary lines in the boundary line set, extract discrete coordinate points representing the spatial position of the loading structure to construct an anchor point set; Arrange the anchor point set according to the image frame sequence, establish the correspondence between anchor points in each image frame, and form inter-frame anchor point matching pairs; Based on the consistency of spatial distance and boundary direction between inter-frame anchor point matching pairs, anchor point trajectory sequences are calculated, and an anchor point tracking map is constructed based on all anchor point trajectory sequences to represent the spatial path of the loading structure as it changes over time.

[0007] In a preferred embodiment, in S2, the construction of the structure segmentation model includes: extracting structural region images containing vehicle doors, box doors, can lids, and tarpaulin strips based on annotated image frame sequences, and constructing a training sample set according to structural region categories; calling an image encoding network to perform feature extraction on the input image frames to generate texture feature maps and edge feature maps; inputting the texture feature maps and edge feature maps into a feature fusion module to perform feature aggregation and form a fused feature map; inputting the fused feature map into a region classification module to output a structural region map, wherein each pixel corresponds to a structural category label; A forward inference operation is performed on the image frame to be processed, and the corresponding structural region map is output. The structural region map is used for boundary line extraction and anchor point set construction.

[0008] In a preferred embodiment, in S3, for each anchor point trajectory in the anchor point tracking map, its spatial coordinate sequence in the image frame sequence is extracted, and a spatial displacement sequence is calculated based on the inter-frame coordinate difference. The spatial displacement sequence is used to represent the spatial offset behavior of the anchor point over time. For the image frame region corresponding to the spatial displacement sequence, the edge contour line of each anchor point region is extracted, and the edge change sequence is calculated based on the inter-frame position difference of the edge contour line. The edge change sequence is used to characterize the response of the structural region at the boundary.

[0009] In a preferred embodiment, in S3, the spatial displacement sequence and the edge change sequence are fused according to the frame position, a disturbance amount calculation operation is performed, and a disturbance amount sequence is constructed based on the fusion result. The disturbance amount sequence reflects the comprehensive disturbance amplitude of the anchor point trajectory between consecutive frames. Extract consecutive frame segments with disturbance values ​​greater than the preset disturbance limit to form an initial set of disturbance segments; Segment merging is performed on adjacent segments in the initial set of disturbance segments to generate a disturbance candidate group, where each segment in the disturbance candidate group corresponds to a continuous disturbance segment in the anchor point trajectory.

[0010] In a preferred embodiment, in S4, each structural segment in the perturbation candidate group is obtained, and the number of perturbation duration frames is calculated based on the start frame and end frame of the structural segment in the image sequence. The number of perturbation duration frames is used to describe the span of the structural segment in the time dimension. The anchor point trajectory of the structural segment within the image frame range covered by the duration of the disturbance is input into the encoder. The rhythm frequency is calculated based on the amplitude fluctuation of the trajectory as the frame order changes, and the maximum amplitude of the trajectory within the frame segment is extracted as the deformation amplitude. The rhythm frequency, deformation amplitude, and number of perturbation duration frames are combined to form a perturbation feature group; texture difference calculation is performed on the image frame region corresponding to the number of perturbation duration frames for the structural segment, and the texture change value of the region in the frame sequence is extracted; The perturbation feature set is combined with the texture change value to generate the corresponding perturbation behavior expression vector, and all perturbation behavior expression vectors are summarized into a perturbation vector set according to the structural segment order.

[0011] In a preferred embodiment, in S5, an abnormal trend discrimination network is constructed, and the perturbation vector set is set as follows: ;in Indicates the first The perturbation vector corresponding to each structural segment. This represents the total number of perturbation vectors. For the perturbation feature dimension, Standardization and normalization are performed to generate a zero-mean, unit-variance perturbation matrix. ; A set of perturbation feature clusters is constructed based on historical anomaly samples. A radius-adjusted K-means++ initialization strategy is then applied to obtain the set of cluster centers for anomaly behavior. ; each of them Indicates the first Each anomaly category center For the perturbation feature dimension, The number of currently defined abnormal behavior categories; Calculate each The weighted Euclidean distance between the cluster and all anomalous cluster centers is defined by the weight matrix. : For each and Calculate the weighted distance: The weight matrix is ​​used for weighting. It includes the weight values ​​corresponding to each feature dimension in the perturbation vector, totaling... One dimension; Indicates the first The standardized perturbation vector corresponding to each structural segment It is the input data of the vector to be judged; Indicates the first Weight coefficients for each feature dimension; The perturbation vector is represented by the first... The weight coefficients corresponding to each dimension; Representing cluster centers In the Component values ​​in each feature dimension; express With cluster center The weighted Euclidean distance between them; express In the Component values ​​in each feature dimension; Indicates the first The squared difference between the perturbation vector and the cluster center in each feature dimension; Weighted Euclidean distance Input a non-linear scoring function and perform piecewise mapping to calculate the similarity score: ;in These represent the upper and lower limits of the threshold range for similarity scores; express and Similarity score between them; Statistics for each Highest similarity score across all cluster centers , ; and will Compared with the preset scoring range If a comparison is made, If so, the corresponding structural segment will be marked as an abnormal trajectory; if If it is found to be true, it will be included in the set of paths to be confirmed; otherwise, it will be left unprocessed or marked as normal.

[0012] The technical effects and advantages of this invention are as follows: This invention improves the ability to identify short-term disturbance-type loading anomalies. In view of the problem that the "spatiotemporal drift of abnormal state" cannot be captured in the existing technology, it proposes a strategy based on anchor point trajectory and disturbance rhythm modeling, and constructs a cross-frame state expression mechanism to realize continuous capture and discrimination of instantaneous behaviors such as can lid loosening and tarpaulin slippage, and solves the problem of recognition blind spots caused by single-frame misjudgment and information incompleteness. This invention introduces an image sequence interpolation mechanism to fill in the missing frame gaps during the main frame acquisition process. Without interrupting trajectory construction, it enhances the temporal integrity of the image sequence, provides a continuous data foundation for subsequent anchor point tracking and disturbance extraction, and improves trajectory stability and the robustness of abnormal behavior recognition. This invention constructs a structural segmentation model and generates an anchor point set by combining boundary line extraction. It uses multi-frame anchor point spatial consistency matching to form an anchor point tracking map, transforming the target structural information in the original image into a traceable dynamic state path, thereby enhancing the ability to model the response behavior of key structures. This invention extracts the rhythmic frequency, deformation amplitude, and duration of perturbation behavior through a rhythm encoder, and calculates the perturbation expression vector by combining image texture changes. It models and classifies the perturbation state in a multi-dimensional feature space, thereby improving the accuracy of distinguishing between real anomalies and inertial perturbations. This invention introduces a cluster center distance assessment and dual threshold scoring mechanism in the abnormal trend discrimination stage. It performs feature-weighted distance discrimination on the perturbation expression vector and outputs clear abnormal state labels or unconfirmed state diversion strategies, thereby achieving controllable false alarm rate. Attached Figure Description

[0013] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Refer to the instruction manual appendix Figure 1An embodiment of the present invention provides a railway freight car loading status detection method based on image recognition, comprising: S1. During the process of a railway freight car passing through the image acquisition area, a linear array imaging device is used to acquire carriage image frames at a constant frame rate. An image sequence is constructed based on wheel axle displacement, acquisition timestamp, and frame number. The image sequence includes a main image frame, relay interpolation frames, and interpolation pseudo-frames. The setting of the main image frame, relay interpolation frames, and interpolation pseudo-frames is used to support the time domain modeling of subsequent state response changes. S2. Perform loading structure region segmentation operation on each image frame in the image sequence. Based on the structural segmentation model of the segmentation operation, identify the loading structure regions corresponding to the vehicle door, box door, can lid, and tarpaulin belt. Construct an anchor point set based on the boundary lines segmented from the loading structure regions. Perform trajectory matching between frames of the anchor point set in the temporal tensor to generate an anchor point tracking map. S3. Perform trajectory extraction operation on the anchor point tracking map, calculate the perturbation amount sequence based on the spatial coordinate change and edge contour change of the anchor point between image frames, and extract structural segments that meet the perturbation conditions from the perturbation amount sequence based on the preset perturbation amount threshold to form a perturbation candidate group. S4. For each structural segment in the perturbation candidate group, apply an encoder to extract the rhythm frequency, deformation amplitude and duration, and construct a perturbation feature set. Combine the image texture changes corresponding to the structural segment to solve the perturbation behavior expression vector, and summarize it into a perturbation vector set. S5. Input the perturbation vector set into the abnormal trend discrimination network, calculate the Euclidean distance with the stored abnormal behavior cluster centers, and generate similarity score results. If the score result is greater than the upper limit of the preset threshold, the corresponding structural segment is marked as an abnormal trajectory; if the score result is within the discrimination range formed by the upper and lower limits of the preset threshold, the disturbed trajectory is included in the set of paths to be confirmed. S6. Perform anomaly annotation on structural segments identified as abnormal trajectories and generate corresponding image frame position identifiers, perturbation behavior expression vectors, and similarity score results to form structured anomaly records; for structural segments in the set of paths to be confirmed, perform image frame backhaul and update the boundary range in the anchor point tracking map. For S6, it should be noted that the anomaly labeling operation is implemented by explicitly marking the corresponding positions of structural segments identified as anomalous trajectories in image frames. This includes: locating the spatial region of the structural segment in each image frame based on its frame index and anchor point trajectory data in the image sequence; overlaying anomaly identification information within this region, such as overlaying color masks, border outlines, or anomaly category labels; and simultaneously storing the perturbation behavior expression vector corresponding to the structural segment together with the similarity score value. This results in a structured anomaly record comprising three elements: image frame position identifier, vector content, and score result, which is used for subsequent retrieval, backtracking, and data export. Image frame backhaul refers to re-inputting the image frames covered by the structural segments in the set of paths to be confirmed into the anchor point analysis module. Based on the latest pixel information and contour boundaries in these image frames, the structural region boundary extraction and anchor point recognition operations are re-executed. By comparing the boundary distribution differences between the initial segmentation results and the backhauled images, the boundary coordinates of the corresponding structural segments in the anchor point set are corrected, thereby updating the time-space path mapping relationship in the anchor point tracking map. This enables the module to have adaptive correction capabilities in the event of slight boundary shifts or unstable boundary recognition, providing a more confident structural basis for subsequent re-judgment or manual review.

[0016] In S1, during the process of a railway freight car passing through the image acquisition area, wheel axle displacement data, image acquisition timestamp data, and image frame number corresponding to each image acquisition time are acquired, and a time positioning index is established based on the relationship between wheel axle displacement and timestamp. Based on the time positioning index, image frames are selected at fixed acquisition intervals during the image acquisition process to form a main image frame set, which constitutes the time axis of the image sequence; Between every two adjacent main image frames, image frames whose acquisition time falls between the two frames are selected to construct a relay interpolation frame set, which is used to supplement the transition state information between the main image frame sets. For acquisition time points not covered by the main image frame set and the relay interpolation frame set, the image interpolation results between adjacent image frames are calculated using an interpolation algorithm to generate an interpolation pseudo-frame set. The interpolation pseudo-frame set is used to construct the missing frame positions in the image time series. The interpolation algorithm includes using a linear interpolation method based on pixel intensity or a bilateral interpolation method. The main image frame set, the relay interpolation frame set, and the interpolation pseudo-frame set are merged in chronological order to generate an image sequence.

[0017] In S2, a structural segmentation operation is performed on each image frame in the image sequence. The structural segmentation model is called, and the corresponding structural region map is output in each image frame based on the edge features of the car door, box door, can lid and tarpaulin belt. Based on the boundary contours of the structural regions described in the structural region diagram, perform a boundary line extraction operation to extract the closed boundary lines of the structural regions and construct a set of boundary lines. Based on the curvature change points and corner points of the boundary lines in the boundary line set, extract discrete coordinate points representing the spatial position of the loading structure to construct an anchor point set; Arrange the anchor point set according to the image frame sequence, establish the correspondence between anchor points in each image frame, and form inter-frame anchor point matching pairs; Based on the consistency of spatial distance and boundary direction between inter-frame anchor point matching pairs, anchor point trajectory sequences are calculated, and an anchor point tracking map is constructed based on all anchor point trajectory sequences to represent the spatial path of the loading structure as it changes over time.

[0018] In S2, the construction of the structure segmentation model includes: extracting structural region images containing vehicle doors, box doors, can lids, and tarpaulin strips based on the labeled image frame sequence, and constructing a training sample set according to the structural region category; calling an image encoding network to perform feature extraction on the input image frames to generate texture feature maps and edge feature maps; inputting the texture feature maps and edge feature maps into a feature fusion module to perform feature aggregation and form a fused feature map; inputting the fused feature map into a region classification module to output a structural region map, where each pixel corresponds to a structural category label; A forward inference operation is performed on the image frame to be processed, and the corresponding structural region map is output. The structural region map is used for boundary line extraction and anchor point set construction.

[0019] In S3, for each anchor point trajectory in the anchor point tracking map, its spatial coordinate sequence in the image frame sequence is extracted, and a spatial displacement sequence is calculated based on the inter-frame coordinate difference. The spatial displacement sequence is used to represent the spatial offset behavior of the anchor point as time changes. For the image frame region corresponding to the spatial displacement sequence, the edge contour line of each anchor point region is extracted, and the edge change sequence is calculated based on the inter-frame position difference of the edge contour line. The edge change sequence is used to characterize the response of the structural region at the boundary.

[0020] In S3, the spatial displacement sequence and the edge change sequence are fused according to the frame position, the disturbance amount calculation operation is performed, and the disturbance amount sequence is constructed based on the fusion result. The disturbance amount sequence reflects the comprehensive disturbance amplitude of the anchor point trajectory between consecutive frames. Extract consecutive frame segments with disturbance values ​​greater than the preset disturbance limit to form an initial set of disturbance segments; Segment merging is performed on adjacent segments in the initial set of disturbance segments to generate a disturbance candidate group, where each segment in the disturbance candidate group corresponds to a continuous disturbance segment in the anchor point trajectory.

[0021] In S4, each structural segment in the perturbation candidate group is obtained, and the number of perturbation duration frames is calculated based on the start frame and end frame of the structural segment in the image sequence. The number of perturbation duration frames is used to describe the span of the structural segment in the time dimension. The anchor point trajectory of the structural segment within the image frame range covered by the duration of the disturbance is input into the encoder. The rhythm frequency is calculated based on the amplitude fluctuation of the trajectory as the frame order changes, and the maximum amplitude of the trajectory within the frame segment is extracted as the deformation amplitude. The rhythm frequency, deformation amplitude, and number of perturbation duration frames are combined to form a perturbation feature group, which is used to characterize the perturbation characteristics of the structural segment in two dimensions: time and motion amplitude. Texture difference calculation is performed on the image frame region corresponding to the number of perturbation duration frames of the structural segment to extract the texture change value of the region within the frame sequence. The texture change value is used to supplement the change information of the structural segment in the image dimension. The perturbation feature set is combined with the texture change value to generate the corresponding perturbation behavior expression vector, and all perturbation behavior expression vectors are summarized into a perturbation vector set according to the structural segment order.

[0022] In S5, an anomaly trend discrimination network is constructed, and the perturbation vector set is set as follows: ;in Indicates the first The perturbation vector corresponding to each structural segment. This represents the total number of perturbation vectors. For the perturbation feature dimension, Standardization and normalization are performed to generate a zero-mean, unit-variance perturbation matrix. ; A set of perturbation feature clusters is constructed based on historical anomaly samples. A radius-adjusted K-means++ initialization strategy is then applied to obtain the set of cluster centers for anomaly behavior. ; each of them Indicates the first Each anomaly category center For the perturbation feature dimension, The number of currently defined abnormal behavior categories; Calculate each The weighted Euclidean distance between the cluster and all anomalous cluster centers is defined by the weight matrix. : For each and Calculate the weighted distance: The weight matrix is ​​generated by inversely reflecting the historical variability of the perturbation features, highlighting the stable feature dimension; wherein the weight matrix It includes the weight values ​​corresponding to each feature dimension in the perturbation vector, totaling... One dimension; Indicates the first The standardized perturbation vector corresponding to each structural segment It is the input data of the vector to be judged; Indicates the first Weight coefficients for each feature dimension; The perturbation vector is represented by the first... The weight coefficients corresponding to each dimension; Representing cluster centers In the Component values ​​in each feature dimension; express With cluster center The weighted Euclidean distance between them is used to measure the degree of similarity between the two. This indicates that for all feature dimensions from 1 to... The values ​​are accumulated to calculate the overall distance. express In the Component values ​​in each feature dimension; Indicates the first The squared difference between the perturbation vector and the cluster center in each feature dimension is used to measure the degree of deviation in that dimension. Weighted Euclidean distance Input a non-linear scoring function and perform piecewise mapping to calculate the similarity score: ;in These represent the upper and lower limits of the threshold range for similarity scores; express and Similarity score between them; Statistics for each Highest similarity score across all cluster centers , ; and will Compared with the preset scoring range If a comparison is made, If so, the corresponding structural segment will be marked as an abnormal trajectory; if If so, it will be included in the set of paths to be confirmed; otherwise, it will be left unprocessed or marked as normal. Based on the scoring trend and weight matrix drift of the set of paths to be confirmed, a scoring offset map is constructed to determine whether local cluster center relocation is triggered, and the scoring results and path marking results are synchronously written into the abnormal trajectory database.

[0023] It should be noted that the fundamental goal of this solution is to solve the problem of the inability to effectively identify the abnormal state of the loading structure of railway freight cars in operation. The key issue is that the disturbance behavior exhibits spatiotemporal drift characteristics in the image sequence, which can easily lead to misjudgment or missed judgment. In image acquisition and sequence construction (S1), image frames are continuously acquired at a fixed frame rate using a linear scan camera to ensure continuity in the time domain. However, due to vehicle speed fluctuations, image noise, or occlusion, there may be unavoidable missing frames or quality fluctuations between actual image frames. Therefore, an interpolation pseudo-frame mechanism is introduced. Based on the spatial gradient and timestamp difference between adjacent main image frames and relay interpolation frames, a transition frame is reconstructed using linear interpolation or bilinear interpolation based on feature mapping. This fills the gaps in the image time series and maintains the temporal integrity of the data. This mechanism ensures that there will be no breakpoint logic problems in subsequent time series modeling. In the loading structure region identification and anchor point construction (S2), the boundary positions of the loading components are extracted from each frame of the image through a structural segmentation model. This model includes the use of U-Net structure or MaskR-CNN architecture. The input is the original image frame, and the output is a pixel-level segmentation mask containing structures such as doors and tarpaulin strips. Its significance is to exclude non-structural regions and retain only key structural response regions. Then, the intersection points of the boundary lines are extracted as anchor points (i.e., key points of structural response). Each anchor point is assigned spatial coordinates in the image frame. Then, trajectory matching is performed on these anchor points on the time axis, that is, finding combinations of anchor points with similar spatial positions and consistent categories in consecutive frames to generate an anchor point tracking map. This process forms a quantifiable structural state evolution path. Third, in the disturbance identification and disturbance vector construction stage (S3, S4), the disturbance amount is calculated for each anchor point trajectory, defined as the difference in spatial coordinates of the anchor point between adjacent frames and the degree of boundary deformation. Based on the disturbance amount sequence and combined with the preset disturbance threshold, local mutation segments are selected as structural segments, i.e., preliminary suspected abnormal areas. Then, a temporal rhythm encoder (including the use of GRU or LSTM networks) is applied to each structural segment to extract three types of indicators: periodic rhythm frequency, local deformation amplitude, and continuous duration. The essence of this process is to encode the discrete trajectory disturbance behavior into a unified disturbance behavior expression vector, i.e., the disturbance vector, to form a vector set as the input basis for anomaly identification. Its value lies in capturing the spatiotemporal features of the disturbance through multi-dimensional features, thereby improving the stability and robustness of identification. In the anomaly trend discrimination mechanism (S5), the system introduces cluster centers as anomaly paradigm references. Each cluster center represents a known anomaly structural response category, which can be established through offline clustering of historical samples (such as K-Means). The center vector is the mean of the perturbation features of the corresponding category. In real-time analysis, the Euclidean distance between each perturbation vector and each cluster center is calculated. Feature weighting coefficients are used to control the influence weight of each dimension. The score value is the weighted minimum of all center distances. If the score value is higher than the upper limit of the threshold, the structural segment is marked as anomaly; if the score value is between the upper and lower limits, it is temporarily listed as pending confirmation. In the anomaly labeling and image backhaul update stage (S6), for trajectories that have been identified as anomalies, the system performs visual labeling on the corresponding image frames using the spatial location of structural fragments, and combines the output perturbation vector with the score value to form a structured anomaly record. For trajectories to be confirmed, image frame backhaul is performed, that is, the relevant images and anchor point trajectories are re-loaded into the system for verification, and the boundary positions in the anchor point tracking map are updated. The purpose is to improve the accuracy of the judgment and reduce the error in the fuzzy interval.

[0024] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting the loading status of railway freight cars based on image recognition, characterized in that, include: S1. During the process of railway freight cars passing through the image acquisition area, a linear array imaging device is used to acquire car image frames at a constant frame rate. An image sequence is constructed based on wheel axle displacement, acquisition timestamp, and frame number. The image sequence includes the main image frame, relay interpolation frames, and interpolation pseudo-frames. S2. Perform loading structure region segmentation operation on each image frame in the image sequence. Based on the structural segmentation model of the segmentation operation, identify the loading structure regions corresponding to the vehicle door, box door, can lid, and tarpaulin belt. Construct an anchor point set based on the boundary lines segmented from the loading structure regions. Perform trajectory matching between frames of the anchor point set in the temporal tensor to generate an anchor point tracking map. S3. Perform trajectory extraction operation on the anchor point tracking map, calculate the perturbation amount sequence based on the spatial coordinate change and edge contour change of the anchor point between image frames, and extract structural segments that meet the perturbation conditions from the perturbation amount sequence based on the preset perturbation amount threshold to form a perturbation candidate group. S4. For each structural segment in the perturbation candidate group, apply an encoder to extract the rhythm frequency, deformation amplitude and duration, and construct a perturbation feature set. Combine the image texture changes corresponding to the structural segment to solve the perturbation behavior expression vector, and summarize it into a perturbation vector set. S5. Input the perturbation vector set into the abnormal trend discrimination network, calculate the Euclidean distance with the stored abnormal behavior cluster centers, and generate similarity score results. If the score is greater than the preset threshold, the corresponding structural segment will be marked as an abnormal trajectory. If the scoring result is within the discrimination range formed by the upper and lower limits of the preset threshold, the disturbance trajectory will be included in the set of paths to be confirmed. S6. Perform anomaly annotation on structural segments identified as abnormal trajectories and generate corresponding image frame position identifiers, disturbance behavior expression vectors, and similarity scores to form structured anomaly records; for structural segments in the set of paths to be confirmed, perform image frame backhaul and update the boundary range in the anchor point tracking map.

2. The method for detecting the loading status of railway freight cars based on image recognition according to claim 1, characterized in that: In S1, during the process of a railway freight car passing through the image acquisition area, wheel axle displacement data, image acquisition timestamp data, and image frame number corresponding to each image acquisition time are acquired, and a time positioning index is established based on the relationship between wheel axle displacement and timestamp. Based on the time positioning index, image frames are selected at fixed acquisition intervals during the image acquisition process to form a main image frame set, which constitutes the time axis of the image sequence; Between every two adjacent main image frames, image frames whose acquisition time falls between the two frames are selected to construct a relay interpolation frame set, which is used to supplement the transition state information between the main image frame sets. For acquisition time points not covered by the main image frame set and the relay interpolation frame set, the image interpolation results between adjacent image frames are calculated using an interpolation algorithm to generate an interpolation pseudo-frame set. The interpolation pseudo-frame set is used to construct the missing frame positions in the image time series. The interpolation algorithm includes using a linear interpolation method based on pixel intensity. The main image frame set, the relay interpolation frame set, and the interpolation pseudo-frame set are merged in chronological order to generate an image sequence.

3. The method for detecting the loading status of railway freight cars based on image recognition according to claim 2, characterized in that: In S2, a structural segmentation operation is performed on each image frame in the image sequence. The structural segmentation model is called, and the corresponding structural region map is output in each image frame based on the edge features of the car door, box door, can lid and tarpaulin belt. Based on the boundary contours of the structural regions described in the structural region diagram, perform a boundary line extraction operation to extract the closed boundary lines of the structural regions and construct a set of boundary lines. Based on the curvature change points and corner points of the boundary lines in the boundary line set, extract discrete coordinate points representing the spatial position of the loading structure to construct an anchor point set; Arrange the anchor point set according to the image frame sequence, establish the correspondence between anchor points in each image frame, and form inter-frame anchor point matching pairs; Based on the consistency of spatial distance and boundary direction between inter-frame anchor point matching pairs, anchor point trajectory sequences are calculated, and an anchor point tracking map is constructed based on all anchor point trajectory sequences to represent the spatial path of the loading structure as it changes over time.

4. The method for detecting the loading status of railway freight cars based on image recognition according to claim 3, characterized in that: In S2, the construction of the structure segmentation model includes: extracting structural region images containing vehicle doors, box doors, can lids, and tarpaulin strips based on the labeled image frame sequence, and constructing a training sample set according to the structural region category; calling an image encoding network to perform feature extraction on the input image frames to generate texture feature maps and edge feature maps; inputting the texture feature maps and edge feature maps into a feature fusion module to perform feature aggregation and form a fused feature map; inputting the fused feature map into a region classification module to output a structural region map, where each pixel corresponds to a structural category label; A forward inference operation is performed on the image frame to be processed, and the corresponding structural region map is output. The structural region map is used for boundary line extraction and anchor point set construction.

5. The method for detecting the loading status of railway freight cars based on image recognition according to claim 4, characterized in that: In S3, for each anchor point trajectory in the anchor point tracking map, its spatial coordinate sequence in the image frame sequence is extracted, and a spatial displacement sequence is calculated based on the inter-frame coordinate difference. The spatial displacement sequence is used to represent the spatial offset behavior of the anchor point as time changes. For the image frame region corresponding to the spatial displacement sequence, the edge contour line of each anchor point region is extracted, and the edge change sequence is calculated based on the inter-frame position difference of the edge contour line. The edge change sequence is used to characterize the response of the structural region at the boundary.

6. The method for detecting the loading status of railway freight cars based on image recognition according to claim 5, characterized in that: In S3, the spatial displacement sequence and the edge change sequence are fused according to the frame position, the disturbance amount calculation operation is performed, and the disturbance amount sequence is constructed based on the fusion result. The disturbance amount sequence reflects the comprehensive disturbance amplitude of the anchor point trajectory between consecutive frames. Extract consecutive frame segments with disturbance values ​​greater than the preset disturbance limit to form an initial set of disturbance segments; Segment merging is performed on adjacent segments in the initial set of disturbance segments to generate a disturbance candidate group, where each segment in the disturbance candidate group corresponds to a continuous disturbance segment in the anchor point trajectory.

7. The method for detecting the loading status of railway freight cars based on image recognition according to claim 6, characterized in that: In S4, each structural segment in the perturbation candidate group is obtained, and the number of perturbation duration frames is calculated based on the start frame and end frame of the structural segment in the image sequence. The number of perturbation duration frames is used to describe the span of the structural segment in the time dimension. The anchor point trajectory of the structural segment within the image frame range covered by the duration of the disturbance is input into the encoder. The rhythm frequency is calculated based on the amplitude fluctuation of the trajectory as the frame order changes, and the maximum amplitude of the trajectory within the frame segment is extracted as the deformation amplitude. The rhythm frequency, deformation amplitude, and number of frames of disturbance duration are combined to form a disturbance feature group; Perform texture difference calculation on the image frame region corresponding to the number of frames of perturbation duration for the structural fragment, and extract the texture change value of the region within the frame sequence; The perturbation feature set is combined with the texture change value to generate the corresponding perturbation behavior expression vector, and all perturbation behavior expression vectors are summarized into a perturbation vector set according to the structural segment order.

8. The method for detecting the loading status of railway freight cars based on image recognition according to claim 7, characterized in that: In S5, an anomaly trend discrimination network is constructed, and the perturbation vector set is set as follows: ;in Indicates the first The perturbation vector corresponding to each structural segment. This represents the total number of perturbation vectors. For the perturbation feature dimension, Standardization and normalization are performed to generate a zero-mean, unit-variance perturbation matrix. ; A set of perturbation feature clusters is constructed based on historical anomaly samples. A radius-adjusted K-means++ initialization strategy is then applied to obtain a set of cluster centers for anomaly behavior. ; each of them Indicates the first Each anomaly category center For the perturbation feature dimension, The number of currently defined abnormal behavior categories; Calculate each The weighted Euclidean distance between the cluster and all anomalous cluster centers is defined by the weight matrix. : For each and Calculate the weighted distance: The weight matrix is ​​used for weighting. It includes the weight values ​​corresponding to each feature dimension in the perturbation vector, totaling... One dimension; Indicates the first The standardized perturbation vector corresponding to each structural segment It is the input data of the vector to be judged; Indicates the first Weight coefficients for each feature dimension; The perturbation vector is represented by the first... The weight coefficients corresponding to each dimension; Representing cluster centers In the Component values ​​in each feature dimension; express With cluster center The weighted Euclidean distance between them; express In the Component values ​​in each feature dimension; Indicates the first The squared difference between the perturbation vector and the cluster center in each feature dimension; Weighted Euclidean distance Input a non-linear scoring function and perform piecewise mapping to calculate the similarity score: ;in These represent the upper and lower limits of the threshold range for similarity scores; express and Similarity score between them; Statistics for each Highest similarity score across all cluster centers , ; and will Compared with the preset scoring range If a comparison is made, If so, the corresponding structural segment will be marked as an abnormal trajectory; if If it is found to be true, it will be included in the set of paths to be confirmed; otherwise, it will be left unprocessed or marked as normal.

Images