Railway container loading state intelligent identification method and system

Abstract

This invention discloses an intelligent identification method and system for railway container loading status, relating to the field of railway intelligent detection technology. The method includes acquiring multi-view original images of containers on a moving train, and performing dynamic perspective correction and motion blur suppression on the original images based on track geometric parameters and the real-time train speed to obtain a geometrically standardized front view image of the container. Target detection is performed on the geometrically standardized container front view image, outputting candidate container bounding boxes and container type categories. An undirected topological graph is constructed based on the projection positions of the candidate containers on the flatcar. A graph neural network is used to infer the loading configuration type of each container and the loading compliance judgment result of the entire flatcar. Based on the flatcar chassis area corresponding to each container in the geometrically standardized container front view image, the frame sinking amount and wheel axle spacing compression ratio are extracted as physical features, combined with the deep semantic features extracted from the corresponding container image area.

Description

Technical Field

[0001] This invention relates to the field of intelligent railway inspection technology, and in particular to an intelligent identification method and system for the loading status of railway containers. Background Technology

[0002] Railway intelligent inspection technology refers to a technical system that integrates advanced technologies such as computer vision, artificial intelligence, sensor networks, edge computing, and big data analysis to achieve automated, real-time, and high-precision status perception and intelligent diagnosis of railway infrastructure, mobile equipment, and operating environment. This technology automatically collects data during train operation or line inspection by deploying multi-source sensing devices such as high-definition cameras, LiDAR, millimeter-wave radar, and vibration sensors. Utilizing methods such as deep learning, image processing, pattern recognition, and physical modeling, it achieves intelligent identification, assessment, and early warning of defects in key components, loading anomalies, equipment malfunctions, and safety hazards. This replaces traditional manual inspections, improving the safety, efficiency, and intelligent operation and maintenance level of railway transportation.

[0003] Traditional railway container loading status inspection mainly relies on on-site personnel's visual inspection or simple tool measurement, which is difficult to cover high-speed passing trains, has blind spots, and cannot achieve all-weather, all-section automated monitoring. It is prone to misjudgment in complex scenarios such as dirty, deformed, obstructed, or changing lighting conditions, and cannot meet the real-time detection needs of trains in motion. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an intelligent identification method for the loading status of railway containers, which solves the problems of traditional railway container loading status inspection mainly relying on on-site personnel's visual inspection or simple tool measurement, which is difficult to cover high-speed passing trains, has blind spots, cannot achieve all-weather, all-section automated monitoring, is prone to misjudgment in complex scenarios such as dirty, deformed, obstructed or light changes, and cannot meet the real-time detection needs of trains in motion.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for intelligent identification of the loading status of railway containers, comprising: Multi-view original images of containers on a moving train are acquired, and dynamic perspective correction and motion blur suppression are performed on the original images based on track geometry parameters and real-time train speed to obtain geometrically normalized front view images of containers. Target detection is performed on the geometrically standardized front view image of the container, and candidate container bounding boxes and container type categories are output. An undirected topological graph is constructed based on the projection position of the candidate containers on the flatcar. The loading configuration type of each container and the loading compliance judgment result of the entire flatcar are obtained through graph neural network reasoning. Based on the flatcar chassis area corresponding to each container in the geometrically normalized front view image of the container, the chassis sinking amount and wheel axle spacing compression ratio are extracted as physical features, and combined with the deep semantic features extracted from the corresponding container image area, the empty and heavy status of each container is determined after fusion. High-resolution sub-images are cropped from the preset area of ​​the door corner fitting according to the box type. Sub-pixel-level precise positioning of the keyhole center is achieved through heat map regression and two-dimensional Gaussian fitting. The locking state category is identified based on the geometric deviation between the precise positioning result and the standard lock model. The confidence levels of loading configuration type, loading compliance judgment result, empty / loaded status and locked status category are weighted and fused to generate a structured loading status report; The structured loading status report is transmitted to the railway transport management system in real time.

[0007] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the steps include: performing target detection on a geometrically standardized front view image of a container, outputting candidate container bounding boxes and container type categories, constructing an undirected topological graph based on the projection positions of the candidate containers on the flatcar, and obtaining the loading configuration type of each container and the loading compliance judgment result of the entire flatcar through graph neural network reasoning. The specific steps are as follows: The geometrically normalized front view image of the container is input into the target detection network based on the improved EfficientRep structure. Forward propagation is performed to obtain a set of bounding boxes of N candidate containers. Each bounding box contains four parameters: center x-coordinate, center y-coordinate, width and height, as well as the corresponding container type label. A two-dimensional plane coordinate system is established with the center line of the flatbed as the reference. The center point of each bounding box is mapped to this coordinate system to obtain the node position. Traverse all node pairs. If the Euclidean distance between any two nodes is less than a preset distance threshold, and the absolute value of the difference between the direction angles derived from the aspect ratios of their respective bounding boxes is less than a preset angle tolerance, then establish an undirected edge between the two nodes. Construct a graph structure where the number of nodes equals the number of candidate boxes. The initial feature vector of each node is composed of the center x-coordinate, center y-coordinate, width, height, and the embedding encoding of the box type of the corresponding bounding box. The edge set is determined by the adjacency condition. The feature vector of each edge contains the Euclidean distance and the difference in orientation angle between the two connected nodes. The constructed graph structure is input into a graph neural network containing three graph convolutional layers. The node feature updates in each layer follow a message-passing mechanism, specifically expressed as follows: ; in, Indicates the first The first in the layer The feature vector of each node Represents a node The set of neighboring nodes, Neighboring nodes In the The feature vector of the layer, For connecting nodes and The edge feature vectors, This represents a vector concatenation operation. For the first Layer-learnable weight matrix, It is a non-linear activation function; After three layers of graph convolution operations, the final node features are input into the fully connected classification head, which outputs the loading configuration type of each box. The loading configuration type includes three cases: single box, double boxes side by side, or stacked boxes. Meanwhile, in accordance with the constraints on flatcar loading layout in the railway industry standard TB / T 3571, the loading configuration type of all boxes and their positions on the flatcar are logically verified. If there are violations of minimum spacing requirements, excessive lateral offset of the center of gravity, or non-permitted stacking combinations, the entire flatcar loading is deemed non-compliant.

[0008] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the method involves: extracting the chassis area corresponding to each container in the geometrically normalized front view image of the flatcar, using the frame sinking amount and wheel axle spacing compression ratio as physical features, and combining them with the deep semantic features extracted from the corresponding container image area to determine the empty / load status of each container. The specific steps are as follows: Based on the horizontal projection range of the bounding box output by the target detection in the geometrically normalized image, a rectangular sub-image covering the flatcar chassis and track area is extracted downwards. Edge detection and line fitting are performed on the sub-image to extract the lower edges of the left and right side beams respectively, and the average distance between the two fitted lines in the vertical direction is calculated as the frame sinking amount. Detect the center point of the wheel axle in the base plate diagram, arrange them in order from left to right, calculate the actual distance between the center points of adjacent wheel axles, and take the arithmetic mean of all adjacent distances as the measured wheel axle distance; Divide the measured wheel-axle distance by the standard wheel-axle distance of the same type of flatcar under no-load conditions to obtain the wheel-axle distance compression ratio. The frame sag and the wheel axle spacing compression ratio are combined to form a two-dimensional physical feature vector; The complete box image region is then input into the pre-trained ResNet-34 backbone network, and the output of the global average pooling layer is taken as a 512-dimensional deep semantic feature vector. The two-dimensional physical feature vector is concatenated with the 512-dimensional deep semantic feature vector to form a joint feature vector, which is then input into a discriminant network containing two fully connected layers, and outputs a binary classification result of empty or full boxes. Among them, the chassis undersinking reflects the degree of elastic deformation of the car body caused by the load, and the wheel axle spacing compression ratio reflects the geometric contraction effect of the bogie after being compressed. Together, they constitute a physical observable measurement directly related to the load, which complements the purely visual semantic features.

[0009] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the steps include: cropping a high-resolution sub-image from a preset area of ​​the container door corner piece according to the container type, achieving sub-pixel-level precise positioning of the lock hole center through heatmap regression and two-dimensional Gaussian fitting, and identifying the locking status category based on the geometric deviation between the precise positioning result and the standard lock model. Based on the container type, find the theoretical position of the corresponding container door corner fitting in the ISO 6346 standard, and then crop a high-resolution sub-image of a fixed size from the geometrically normalized front view image of the container with that theoretical position as the center. Input the high-resolution subgraph into the stacked hourglass network and output a probability heatmap of the keyhole center location; The point with the maximum response in the location heatmap is used as the coarse location coordinate. Define a fixed-size neighborhood window around the coarse positioning coordinates and extract all heatmap response values ​​within the window; A two-dimensional Gaussian distribution function is fitted to the heatmap response values ​​within the window. The specific expression is as follows: ; in, Indicates the heatmap on coordinates The response value at that location, Peak amplitude, and These are the coordinates of the Gaussian distribution center, i.e., the sub-pixel level precise positioning coordinates of the keyhole center. and These are the standard deviations in the horizontal and vertical directions, respectively. The above equations are solved using the nonlinear least squares method to obtain the optimal parameters. and ; Will and Mapping back to the original image coordinate system yields the center point of the keyhole in the world coordinate system. The keyhole contour is extracted based on the center point. The root mean square distance from the contour point to the ideal circle is calculated as the roundness error. At the same time, the relative deviation between the diameter of the outer circle of the contour and the diameter of the standard keyhole is calculated as the diameter deviation. The roundness error and diameter deviation are input into the support vector machine classifier, which outputs the locking state category. The locking state category includes four situations: normal locking, not inserted, half-locked, and foreign object occlusion.

[0010] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the steps for generating a structured loading status report by weighted fusion of the confidence levels of loading configuration type, loading compliance judgment result, empty / loaded status, and locking status categories are as follows: The softmax probability value of each container loading configuration type is obtained from the output of the graph neural network classification head and used as the loading configuration confidence. Obtain a Boolean compliance flag from the compliance verification module. If the compliance meets the rules, the confidence level is 1.0; otherwise, it is 0.0. The probability value of a full box is obtained from the empty-weight discrimination network and used as the confidence level of the empty-weight state. The output values ​​of the decision functions for each category are obtained from the locked state classifier and then normalized to serve as the locked state confidence. The overall confidence level is calculated as a weighted average of the four confidence levels. The weighting coefficients are dynamically adjusted based on the historical false alarm rate to ensure that high-risk items have higher weights. Combining the results of optical character recognition of container numbers, statistical values ​​of the number of containers, determination of the empty and heavy status of each container, category of locking status of each container, loading compliance mark and overall confidence level, the data is encapsulated into a structured loading status report according to a predefined JSON Schema.

[0011] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the steps of acquiring multi-view original images of containers on a moving train, and performing dynamic perspective correction and motion blur suppression on the original images based on track geometric parameters and real-time train speed to obtain a geometrically standardized front view image of the container are as follows: Train passing image acquisition cameras are deployed on the left, right and above the track respectively. The cameras are area array cameras, line array cameras or combinations thereof. Multiple cameras are strictly synchronized through hardware trigger signals, and the vertical resolution is not less than 2048. By measuring the instantaneous speed of the train in real time, the camera exposure time or line frequency parameters are dynamically adjusted according to the speed to ensure that the length of the blur caused by the train's movement in the image does not exceed a preset threshold. Using the pre-completed camera calibration results, obtain the intrinsic and extrinsic parameter matrices for each camera; Based on the known track gauge of 1435 mm and the standard height of the flatcar bottom from the track surface, a homography transformation matrix from the world coordinate system to the unified frontal plane is constructed. The homography transformation matrix is ​​applied to the raw images captured by each camera to project images from different perspectives onto the same frontal plane with the center line of the track as the reference, eliminating perspective distortion caused by differences in shooting angles and generating a geometrically normalized frontal view image of the container.

[0012] As a preferred embodiment of the intelligent identification method for railway container loading status described in this invention, the specific steps for transmitting the structured loading status report to the railway transportation management system in real time are as follows: Structured loading status reports are pushed to edge computing servers deployed in the freight yard via railway-specific industrial Ethernet; The edge computing server performs integrity checks on the received reports, including whether required fields exist, whether the confidence value is within the range of [0,1], and whether the timestamp is reasonable. After verification, the report is appended with a unique device identifier and a collection timestamp accurate to milliseconds, and the report content is encrypted using the national cryptographic SM4 algorithm; The encrypted data is uploaded to the China State Railway Group TMS cloud platform via a data security gateway that complies with railway network security standards; The TMS cloud platform receives and decrypts the report, parsing its various fields. If the loading compliance flag is abnormal, or if the locking status of any container is not in the normal locking category, the system will automatically generate a dispatch intervention instruction, including suspending the formation, notifying the site for inspection or re-weighing, and simultaneously creating a safety warning work order and pushing it to the relevant station duty terminal and freight dispatch center.

[0013] Secondly, the present invention provides an intelligent identification system for the loading status of railway containers, comprising: The system includes an image acquisition and correction module, a box topology modeling module, an empty / load status discrimination module, a locked status recognition module, a multi-source confidence fusion module, and a report generation and transmission module. The image acquisition and correction module is used to synchronously acquire original images of the container through a multi-view industrial camera during the train's movement, and dynamically adjust the exposure time in combination with track geometry parameters and the real-time speed of the train. It also applies homography transformation to eliminate perspective distortion and motion blur, and outputs a geometrically standardized front view image of the container. The box topology modeling module is used to perform target detection on geometrically normalized images, obtain candidate box bounding boxes and box type categories, construct an undirected topology graph with the projection position of the box on the flatcar as nodes and spatial proximity as edges, infer the loading configuration type of each box through a three-layer graph convolutional network, and verify the layout compliance of the entire flatcar according to railway loading specifications. The empty / load status discrimination module is used to extract the corresponding flatbed image of each box, extract the frame sinking amount and wheel axle spacing compression ratio as physical deformation features, and extract deep semantic features from the box image area. The two types of features are then concatenated and input into the discrimination network to output the classification result of empty or loaded box. The locking status recognition module is used to locate the corner fitting area of ​​the door according to the box type, crop the high-resolution sub-image and obtain the coarse positioning of the lock hole through heat map regression, and then achieve sub-pixel level precise positioning through two-dimensional Gaussian fitting. Based on the precise positioning point, the roundness error and diameter deviation of the lock hole outline are calculated, and finally the locking status category is identified. The multi-source confidence fusion module is used to obtain the confidence levels of loading configuration type, loading compliance judgment, empty / loaded status and locked status respectively, perform weighted average according to preset weights to generate comprehensive confidence level, and integrate information such as container number, quantity and status label to form structured data; The report generation and transmission module is used to attach equipment identifiers and timestamps to the structured loading status report, and after integrity verification and encryption with the national cryptographic standard SM4, upload it to the transportation management system through the railway dedicated network. When an abnormal status is detected, it automatically triggers dispatch intervention instructions and safety warning work orders.

[0014] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the intelligent identification method for railway container loading status as described in the first aspect of the present invention.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the intelligent identification method for railway container loading status as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: Through core technologies such as multi-view image dynamic correction, graph neural network topology reasoning, physical-semantic feature fusion discrimination, sub-pixel level keyhole positioning, and multi-source confidence weighted fusion, it achieves fully automatic, high-precision, and real-time intelligent identification of the loading status of railway containers, effectively solving the problems of low efficiency, high missed detection rate, and inability to quantify and evaluate traditional manual inspections; the system can simultaneously judge the compliance of container configuration, empty / load status, and locking integrity, improve the level of railway freight safety supervision, reduce the risk of derailment, off-center loading, or container detachment caused by abnormal loading, and provide structured data support for transportation scheduling, promoting the upgrading of railway container transportation towards intelligence, digitalization, and standardization. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the intelligent identification method for railway container loading status in Example 1.

[0019] Figure 2 This is a schematic diagram of the intelligent identification system for railway container loading status in Example 1. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, referring to Figure 1 and Figure 2 As one embodiment of the present invention, this embodiment provides a method for intelligent identification of the loading status of railway containers, including the following steps: S1. Collect multi-view original images of containers on a moving train, and perform dynamic perspective correction and motion blur suppression on the original images based on track geometry parameters and real-time train speed to obtain a geometrically standardized front view image of the container.

[0024] Furthermore, train passing image acquisition cameras are deployed on the left, right and above the track, respectively. The cameras are area array cameras, line array cameras or combinations thereof. Multiple cameras are strictly synchronized through hardware trigger signals, and the vertical resolution is not less than 2048. By measuring the instantaneous speed of the train in real time, the camera exposure time or line frequency parameters are dynamically adjusted according to the speed to ensure that the length of the blur caused by the train movement in the image does not exceed the preset threshold. Industrial cameras are deployed on the left, right and above the track respectively. The three cameras are strictly synchronized through hardware trigger signals, with a frame rate of not less than 30 frames per second and a resolution of not less than 3840×2160. The instantaneous speed of the train is measured in real time by millimeter-wave radar, and the camera exposure time is dynamically adjusted according to the speed to ensure that the length of the blur caused by the train's movement in the image does not exceed 2 pixels. Using the pre-completed camera calibration results, obtain the intrinsic and extrinsic parameter matrices for each camera; Based on the known track gauge of 1435 mm and the standard height of the flatcar bottom from the track surface, a homography transformation matrix from the world coordinate system to the unified frontal plane is constructed. The homography transformation matrix is ​​applied to the raw images captured by each camera to project images from different perspectives onto the same frontal plane with the center line of the track as the reference, eliminating perspective distortion caused by differences in shooting angles and generating a geometrically normalized frontal view image of the container.

[0025] It should be noted that by deploying high-resolution synchronous cameras in multiple directions along the track, dynamically adjusting exposure parameters based on the real-time speed of the train, and then using homography transformation based on track geometry priors to perform unified projection correction on multi-view images, the image distortion problem caused by motion blur and shooting angle differences when the train passes at high speed is effectively overcome, thereby improving the input quality of subsequent visual analysis and the robustness of the system.

[0026] S2. Perform target detection on the geometrically standardized front view image of the container, output the candidate container bounding box and container type category, and construct an undirected topology graph based on the projection position of the candidate container on the flatcar. Use graph neural network reasoning to obtain the loading configuration type of each container and the loading compliance judgment result of the entire flatcar.

[0027] Furthermore, the geometrically normalized front view image of the container is input into the target detection network based on the improved EfficientRep structure, and forward propagation is performed to obtain a set of bounding boxes of N candidate containers. Each bounding box contains four parameters: center x-coordinate, center y-coordinate, width and height, as well as the corresponding container type category label. A two-dimensional plane coordinate system is established with the center line of the flatbed as the reference. The center point of each bounding box is mapped to this coordinate system to obtain the node position. Traverse all node pairs. If the Euclidean distance between any two nodes is less than a preset distance threshold, and the absolute value of the difference between the direction angles derived from the aspect ratios of their respective bounding boxes is less than a preset angle tolerance, then establish an undirected edge between the two nodes. Construct a graph structure where the number of nodes equals the number of candidate boxes. The initial feature vector of each node is composed of the center x-coordinate, center y-coordinate, width, height, and the embedding encoding of the box type of the corresponding bounding box. The edge set is determined by the adjacency condition. The feature vector of each edge contains the Euclidean distance and the difference in orientation angle between the two connected nodes. The constructed graph structure is input into a graph neural network containing three graph convolutional layers. The node feature updates in each layer follow a message-passing mechanism, specifically expressed as follows: ; in, Indicates the first The first in the layer The feature vector of each node Represents a node The set of neighboring nodes, Neighboring nodes In the The feature vector of the layer, For connecting nodes and The edge feature vectors, This represents a vector concatenation operation. For the first Layer-learnable weight matrix, It is a non-linear activation function; After three layers of graph convolution operations, the final node features are input into the fully connected classification head, which outputs the loading configuration type of each box. The loading configuration type includes three cases: single box, double boxes side by side, or stacked boxes. Meanwhile, in accordance with the constraints on flatcar loading layout in the railway industry standard TB / T 3571, the loading configuration type of all boxes and their positions on the flatcar are logically verified. If there are violations of minimum spacing requirements, excessive lateral offset of the center of gravity, or non-permitted stacking combinations, the entire flatcar loading is deemed non-compliant.

[0028] It should be noted that by using an improved target detection network combined with an undirected topology graph constructed based on physical location relationships, and introducing a graph neural network for context-aware reasoning, not only can the loading form of a single container be accurately identified, but the layout can also be judged from the global perspective of the entire flatcar to determine whether it complies with railway safety regulations. This solves the technical bottleneck of traditional target detection methods that ignore spatial constraints between containers and are difficult to assess overall compliance.

[0029] S3. Based on the geometrically standardized front view image of the container, extract the flatcar chassis area corresponding to each container, extract the frame sinking amount and wheel axle spacing compression ratio as physical features, and combine them with the deep semantic features extracted from the corresponding container image area to determine the empty and heavy status of each container.

[0030] Furthermore, based on the horizontal projection range of the bounding box output by the target detection in the geometrically normalized image, a rectangular sub-image covering the flatcar chassis and track area is extracted downwards. Edge detection and line fitting are performed on the sub-image to extract the lower edges of the left and right side beams respectively, and the average distance between the two fitted lines in the vertical direction is calculated as the frame sinking amount. Detect the center point of the wheel axle in the base plate diagram, arrange them in order from left to right, calculate the actual distance between the center points of adjacent wheel axles, and take the arithmetic mean of all adjacent distances as the measured wheel axle distance; Divide the measured wheel-axle distance by the standard wheel-axle distance of the same type of flatcar under no-load conditions to obtain the wheel-axle distance compression ratio. The frame sag and the wheel axle spacing compression ratio are combined to form a two-dimensional physical feature vector; The complete box image region is then input into the pre-trained ResNet-34 backbone network, and the output of the global average pooling layer is taken as a 512-dimensional deep semantic feature vector. The two-dimensional physical feature vector is concatenated with the 512-dimensional deep semantic feature vector to form a joint feature vector, which is then input into a discriminant network containing two fully connected layers, and outputs a binary classification result of empty or full boxes. Among them, the chassis undersinking reflects the degree of elastic deformation of the car body caused by the load, and the wheel axle spacing compression ratio reflects the geometric contraction effect of the bogie after being compressed. Together, they constitute a physical observable measurement directly related to the load, which complements the purely visual semantic features.

[0031] It should be noted that the mechanical response features such as the chassis sinking and wheel axle spacing compression ratio extracted from the chassis area are fused with the deep semantic features of the box appearance for discrimination. This breaks through the limitation of relying solely on visual appearance information to judge the empty and heavy state, enabling the system to maintain a high-precision load status recognition capability even under complex working conditions such as dirt, obstruction or changes in lighting on the box surface.

[0032] S4. According to the box type, a high-resolution sub-image is cropped in the preset area of ​​the box door corner piece. The center of the key hole is accurately located at the sub-pixel level through heat map regression and two-dimensional Gaussian fitting. The locking state category is identified based on the geometric deviation between the accurate positioning result and the standard lock model.

[0033] Furthermore, based on the container type, the theoretical position of the corresponding container door corner fitting in the ISO 6346 standard is queried, and a high-resolution sub-image of a fixed size is cropped from the geometrically normalized front view image of the container with the theoretical position as the center. Input the high-resolution subgraph into the stacked hourglass network and output a probability heatmap of the keyhole center location; The point with the maximum response in the location heatmap is used as the coarse location coordinate. Define a fixed-size neighborhood window around the coarse positioning coordinates and extract all heatmap response values ​​within the window; A two-dimensional Gaussian distribution function is fitted to the heatmap response values ​​within the window. The specific expression is as follows: ; in, Indicates the heatmap on coordinates The response value at that location, Peak amplitude, and These are the coordinates of the Gaussian distribution center, i.e., the sub-pixel level precise positioning coordinates of the keyhole center. and These are the standard deviations in the horizontal and vertical directions, respectively. The above equations are solved using the nonlinear least squares method to obtain the optimal parameters. and ; Will and Mapping back to the original image coordinate system yields the center point of the keyhole in the world coordinate system. The keyhole contour is extracted based on the center point. The root mean square distance from the contour point to the ideal circle is calculated as the roundness error. At the same time, the relative deviation between the diameter of the outer circle of the contour and the diameter of the standard keyhole is calculated as the diameter deviation. The roundness error and diameter deviation are input into the support vector machine classifier, which outputs the locking state category. The locking state category includes four situations: normal locking, not inserted, half-locked, and foreign object occlusion.

[0034] It should be noted that by combining heatmap regression for initial localization with two-dimensional Gaussian fitting for refinement, sub-pixel-level positioning accuracy of the keyhole center was achieved. Furthermore, the degree of locking anomaly was quantified based on geometric deviation, effectively avoiding the failure problem of traditional edge detection methods in rust, oil, or low-contrast scenarios, and improving the reliability and adaptability of locking status recognition.

[0035] S5. The confidence levels of the loading configuration type, loading compliance judgment result, empty / loaded status and locked status categories are weighted and integrated to generate a structured loading status report.

[0036] Furthermore, the softmax probability value of each container loading configuration type is obtained from the output of the graph neural network classification head, and used as the loading configuration confidence. Obtain a Boolean compliance flag from the compliance verification module. If the compliance meets the rules, the confidence level is 1.0; otherwise, it is 0.0. The probability value of a full box is obtained from the empty-weight discrimination network and used as the confidence level of the empty-weight state. The output values ​​of the decision functions for each category are obtained from the locked state classifier and then normalized to serve as the locked state confidence. The overall confidence level is calculated as a weighted average of the four confidence levels. The weighting coefficients are dynamically adjusted based on the historical false alarm rate to ensure that high-risk items have higher weights. Combining the results of optical character recognition of container numbers, statistical values ​​of the number of containers, determination of the empty and heavy status of each container, category of locking status of each container, loading compliance mark and overall confidence level, the data is encapsulated into a structured loading status report according to a predefined JSON Schema.

[0037] It should be noted that a confidence-weighted fusion mechanism is introduced for the state discrimination results of multiple dimensions. The weights of each modality are dynamically adjusted based on historical performance, so that high-risk items take the lead in decision-making. This not only enhances the system's sensitivity to abnormal states, but also provides clear, traceable, and operable data support for subsequent business systems through structured report output.

[0038] S6. Transmit the structured loading status report to the railway transportation management system in real time.

[0039] Furthermore, structured loading status reports are pushed to edge computing servers deployed in the freight yard via a dedicated railway industrial Ethernet network; The edge computing server performs integrity checks on the received reports, including whether required fields exist, whether the confidence value is within the range of [0,1], and whether the timestamp is reasonable. After verification, the report is appended with a unique device identifier and a collection timestamp accurate to milliseconds, and the report content is encrypted using the national cryptographic SM4 algorithm; The encrypted data is uploaded to the China State Railway Group TMS cloud platform via a data security gateway that complies with railway network security standards; The TMS cloud platform receives and decrypts the report, parsing its various fields. If the loading compliance flag is abnormal, or if the locking status of any container is not in the normal locking category, the system will automatically generate a dispatch intervention instruction, including suspending the formation, notifying the site for inspection or re-weighing, and simultaneously creating a safety warning work order and pushing it to the relevant station duty terminal and freight dispatch center.

[0040] It should be noted that by performing local verification and encryption through edge computing nodes and uploading structured reports to the transportation management cloud platform through a dedicated railway safety channel, a closed-loop linkage from perception to response is achieved. When loading anomalies or locking failures are detected, the system can automatically trigger scheduling intervention and early warning processes, effectively shortening emergency response time and improving the safety management level of the entire railway freight process.

[0041] This embodiment also provides a railway container loading status intelligent identification system, including: The system includes an image acquisition and correction module, a box topology modeling module, an empty / load status discrimination module, a locked status recognition module, a multi-source confidence fusion module, and a report generation and transmission module. The image acquisition and correction module is used to synchronously acquire original images of containers through multi-view industrial cameras during train travel, and dynamically adjust the exposure time by combining track geometry parameters and real-time train speed. It applies homography transformation to eliminate perspective distortion and motion blur, and outputs a geometrically standardized front view image of the container. The box topology modeling module is used to perform object detection on geometrically normalized images, obtain candidate box bounding boxes and box type categories, construct an undirected topology graph with the projection position of the box on the flatcar as nodes and spatial proximity as edges, infer the loading configuration type of each box through a three-layer graph convolutional network, and verify the layout compliance of the entire flatcar according to railway loading specifications. The empty / load status discrimination module is used to extract the corresponding flatbed image of each box, extract the frame sinking amount and wheel axle spacing compression ratio as physical deformation features, and extract deep semantic features from the box image area. The two types of features are concatenated and input into the discrimination network to output the classification result of empty or loaded box. The lock status recognition module is used to locate the corner fitting area of ​​the door according to the box type, crop the high-resolution sub-image and obtain the coarse positioning of the lock hole through heat map regression, and then achieve sub-pixel level precise positioning through two-dimensional Gaussian fitting. Based on the precise positioning point, the roundness error and diameter deviation of the lock hole outline are calculated, and finally the lock status category is identified. The multi-source confidence fusion module is used to obtain the confidence levels of loading configuration type, loading compliance judgment, empty / loaded status and locked status respectively, perform weighted average according to preset weights to generate comprehensive confidence level, and integrate information such as container number, quantity and status label to form structured data; The report generation and transmission module is used to attach equipment identifiers and timestamps to the structured loading status report, and after integrity verification and encryption with the national cryptographic standard SM4, it is uploaded to the transportation management system through the railway dedicated network. When an abnormal status is detected, it automatically triggers dispatch intervention instructions and safety warning work orders.

[0042] This embodiment also provides a computer device applicable to the intelligent identification method for railway container loading status, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent identification method for railway container loading status as proposed in the above embodiment.

[0043] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0044] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the intelligent identification method for railway container loading status as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0045] In summary, this invention achieves fully automated, high-precision, and real-time intelligent identification of the loading status of railway containers through core technologies such as multi-view image dynamic correction, graph neural network topology reasoning, physical-semantic feature fusion discrimination, sub-pixel level keyhole positioning, and multi-source confidence weighted fusion. It effectively solves the problems of low efficiency, high missed detection rate, and inability to quantify and evaluate traditional manual inspections. The system can simultaneously determine the compliance of container configuration, empty / load status, and lock integrity, improving the level of railway freight safety supervision, reducing the risk of derailment, off-center loading, or container detachment caused by abnormal loading, and providing structured data support for transportation scheduling, thus promoting the upgrading of railway container transportation towards intelligence, digitalization, and standardization.

[0046] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent identification of the loading status of railway containers, characterized in that: include: Multi-view original images of containers on a moving train are acquired, and dynamic perspective correction and motion blur suppression are performed on the original images based on track geometry parameters and real-time train speed to obtain geometrically normalized front view images of containers. Target detection is performed on the geometrically standardized front view image of the container, and candidate container bounding boxes and container type categories are output. An undirected topological graph is constructed based on the projection position of the candidate containers on the flatcar. The loading configuration type of each container and the loading compliance judgment result of the entire flatcar are obtained through graph neural network reasoning. Based on the flatcar chassis area corresponding to each container in the geometrically normalized front view image of the container, the chassis sinking amount and wheel axle spacing compression ratio are extracted as physical features, and combined with the deep semantic features extracted from the corresponding container image area, the empty and heavy status of each container is determined after fusion. High-resolution sub-images are cropped from the preset area of ​​the door corner fitting according to the box type. Sub-pixel-level precise positioning of the keyhole center is achieved through heat map regression and two-dimensional Gaussian fitting. The locking state category is identified based on the geometric deviation between the precise positioning result and the standard lock model. The confidence levels of loading configuration type, loading compliance judgment result, empty / loaded status and locked status category are weighted and fused to generate a structured loading status report; The structured loading status report is transmitted to the railway transport management system in real time.

2. The intelligent identification method for railway container loading status as described in claim 1, characterized in that: The process involves performing target detection on the geometrically standardized front view image of the container, outputting candidate container bounding boxes and container type categories, and constructing an undirected topological graph based on the projection positions of the candidate containers on the flatcar. Then, using a graph neural network, the loading configuration type of each container and the loading compliance judgment result of the entire flatcar are obtained. The specific steps are as follows: The geometrically normalized front view image of the container is input into the target detection network based on the improved EfficientRep structure. Forward propagation is performed to obtain a set of bounding boxes of N candidate containers. Each bounding box contains four parameters: center x-coordinate, center y-coordinate, width and height, as well as the corresponding container type label. A two-dimensional plane coordinate system is established with the center line of the flatbed as the reference. The center point of each bounding box is mapped to this coordinate system to obtain the node position. Traverse all node pairs. If the Euclidean distance between any two nodes is less than a preset distance threshold, and the absolute value of the difference between the direction angles derived from the aspect ratios of their respective bounding boxes is less than a preset angle tolerance, then establish an undirected edge between the two nodes. Construct a graph structure where the number of nodes equals the number of candidate boxes. The initial feature vector of each node is composed of the center x-coordinate, center y-coordinate, width, height, and the embedding encoding of the box type of the corresponding bounding box. The edge set is determined by the adjacency condition. The feature vector of each edge contains the Euclidean distance and the difference in orientation angle between the two connected nodes. The constructed graph structure is input into a graph neural network containing three graph convolutional layers. The node feature updates in each layer follow a message-passing mechanism, specifically expressed as follows: ; in, Indicates the first The first in the layer The feature vector of each node Represents a node The set of neighboring nodes, For neighboring nodes In the The feature vector of the layer, For connecting nodes and The edge feature vectors, This represents a vector concatenation operation. For the first Layer-learnable weight matrix, It is a non-linear activation function; After three layers of graph convolution operations, the final node features are input into the fully connected classification head, which outputs the loading configuration type of each box. The loading configuration type includes three cases: single box, double boxes side by side, or stacked boxes. Meanwhile, in accordance with the constraints on flatcar loading layout in the railway industry standard TB / T 3571, the loading configuration type of all boxes and their positions on the flatcar are logically verified. If there are violations of minimum spacing requirements, excessive lateral offset of the center of gravity, or non-permitted stacking combinations, the entire flatcar loading is deemed non-compliant.

3. The intelligent identification method for railway container loading status as described in claim 2, characterized in that: The flatcar chassis region corresponding to each container in the geometrically normalized container front view image is used to extract the frame sinking amount and wheel axle spacing compression ratio as physical features. These features are then combined with the deep semantic features extracted from the corresponding container image region to determine the empty / load status of each container. The specific steps are as follows: Based on the horizontal projection range of the bounding box output by the target detection in the geometrically normalized image, a rectangular sub-image covering the flatcar chassis and track area is extracted downwards. Edge detection and line fitting are performed on the sub-image to extract the lower edges of the left and right side beams respectively, and the average distance between the two fitted lines in the vertical direction is calculated as the frame sinking amount. Detect the center point of the wheel axle in the base plate diagram, arrange them in order from left to right, calculate the actual distance between the center points of adjacent wheel axles, and take the arithmetic mean of all adjacent distances as the measured wheel axle distance; Divide the measured wheel-axle distance by the standard wheel-axle distance of the same type of flatcar under no-load conditions to obtain the wheel-axle distance compression ratio. The frame sag and the wheel axle spacing compression ratio are combined to form a two-dimensional physical feature vector; The complete box image region is then input into the pre-trained ResNet-34 backbone network, and the output of the global average pooling layer is taken as a 512-dimensional deep semantic feature vector. The two-dimensional physical feature vector is concatenated with the 512-dimensional deep semantic feature vector to form a joint feature vector, which is then input into a discriminant network containing two fully connected layers, and outputs a binary classification result of empty or full boxes. Among them, the chassis undersinking reflects the degree of elastic deformation of the car body caused by the load, and the wheel axle spacing compression ratio reflects the geometric contraction effect of the bogie after being compressed. Together, they constitute a physical observable measurement directly related to the load, which complements the purely visual semantic features.

4. The intelligent identification method for railway container loading status as described in claim 3, characterized in that: The process involves cropping a high-resolution sub-image from a preset area on the corner piece of the door according to the box type, achieving sub-pixel-level precise positioning of the keyhole center through heatmap regression and two-dimensional Gaussian fitting, and identifying the locking state category based on the geometric deviation between the precise positioning result and the standard lock model. The specific steps are as follows: Based on the container type, find the theoretical position of the corresponding container door corner fitting in the ISO 6346 standard, and then crop a high-resolution sub-image of a fixed size from the geometrically normalized front view image of the container with that theoretical position as the center. Input the high-resolution subgraph into the stacked hourglass network and output a probability heatmap of the keyhole center location; The point with the maximum response in the location heatmap is used as the coarse location coordinate. Define a fixed-size neighborhood window around the coarse positioning coordinates and extract all heatmap response values ​​within the window; A two-dimensional Gaussian distribution function is fitted to the heatmap response values ​​within the window. The specific expression is as follows: ; in, Indicates the heatmap on coordinates The response value at that location, Peak amplitude, and These are the coordinates of the Gaussian distribution center, i.e., the sub-pixel level precise positioning coordinates of the keyhole center. and These are the standard deviations in the horizontal and vertical directions, respectively. The optimal parameters are obtained by solving the above equations using the nonlinear least squares method. and ; Will and Mapping back to the original image coordinate system yields the center point of the keyhole in the world coordinate system. The keyhole contour is extracted based on the center point. The root mean square distance from the contour point to the ideal circle is calculated as the roundness error. At the same time, the relative deviation between the diameter of the outer circle of the contour and the diameter of the standard keyhole is calculated as the diameter deviation. The roundness error and diameter deviation are input into the support vector machine classifier, which outputs the locking state category. The locking state category includes four situations: normal locking, not inserted, half-locked, and foreign object occlusion.

5. The intelligent identification method for railway container loading status as described in claim 4, characterized in that: The steps for generating a structured loading status report are as follows: The confidence levels of the loading configuration type, loading compliance judgment result, empty / loaded status, and locked status categories are weighted and fused together. The softmax probability value of each container loading configuration type is obtained from the output of the graph neural network classification head and used as the loading configuration confidence. Obtain a Boolean compliance flag from the compliance verification module. If the compliance meets the rules, the confidence level is 1.0; otherwise, it is 0.

0. The probability value of a full box is obtained from the empty-weight discrimination network and used as the confidence level of the empty-weight state. The output values ​​of the decision functions for each category are obtained from the locked state classifier and then normalized to serve as the locked state confidence. The overall confidence level is calculated as a weighted average of the four confidence levels. The weighting coefficients are dynamically adjusted based on the historical false alarm rate to ensure that high-risk items have higher weights. Combining the results of optical character recognition of container numbers, statistical values ​​of the number of containers, determination of the empty and heavy status of each container, category of locking status of each container, loading compliance mark and overall confidence level, the data is encapsulated into a structured loading status report according to a predefined JSON Schema.

6. The intelligent identification method for railway container loading status as described in claim 5, characterized in that: The process involves acquiring multi-view original images of containers on a moving train, and performing dynamic perspective correction and motion blur suppression on the original images based on track geometry parameters and the real-time speed of the train to obtain a geometrically normalized front view image of the container. The specific steps are as follows: Train passing image acquisition cameras are deployed on the left, right and above the track respectively. The cameras are area array cameras, line array cameras or combinations thereof. Multiple cameras are strictly synchronized through hardware trigger signals, and the vertical resolution is not less than 2048. By measuring the instantaneous speed of the train in real time, the camera exposure time or line frequency parameters are dynamically adjusted according to the speed to ensure that the length of the blur caused by the train's movement in the image does not exceed a preset threshold. Using the pre-completed camera calibration results, obtain the intrinsic and extrinsic parameter matrices for each camera; Based on the known track gauge of 1435 mm and the standard height of the flatcar bottom from the track surface, a homography transformation matrix from the world coordinate system to the unified frontal plane is constructed. The homography transformation matrix is ​​applied to the raw images captured by each camera to project images from different perspectives onto the same frontal plane with the center line of the track as the reference, eliminating perspective distortion caused by differences in shooting angles and generating a geometrically normalized frontal view image of the container.

7. The intelligent identification method for railway container loading status as described in claim 6, characterized in that: The specific steps for transmitting the structured loading status report to the railway transportation management system in real time are as follows: Structured loading status reports are pushed to edge computing servers deployed in the freight yard via railway-specific industrial Ethernet; The edge computing server performs integrity checks on the received reports, including whether required fields exist, whether the confidence value is within the range of [0,1], and whether the timestamp is reasonable. After verification, the report is appended with a unique device identifier and a collection timestamp accurate to milliseconds, and the report content is encrypted using the national cryptographic SM4 algorithm; The encrypted data is uploaded to the China State Railway Group TMS cloud platform via a data security gateway that complies with railway network security standards; The TMS cloud platform receives and decrypts the report, parsing its various fields. If the loading compliance flag is abnormal, or if the locking status of any container is not in the normal locking category, the system will automatically generate a dispatch intervention instruction, including suspending the marshalling, notifying the site for inspection or new weighing, and simultaneously create a safety warning work order, which will be pushed to the relevant station duty terminal and freight dispatch center.

8. A railway container loading status intelligent identification system, based on the railway container loading status intelligent identification method according to any one of claims 1 to 7, characterized in that: include: The system includes an image acquisition and correction module, a box topology modeling module, an empty / load status discrimination module, a locked status recognition module, a multi-source confidence fusion module, and a report generation and transmission module. The image acquisition and correction module is used to synchronously acquire original images of the container through a multi-view industrial camera during the train's movement, and dynamically adjust the exposure time in combination with track geometry parameters and the real-time speed of the train. It also applies homography transformation to eliminate perspective distortion and motion blur, and outputs a geometrically standardized front view image of the container. The box topology modeling module is used to perform target detection on geometrically normalized images, obtain candidate box bounding boxes and box type categories, construct an undirected topology graph with the projection position of the box on the flatcar as nodes and spatial proximity as edges, infer the loading configuration type of each box through a three-layer graph convolutional network, and verify the layout compliance of the entire flatcar according to railway loading specifications. The empty / load status discrimination module is used to extract the corresponding flatbed image of each box, extract the frame sinking amount and wheel axle spacing compression ratio as physical deformation features, and extract deep semantic features from the box image area. The two types of features are then concatenated and input into the discrimination network to output the classification result of empty or loaded box. The locking status recognition module is used to locate the corner fitting area of ​​the door according to the box type, crop the high-resolution sub-image and obtain the coarse positioning of the lock hole through heat map regression, and then achieve sub-pixel level precise positioning through two-dimensional Gaussian fitting. Based on the precise positioning point, the roundness error and diameter deviation of the lock hole outline are calculated, and finally the locking status category is identified. The multi-source confidence fusion module is used to obtain the confidence levels of loading configuration type, loading compliance judgment, empty / loaded status and locked status respectively, perform weighted average according to preset weights to generate comprehensive confidence level, and integrate information such as container number, quantity and status label to form structured data; The report generation and transmission module is used to attach equipment identifiers and timestamps to the structured loading status report, and after integrity verification and encryption with the national cryptographic standard SM4, upload it to the transportation management system through the railway dedicated network. When an abnormal status is detected, it automatically triggers dispatch intervention instructions and safety warning work orders.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the intelligent identification method for the loading status of railway containers as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the intelligent identification method for the loading status of railway containers as described in any one of claims 1 to 7.

Images