Railway goods yard safety operation automatic control method based on deep learning

Through the automatic control method of railway freight yard safety operations based on deep learning, the static and dynamic unsafe state of railway freight yards is identified and warned about, which solves the problem of insufficient research on railway freight yard operation safety and reliance on manual supervision, and improves the operation safety and resource utilization efficiency .

CN118991869APending Publication Date: 2024-11-22SOUTHWEST JIAOTONG UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411079975.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-12-12
Filing Date
2024-08-07
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Inadequate research on the safety of railway freight yards, relying on manual supervision, insufficient application of target detection and behavior recognition technologies, resulting in safety hazards and waste of resources.

Method used

The automatic control method of safety operations of railway freight yards based on deep learning is adopted, and the monitoring item points are divided into static and dynamic unsafe states, and a convolutional neural network and a three-dimensional convolutional neural network are used for identification, and early warnings are issued based on the danger and urgency.

Benefits of technology

It improves the safety of railway freight yard operations, reduces human and material resources consumption, and achieves more efficient safety monitoring and resource optimization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118991869A_ABST
    Figure CN118991869A_ABST
Patent Text Reader

Abstract

The invention discloses an automatic control method for safe operation of a railway freight yard based on deep learning, which comprises the following steps: firstly, analyzing the existing railway freight yard operation, including the whole process of sending operation, on-the-way operation and arrival operation, and extracting key links of the railway freight yard operation, which need to be emphatically supervised, as monitoring item points; dividing the unsafe state into a static unsafe state and a dynamic unsafe state; according to the characteristics of each monitoring item point, different deep learning algorithms are adopted for recognition, a target detection algorithm of deep learning is adopted in a static unsafe state, a behavior recognition algorithm of deep learning is adopted in a dynamic unsafe state, and the recognized unsafe states are automatically warned in time. According to the invention, automatic control research on the railway freight yard operation process is carried out by means of a deep learning technology, a railway freight yard operation intelligent supervision system from identification to early warning is established, and the system has great significance in maintaining normal operation of a railway and improving the cargo transportation safety monitoring and early warning level.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of transportation, and in particular to an automatic control method for safe operation of a railway freight yard based on deep learning. Background Art

[0002] Railway freight yard operation is a key link in maintaining the normal operation of railways, and railway freight safety is an important task that railway transportation development must always focus on and an important foundation for sustainable development. Since the reform of railway freight organization in 2013, the railway has continued to promote the informatization of freight information systems. By implementing intelligent supervision, the accident rate of railway freight yard operations has been reduced, which is of great significance to maintaining the safety of railway freight yards. The shortcomings of current research are mainly reflected in the following aspects:

[0003] (1) Insufficient research on railway freight yard operation safety issues: Regarding railway freight yard operations, most scholars have conducted research on railway freight yard operation process optimization and equipment status monitoring. The key technologies for intelligent rail transit construction are also mainly used for construction safety, bridges and building structures, etc. There is a lack of research on the unsafe behaviors of railway freight yard operators and the unsafe conditions of objects.

[0004] (2) Operation supervision is overly dependent on manual labor: Traditional railway equipment does not have functions such as intelligent identification and remote control. The supervision of railway freight yard operations still relies on front-end cameras installed in freight stations. At the same time, a crude security operation model based on manual monitoring and experience-based decision-making is used to identify safety risks in railway freight yard operations. A large amount of safety management, operation and monitoring work relies on manual labor. The work intensity is high, and personnel fatigue can easily lead to safety hazards.

[0005] (3) Few studies on target detection and behavior recognition: Although deep learning has been widely used in the railway field, there are relatively few target detection and behavior recognition methods used. Even if there are some, they are limited to the fault detection of railway foreign object intrusion, truck component failure, track plate cracks and other facilities and equipment, and have not been applied to the supervision of railway freight yard operations.

[0006] The above-mentioned research deficiencies in automatic control of railway freight yard safety operations are mainly reflected in the lack of research on railway freight yard operation safety issues, excessive reliance on manual operation supervision, and few related studies on target detection and behavior recognition. These problems need to be solved urgently to pursue the safety of railway freight yard operations, ensure the personal safety of operators, maintain the normal operation of railways, and reduce the consumption of railway human and material resources. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides an automatic control method for safe operation of railway freight yard based on deep learning.

[0008] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0009] A railway freight yard safety operation automatic control method based on deep learning comprises the following steps:

[0010] S1. Extract monitoring items in the whole process of sending operation, en route operation and arrival operation of the existing railway freight yard, and divide the extracted monitoring items into static unsafe state and dynamic unsafe state;

[0011] S2. According to the characteristics of different monitoring points, different deep learning algorithms are used to identify unsafe conditions;

[0012] S3. Issue a warning for the identified unsafe state.

[0013] Furthermore, the static unsafe state in S1 includes obstacle detection items and object status detection items, wherein the obstacle detection items include platform residue detection, track and line obstacle detection, vehicle body and container debris detection; the object status detection items include cargo stacking status detection, vehicle status detection and operating equipment placement status detection.

[0014] Furthermore, the dynamic unsafe conditions in S1 include action recognition detection items, pedestrian detection items, vehicle detection items, distance detection items and fire and explosion prevention detection items, among which the action recognition detection items include door opening and closing recognition, height operation recognition, labor safety recognition, and door crane operation recognition; the pedestrian detection items include issuing an alarm when a pedestrian is detected and issuing an alarm when a pedestrian is not detected; the vehicle detection items include road vehicle monitoring and vehicle speed monitoring; the distance detection items include cargo distance detection and vehicle distance detection.

[0015] Furthermore, in S2, according to the characteristics of different monitoring items, a target detection algorithm based on deep learning is used to identify the static unsafe state; and a deep learning behavior recognition algorithm is used to identify the dynamic unsafe state.

[0016] Furthermore, the specific method of using the deep learning-based target detection algorithm to identify the static unsafe state is:

[0017] A1. Capture images of data collected by monitoring probes at static unsafe state identification points;

[0018] A2. Use convolutional neural network to extract basic features of the captured images;

[0019] A3. Set the number of convolution kernels in the convolutional neural network to the maximum limit to extract a set of all features that cannot be recognized by the human eye.

[0020] Furthermore, the specific method of identifying the dynamic unsafe state based on the deep learning behavior recognition algorithm is:

[0021] B1. Capture video data from the data collected by the monitoring probes at the dynamic unsafe state identification points;

[0022] B2. Process multiple consecutive frames of captured video data, associate the previous and next true setting images, and identify the dynamic unsafe state.

[0023] Furthermore, the specific method of issuing an early warning for the identified unsafe state in S3 is:

[0024] S31. Based on the four-quadrant method, the horizontal axis is defined as the degree of danger, and the vertical axis is defined as the degree of emergency, wherein the first to fourth quadrants are respectively a dangerous and emergency module, an emergency but not dangerous module, a dangerous but not emergency module, and a non-dangerous and non-emergency module;

[0025] S32. Based on the four divided quadrants, the railway freight yard operation automatic control warning is divided into three categories, among which Class I warnings include the first and third quadrants, Class II warnings correspond to the second quadrant, and Class III warnings correspond to the fourth quadrant. After using the monitoring probe to identify dynamic and static unsafe behaviors, different types of warnings are issued according to the danger and urgency of the identification results.

[0026] The present invention has the following beneficial effects:

[0027] 1. It is proposed to use deep learning target detection and behavior recognition to monitor railway freight yard operations, and the key links of deep learning application are analyzed by static unsafe state and dynamic unsafe state. The detection content is explained in detail, the monitoring scope is appropriately narrowed, and more efficient safety monitoring is achieved.

[0028] 2. Improve the safety of railway freight yard operations. Railway freight operations involve a large number of vehicles, equipment and personnel. The automatic control system can detect safety hazards and abnormal situations in a timely manner, such as equipment failures and abnormal operations, by real-time monitoring and analyzing video data, so as to warn and take corresponding safety measures to reduce the occurrence of accidents.

[0029] 3. Save human and material resources. In order to solve the problems of excessive reliance on manual labor and human fatigue in previous railway freight yard operation monitoring, real-time monitoring of the operation process can optimize the operation plan and rationally allocate resources, thereby improving operation efficiency and reducing resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The figure is a flow chart of the automatic control method for railway freight yard safety operations based on deep learning.

[0031] Figure 2 Schematic diagram of feature extraction of a two-dimensional convolutional neural network according to an embodiment of the present invention.

[0032] Figure 3 This is an embodiment of the present invention in which a convolutional neural network is used to identify the opening and closing doors in railway freight yard operations.

[0033] Figure 4 Schematic diagram of feature extraction of a three-dimensional convolutional neural network according to an embodiment of the present invention.

[0034] Figure 5 This is the automatic warning category of the embodiment of the present invention. DETAILED DESCRIPTION

[0035] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0036] A deep learning-based automatic control method for railway freight yard safety operations, such as Figure 1 As shown, the following steps are included:

[0037] S1. Extract monitoring items in the whole process of sending operation, en route operation and arrival operation of the existing railway freight yard, and divide the extracted monitoring items into static unsafe state and dynamic unsafe state;

[0038] Static unsafe state identification. The identification of static unsafe state of railway freight yard operation mainly relies on the target detection method of convolutional neural network. With the help of computer, the data and image information captured by the monitoring probe are processed to detect the location, size, shape and other characteristics of the target object, such as obstacle detection of important facilities and status detection of operating vehicles. Based on the analysis of key operation links, the extracted static unsafe state identification items are shown in Table 1.

[0039] Table 1 Static unsafe state identification items

[0040]

[0041] 1) Obstacle Detection

[0042] ① Platform debris detection: Regarding the debris detection on railway platforms, the main contents include whether there are items left on the platform, whether the remaining goods on the platform are cleaned up in time after loading and unloading, and whether light objects such as plastic sheets and woven bags are cleaned up in time after loading and unloading.

[0043] ② Track and line obstacle detection: During railway operation, obstacles such as pedestrians, floating objects, and mud and rock flows often invade the railway line, seriously threatening the safety of train operation.

[0044] ③ Vehicle body and container debris detection: Mainly detect whether there are debris on the vehicle exterior, bottom, tanker, and container top. Especially when transporting dangerous goods, the presence of debris around the vehicle body will seriously affect transportation safety.

[0045] 2) Object state detection

[0046] Object status detection is mainly used to identify unsafe conditions of goods and facilities and equipment to prevent safety accidents caused by management or operational negligence.

[0047] ① Cargo stacking: According to the requirements of the "Technical Rules for Safety in Railway Cargo Loading and Unloading", goods that are easy to roll should be stacked perpendicular to the track, smoothly and neatly, and tightly packed to prevent bumps and squeezing. They should not be stacked diagonally or staggered.

[0048] ② Placement of operating equipment: mainly includes the use of loaders to load bulk cargo, whether the regulations on sequential loading and flat roofing between vehicles are implemented, and whether the belt conveyor is parked perpendicular to the line.

[0049] (2) Dynamic unsafe state identification. The detection of dynamic unsafe state is more complicated. Based on deep learning, the captured video data is processed and time features are added to identify video clips with contextual connections. Based on the above analysis of key operation links, the extracted dynamic unsafe state identification items are shown in Table 2.

[0050] Table 2 Dynamic unsafe state identification items

[0051]

[0052]

[0053] 1) Action Recognition

[0054] ① Door opening and closing recognition: During the operation and after the driver gets on and off the vehicle, check whether the vehicle doors and windows are closed. In addition, the door opening and closing status of the open vehicle should also be monitored.

[0055] ② Identification of high-altitude work

[0056] A. Do workers working at heights wear helmets, chin straps, and non-slip shoes?

[0057] B. Use ladders to climb safely (2m and above)

[0058] C. Transfer of tools for high-altitude operations (2m and above)

[0059] ③Labor safety identification

[0060] A. Whether the operator is standing or walking on the moving vehicle

[0061] B. Whether the workers climb, sit, lie, stand or walk on the cargo

[0062] C. Whether the operating personnel implement the "one stop, two looks, three passes" and "hand comparison, eye observation, and mouth call" system when crossing the railway freight yard line.

[0063] ④ Gantry crane operation identification: When the cargo is lifted to more than 0.5 meters above the ground and the vehicle floor, detect whether there is a person or human body part under the cargo.

[0064] 2) Pedestrian Detection

[0065] ① Alarm when pedestrians are detected: During railway freight yard operations, there are a series of safety regulations, such as prohibiting the entry of unauthorized personnel, prohibiting people from standing in front of the closed doors of boxcars, prohibiting people from crawling under the car, or sitting, lying or walking on the rails, sleepers or roadbeds. When monitoring such operations, pedestrians are strictly prohibited within the monitoring range.

[0066] ② Issue an alarm when no pedestrians are detected: In some railway freight yard operations, operators are required to conduct on-site control, on-site command and daily patrols. For example, when a crane is lifting goods, people cannot stand under the goods, but there should be operators guarding the safety range around the crane; in order to detect whether the field freight forwarders conduct anti-skid inspections in time after taking over and within 6 hours, the monitoring probe needs to monitor the patrol range and identify whether there are people patrolling within a fixed time period.

[0067] 3) Vehicle detection

[0068] ① Road vehicle monitoring: In railway freight yard operations, it is necessary to identify not only trucks but also cranes, forklifts and other loading and unloading vehicles. The main purpose of road monitoring is to prevent vehicle collision accidents. For example, monitor whether there are vehicles or pedestrians traveling in the opposite direction at the corner; monitor whether there are pedestrians passing through the blind spot of the truck operation; monitor whether there are any loading and unloading equipment such as trolleys and forklifts hitting the car door; monitor whether there are any idle vehicles entering the protection area.

[0069] ② Vehicle speed monitoring: Video surveillance should focus on monitoring road crossings, intersections, downhill roads, factory and warehouse gates, production sites and other areas where speeding and accidents are prone to occur.

[0070] 4) Distance estimation

[0071] ① Cargo distance estimation: The main items for cargo distance estimation include whether the cargo is kept more than 1.5 meters away from the outside of the rail head, whether the distance between the cargo stack and the edge of the platform is less than 1 meter, whether the distance between the cargo stack and the power switch and fire-fighting facilities is less than 2 meters, whether the distance between the cargo is less than 1 meter when transporting piece goods, whether the distance between the cargo is less than 1 meter when transporting piece goods, whether the container is suspended when it falls to a height of about 120mm above the lock during loading operations, and whether it is first lifted by a resistance inching to a height of about 100mm above the plane of the vehicle floor and then suspended during unloading operations.

[0072] ② Vehicle distance estimation: The items regarding vehicle distance estimation mainly include whether the forklift is more than 0.3 meters away from the edge of the platform when driving on the platform; whether the operator is less than 3 meters away from the car.

[0073] ③ Pedestrian distance detection: whether pedestrians keep a safe distance of more than 5 meters from locomotives and vehicles when they go around the front and rear of the parked train; whether the lifting personnel stand in a safe place and keep a retreat distance of more than 2 meters from the goods during lifting; whether the auxiliary personnel (including the command personnel) immediately evacuate to a safe distance of more than one car at both ends of the flat car when the reach loader driver continues to work; after confirming that the container has been dropped into the slot and that the container is unlocked from the F-TR lock, whether the auxiliary personnel (including the command personnel) immediately evacuate to a safe distance of more than one car at both ends of the flat car when the reach loader driver continues to work.

[0074] 5) Fire and explosion prevention

[0075] It mainly detects smoke and open flames, with the aim of monitoring smoking and illegal use of open flames in railway freight yards to prevent the further spread of fire.

[0076] S2. According to the characteristics of different monitoring points, different deep learning algorithms are used to identify unsafe conditions;

[0077] According to the identification rules determined above, the static and dynamic unsafe states are detected using the target detection and behavior recognition methods in deep learning respectively. The training set and verification set are formed through historical data, and the test set is formed through real-time captured data to issue early warnings for the identified unsafe states.

[0078] Obtaining video and image data of railway freight yard operations

[0079] Using monitoring probes installed in railway freight yards to conduct real-time monitoring of railway freight yard operations and detect abnormal situations is a common method for obtaining video and image data of the operation site.

[0080] By capturing a large amount of video and image data in key operation links through monitoring probes, it is possible to form training sets, verification sets, and test sets for deep learning algorithms. The training set is used to debug the deep learning algorithm for railway freight yard operations, the verification set is used to check the training effect, and the test set is used to test the actual learning ability of the network.

[0081] Unsafe state identification based on deep learning

[0082] Convolutional neural network (CNN) is a type of feedforward neural network with deep structure and convolutional calculation. It is one of the representative algorithms of deep learning and is widely used in various fields such as image classification, image recognition, and image segmentation. Classic models of convolutional neural network such as LeNet-5, AlexNeT, VGG-16, GoogLeNet, etc. and their improved algorithms can meet the recognition requirements of most dangerous operation links and achieve high recognition accuracy. By inputting a large number of relevant pictures of dangerous operation links in railway freight yards and using deep learning for training and testing, accurate recognition of unsafe operation status in railway freight yards can be achieved.

[0083] (1) Static unsafe state identification based on deep learning target detection algorithm

[0084] The target detection method used for static unsafe behavior identification in railway freight yard operations mainly relies on the two-dimensional convolutional neural network algorithm (2D CNN) in deep learning. Its working principle is as follows: the data collected by the monitoring probe of the static unsafe state identification point is captured, and after the image data of the operation site is captured, the convolutional neural network is used to extract its basic features, such as Figure 2 shown.

[0085] The calculation of 2D convolution in static unsafe state recognition is as follows: the input layer is W i ×H i ×D i The number of filters is set to k, the convolution kernel dimension is w×h, the sliding step is s, the padding value is p, and the output layer is W o ×H o ×D o The parameter relationship between the output layer and the input layer is:

[0086]

[0087] Where W o , H o , D o They are the width, height, and depth of the convolution kernel, respectively.

[0088] When extracting the actual features of vehicles or pedestrians at the work site, the convolution layer can extract all feature sets by adjusting the number of convolution kernels. During operation, the number of convolution kernels determines the number of feature elements extracted, and there is a certain positive proportional relationship between the two. Therefore, considering the different features of the static unsafe state extracted above, it is necessary to set the convolution kernel to the maximum limit number, so that the feature image that cannot be recognized by the human eye can be extracted. Taking the detection of whether the door is opened or closed as an example, the neural network is debugged through a large number of training sets, and after testing with the verification set, the image data is captured in real time at the work site for recognition. The specific recognition process is shown in Figure 3 .

[0089] (2) Dynamic unsafe state identification based on deep learning behavior recognition algorithm

[0090] The target detection method used to identify static unsafe behaviors in railway freight yard operations mainly relies on the three-dimensional convolutional neural network algorithm (3D CNN) in deep learning. Compared with target detection, behavior recognition will add temporal features to identify actions with contextual connections, or use human key points to establish a skeleton model framework feature system. Therefore, behavior recognition often uses video as input, such as Figure 4 shown.

[0091] The calculation of 3D convolution in dynamic unsafe state recognition is as follows: the input layer is W i ×H i ×D i ×C i The number of filters is set to k, the convolution kernel dimension is w×h×d, the sliding step is s, the padding value is p, and the output layer is W o ×H o ×D o ×C o The parameter relationship between the output layer and the input layer is:

[0092]

[0093] Where W o , H o , D o , C o They are the width, height, depth and number of convolution kernels respectively.

[0094] The basic principle is as follows: the data collected by the monitoring probe of the dynamic unsafe state identification point is captured by video data. After capturing continuous frames, the convolutional neural network processes multiple continuous frames in the video, associates the images of the previous and next frames, and can identify dynamic unsafe states. Taking the recognition of the unlocking sequence of the middle door of the open car as an example, the three-dimensional convolutional neural network recognizes the actions of the operator according to the unlocking sequence of the middle door of the open car. Once an incorrect operation occurs, a warning needs to be issued to remind the operator to pay attention.

[0095] S3. Issue a warning for the identified unsafe state.

[0096] 1. Warning category. Based on the four-quadrant method, the horizontal axis is divided into the degree of danger, and the vertical axis is divided into the degree of urgency, resulting in a total of four quadrants: the first to fourth quadrants are the dangerous and urgent module, the urgent but not dangerous module, the dangerous but not urgent module, and the non-dangerous and non-urgent module. Based on the above four quadrants, the railway freight yard operation automatic control warnings are divided into three categories. Class I warnings include the first and third quadrants, Class II warnings correspond to the second quadrant, and Class III warnings correspond to the fourth quadrant. After using monitoring probes to identify dynamic and static unsafe behaviors, different types of warnings are issued according to the danger and urgency of the identification results, such as Figure 4 shown.

[0097] ① Class I warning: Class I warning corresponds to dangerous and urgent events in the first quadrant and dangerous but non-urgent events in the third quadrant, which can easily lead to accidents and threaten personal safety and property safety, such as pedestrians crossing the railway, unsafe distance between pedestrians and vehicles, and workers standing under suspended cargo. Once such events are detected, the monitoring end needs to issue a sharp and continuous alarm and make corresponding prompts to warn the relevant personnel to respond immediately until the target returns to a safe state and stops the alarm.

[0098] ② Class II warning: Class II warnings correspond to emergency but non-dangerous events in the second quadrant. Such events are likely to affect the normal operation of the railway or cause property losses, but will not directly threaten personal safety. For example, workers climb, sit, lie, stand, and walk on the cargo. Once such events are detected, the monitoring end needs to issue a warning and make corresponding prompts to remind the relevant personnel to respond immediately until the target returns to a safe state and the alarm stops.

[0099] ③ Class III warning: Class III warnings correspond to events in the fourth quadrant that are neither dangerous nor urgent. Such events will not threaten the safety of railway personnel and property in a short period of time. Relevant personnel can choose whether to handle the event according to the actual situation. For example, whether the railway freight yard gate and the pick-up and delivery gate are closed in time, whether the gate is guarded by a dedicated person during the pick-up and delivery operation, etc. When such events are detected, the monitoring end still needs to issue a warning and make corresponding prompts, but after receiving the warning, the monitoring personnel can choose to make corresponding handling or choose to cancel the warning.

[0100] 2. Objects of warning. Depending on the type of unsafe behavior identified, different objects need to be warned. For example, if there are debris left on the cargo platform, the fire escape is blocked, or the door to the pick-up and delivery vehicle is not closed in time, the back-end monitoring personnel need to be notified to handle it, and the object of the warning is the monitoring personnel; while behaviors such as speeding, not wearing a helmet, and overtaking in front of a running locomotive need to be notified to the on-site personnel for immediate rectification, and the object of the warning is the on-site personnel.

[0101] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0102] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A railway freight yard safety operation automatic control method based on deep learning, characterized in that: The steps include: S1. Extract monitoring items in the whole process of sending operation, en route operation and arrival operation of the existing railway freight yard, and divide the extracted monitoring items into static unsafe state and dynamic unsafe state; S2. According to the characteristics of different monitoring points, different deep learning algorithms are used to identify unsafe conditions; S3. Issue a warning for the identified unsafe state.

2. The deep learning-based automatic control method for railway freight yard safety operations according to claim 1 is characterized in that: The static unsafe state in S1 includes obstacle detection items and object state detection items, where the obstacle The inspection items include platform debris inspection, track and line obstacle inspection, vehicle body and container debris inspection; object status inspection items include cargo stacking status inspection, vehicle status inspection and operating equipment placement status inspection.

3. The railway freight yard safety operation automatic control method based on deep learning according to claim 1 is characterized in that: The dynamic unsafe conditions in S1 include action recognition detection items, pedestrian detection items, vehicle detection items, distance detection items and fire and explosion prevention detection items, among which the action recognition detection items include door opening and closing recognition, height operation recognition, labor safety recognition, and door crane operation recognition; the pedestrian detection items include issuing an alarm when a pedestrian is detected and issuing an alarm when a pedestrian is not detected; the vehicle detection items include road vehicle monitoring and vehicle speed monitoring; the distance detection items include cargo distance detection and vehicle distance detection.

4. The railway freight yard safety operation automatic control method based on deep learning according to claim 1 is characterized in that: In S2, according to the characteristics of different monitoring items, a target detection algorithm based on deep learning is used to identify the static unsafe state; and a dynamic unsafe state is identified based on a deep learning behavior recognition algorithm.

5. The deep learning-based automatic control method for railway freight yard safety operations according to claim 4 is characterized in that: The specific method of using the deep learning-based target detection algorithm to identify the static unsafe state is: A1. Capture images of data collected by monitoring probes at static unsafe state identification points; A2. Use convolutional neural network to extract basic features of the captured images; A3. Set the number of convolution kernels in the convolutional neural network to the maximum limit to extract a set of all features that cannot be recognized by the human eye. The calculation of 2D convolution in the static unsafe state recognition is as follows: the input layer is W i ×H i ×D i The number of filters is set to k, the convolution kernel dimension is w×h, the sliding step is s, the padding value is p, and the output layer is W o ×H o ×D o The parameter relationship between the output layer and the input layer is: Where W o , H o , D o They are the width, height, and depth of the convolution kernel, respectively.

6. The railway freight yard safety operation automatic control method based on deep learning according to claim 4 is characterized in that: The specific method of identifying the dynamic unsafe state based on the deep learning behavior recognition algorithm is: B1. Capture video data from the data collected by the monitoring probes at the dynamic unsafe state identification points; B2. Process multiple consecutive frames of captured video data, associate the previous and next frame images, and identify dynamic unsafe states. The calculation of 3D convolution in the dynamic unsafe state recognition is as follows: the input layer is W i ×H i ×D i ×C i The number of filters is set to k, the convolution kernel dimension is w×h×d, the sliding step is s, the padding value is p, and the output layer is W o ×H o ×D o ×C o The parameter relationship between the output layer and the input layer is: Where W o , H o , D o , C o They are the width, height, depth and number of convolution kernels respectively.

7. The railway freight yard safety operation automatic control method based on deep learning according to claim 1 is characterized in that: The specific method of giving an early warning for the identified unsafe state in S3 is: S31. Based on the four-quadrant method, the horizontal axis is defined as the danger level, and the vertical axis is defined as the emergency level, wherein the first to fourth quadrants are respectively the dangerous and emergency module, the emergency but not dangerous module, the dangerous but not emergency module, and the non-dangerous and non-emergency module; S32. Based on the four divided quadrants, the railway freight yard operation automatic control warning is divided into three categories, among which Class I warning includes the first and third quadrants, Class II warning corresponds to the second quadrant, and Class III warning corresponds to the fourth quadrant. After the dynamic and static unsafe behaviors are identified by the monitoring probe, different types of warnings are issued according to the danger and urgency of the identification results.