A safety detection method for opening and closing door operations of boxcars in railway freight yards

Abstract

The present invention discloses a safety detection method for boxcar door opening and closing operations in railway freight yards. The method solves the problem that there is no intelligent analysis of operation safety and no safety warning in the boxcar door opening and closing operations in the prior art. The method includes pre-setting the boxcar door reference line in the camera monitoring image; using a deep learning model to identify the boxcar doors and personnel in the monitoring image, grouping the doors to obtain multiple historical door grouping sequences, and identifying the door opening and closing events based on the historical door grouping sequences; calculating the door physical frame based on the preset reference line, and determining the dangerous area on the platform based on the door physical frame and the door opening and closing events; and making an operation safety judgment based on the personnel, door opening and closing events and the dangerous area. The present invention can automatically detect the behavior of the boxcar entering the dangerous area during the opening and closing side operation, and issue an alarm reminder, so as to achieve a safe and efficient response during the freight yard operation.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a safety detection method for opening and closing doors of boxcars in railway freight yards. Background Art

[0002] Railway boxcars have two doors in the middle. Operators must use pull cords to open and close these doors, and are prohibited from standing directly in front of them to prevent the doors from detaching or cargo from falling and injuring people. Currently, personnel control is primarily achieved through human defense, but monitoring efficiency needs to be improved. Although traditional video surveillance is currently deployed in railway freight yards, it lacks intelligent safety analysis for boxcar door opening and closing operations. Consequently, surveillance video is primarily used for post-incident investigations and tracing, failing to fully leverage its advantages. Summary of the Invention

[0003] The present invention mainly solves the problem that there is no intelligent analysis of operation safety and no safety warning in the existing boxcar door opening and closing operations, and provides a safety detection method for boxcar door opening and closing operations in railway freight yards.

[0004] The above technical problems of the present invention are mainly solved by the following technical solutions: A method for detecting the safety of opening and closing doors of boxcars in railway freight yards, which analyzes images of boxcars parked on the platform captured by a camera, comprises the following steps:

[0005] S1. Pre-set a reference line for the door of a boxcar in the camera surveillance image; the camera shoots the boxcar parked on the platform at a 30-degree pitch angle and a horizontal deflection angle of approximately 60 degrees to the left or right.

[0006] S2. Use a deep learning model to identify boxcar doors and personnel in the monitoring image, group the doors to obtain multiple historical door grouping sequences, and identify door opening and closing events based on the historical door grouping sequences;

[0007] S3. Calculate the physical door frame based on the preset reference line, and determine the danger zone on the platform by the physical door frame and the door opening and closing events;

[0008] S4. Make operational safety assessments based on personnel, door opening and closing events, and hazardous areas.

[0009] The present invention can automatically detect the behavior of personnel entering a dangerous area when the boxcar is operating on the side of a switch, and issue an alarm to remind, thereby achieving a safe and efficient response during freight yard operations.

[0010] As a preferred solution, the preset reference line of the boxcar door in the camera monitoring image includes:

[0011] Obtain an image of the boxcar captured by the camera, set an upper horizontal reference line along the upper frame of the boxcar door, and set a lower horizontal reference line along the lower frame of the boxcar door;

[0012] Set multiple vertical reference lines along the vertical direction of the door.

[0013] In this solution, upper and lower horizontal reference lines are pre-set on the camera monitoring image, and several vertical reference lines are set between the upper and lower horizontal reference lines. The horizontal reference lines and vertical reference lines refer to the actual upper and lower horizontal lines of the car door and the vertical lines of the door frame on both sides, which are represented in the image.

[0014] As a preferred solution, the identification of the boxcar door and the personnel in the monitoring image includes:

[0015] Collect one frame of image in real time;

[0016] A trained deep learning model is used to identify the boxcar doors and people in the image. The door recognition result is a door recognition frame, including the coordinates of the upper left vertex and the width and height information of the door recognition frame. The person recognition result is a person recognition frame, including the coordinates of the upper left vertex and the width and height information of the person recognition frame.

[0017] As a preferred solution, the grouping of vehicle doors to obtain multiple historical vehicle door grouping sequences includes:

[0018] Sort the identified doors according to the camera shooting direction. When the camera is facing right, sort the identified doors from left to right according to the center coordinates of the door identification frame. When the camera is facing left, sort the identified doors from right to left according to the center coordinates of the door identification frame to obtain the sorted door sequence.

[0019] Classify the doors that meet the first condition as belonging to the same compartment to obtain a door group sequence, where each door group includes the left and right doors of the same compartment. The first condition is that the distance between the center points of any adjacent first door identification frame and the second door identification frame in the sorted door sequence is less than 3 times the width of the first door identification frame, and the ratio of the area of ​​the first door identification frame to the area of ​​the second door identification frame is less than or equal to 8.

[0020] Traverse each door group in the door group sequence and determine whether it geometrically intersects with the last door group in each historical door group sequence. If so, add the door group to the historical door group sequence. If there is no intersection or no historical door group sequence exists, create a new historical door group sequence and add the door group to the newly created historical door group sequence. After the traversal is complete, multiple historical door group sequences are obtained. In this scheme, geometric intersection refers to whether the door recognition frames of adjacent frames in the camera surveillance image overlap. If there are multiple overlapping door groups, the door group with the most recent time is selected and added to the historical door group sequence containing that door group. A frame of image also includes time information, so both the identified doors and door groups contain time information. Grouping the door recognition results is to group the doors of the same carriage together, specifically filtering out the left and right doors of that carriage.

[0021] As a preferred solution, the identifying door opening and closing events includes:

[0022] Based on the historical door grouping sequence, if the door moves, a door opening and closing event is determined. The door movement threshold is set, and the movement distance of each left door relative to the left door width and the movement distance of each right door relative to the right door width are calculated. The door opening and closing event includes the door opening and closing type, start time, and end time. The door width is the width of the door identification frame, and the door movement distance is the movement distance of the center point of the door identification frame calculated based on the historical door grouping sequence.

[0023] If the left door moves to the left beyond the door movement threshold, it is judged as the left door opening, and the start time and end time are recorded. The historical door grouping sequence is compared with the identified door. If the door moves, the start time is recorded based on the current frame image. If the door stops moving during the movement, the end time is recorded based on the current frame image.

[0024] If the distance the left door moves to the right exceeds the door movement threshold, it is judged as closing the left door, and the start time and end time are recorded.

[0025] If the right door moves to the right beyond the door movement threshold, it is judged as opening the right door, and the start and end times are recorded.

[0026] If the distance the right door moves to the left exceeds the door movement threshold, it is judged as closing the right door, and the start time and end time are recorded.

[0027] As a preferred solution, the physical frame of the boxcar door identified by calculation includes:

[0028] The interpolation method is used to obtain the inclination angle of the vertical reference line of each vertex of the door identification frame passing through the door.

[0029]

[0030] Among them D l D is the distance from the vertex to the left preset vertical reference line. r is the distance from the vertex to the preset vertical reference line on the right, θ l is the inclination angle of the preset vertical reference line on the left relative to the horizontal line, θ r is the tilt angle of the preset vertical reference line on the right relative to the horizontal line; the tilt angles of the four vertices of the door identification frame relative to the horizontal line are calculated using the above formula. The tilt angle θ is the tilt angle of the vertical reference line relative to the horizontal line.

[0031] If the camera is shooting to the right, the closer the door frame is to the right, the smaller its inclination angle is.

[0032] Determine whether the inclination angle of the vertical reference line passing through the upper right vertex of the door identification frame is less than or equal to π / 2. If so, take the intersection of the vertical reference line passing through the upper right vertex of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line. If not, take the intersection of the right border of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line.

[0033] Determine whether the inclination angle of the vertical reference line passing through the lower left vertex of the door identification frame is less than or equal to π / 2. If so, take the intersection of the left border of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line. If not, calculate the intersection of the upper horizontal line and the left border of the door identification frame, and take the intersection of the vertical reference line passing through the intersection and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line;

[0034] If the camera is shooting to the left, the door frame that is closer to the left will have a greater inclination angle.

[0035] Determine whether the inclination angle of the vertical reference line passing through the upper left vertex of the door identification frame is greater than or equal to π / 2. If so, take the intersection of the vertical reference line passing through the upper left vertex of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line. If not, take the intersection of the left border of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line.

[0036] Determine whether the inclination angle of the vertical reference line passing through the lower right vertex of the door identification frame is greater than or equal to π / 2. If so, take the intersection of the right border of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line. If not, calculate the intersection of the upper horizontal reference line and the right border of the door identification frame, and take the intersection of the vertical reference line passing through the intersection and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line;

[0037] The first intersection point is obtained from the above judgment: the intersection of the left border of the left door physical frame and the lower horizontal reference line,

[0038] The second intersection point: the intersection of the right frame of the left door physical frame and the lower horizontal reference line,

[0039] The third intersection point: the intersection of the left border of the right door physical border and the lower horizontal reference line,

[0040] Fourth intersection point: the intersection of the right border of the right door physical frame and the lower horizontal reference line.

[0041] This solution uses an interpolation method to obtain the inclination angle of the vertical reference line passing through each vertex of the door identification frame relative to the horizontal line. According to the camera shooting direction, the intersection of the left and right frames of the door's physical frame and the horizontal reference line below is calculated. Then, the physical frame of the door can be determined based on the vertical reference line.

[0042] As a preferred solution, the method of determining the dangerous area on the platform based on the physical door frame and door opening and closing events includes:

[0043] Calculate the estimated position of the vertex above the door's physical frame on the platform when the door's physical frame is tilted toward the platform, including:

[0044] Obtain the intersection of the vertical reference line passing through the lower vertex of the physical door frame and the upper horizontal reference line, record it as the upper intersection point, and obtain the symmetrical point with the lower horizontal reference line as the symmetry axis, record it as the symmetrical intersection point. The symmetrical intersection point is the estimated position;

[0045] Obtain the symmetrical points of the first, second, third, and fourth intersections respectively, and record them as the first symmetrical intersection point, the second symmetrical intersection point, the third symmetrical intersection point, and the fourth symmetrical intersection point;

[0046] If the left door or the right door is opened, the area enclosed by the first intersection, the first symmetrical intersection, the fourth intersection, and the fourth symmetrical intersection is obtained as the danger zone;

[0047] If the left door is closed, the area enclosed by the first intersection, the first symmetrical intersection, the second intersection, and the second symmetrical intersection is obtained as the danger zone;

[0048] If the right door is closed, the area enclosed by the third intersection, the third symmetrical intersection, the fourth intersection, and the fourth symmetrical intersection is obtained as the danger zone.

[0049] An axisymmetric method is used to obtain the area where the physical frame of the door falls toward the platform. Specifically, the estimated position of the upper vertex of the physical frame of the door on the platform is calculated. The door is then divided into a dangerous area based on the door opening and closing events. When the left or right door is opened, the area connected by the physical frames of the left and right doors falling toward the platform is divided into a dangerous area. When the left or right door is closed, the area where the left or right physical frame falls toward the platform is divided into a dangerous area.

[0050] As a preferred solution, the operation safety judgment based on personnel, door opening and closing events and dangerous areas includes:

[0051] Determine whether any identified personnel appear in the danger zone between the start time and the end time of the door opening and closing event. If so, trigger an alarm and then return to step S2. If not, return to step S2.

[0052] As a preferred solution, the identifying door opening and closing events further includes:

[0053] A door gap threshold is set and the gap distance between the left and right doors in a door group is calculated. Based on the historical door grouping sequence, if a door moves and the gap distance is greater than the door gap threshold, it is considered a non-door opening / closing event; otherwise, it is considered a door opening / closing event. Because the doors of boxcars move when entering and exiting the platform, the gap distance determination is used to filter out door opening / closing events caused by boxcar movement. This means filtering out door opening / closing events where the gap distance remains less than the set door gap threshold throughout the door movement.

[0054] As a preferred solution, the identifying door opening and closing events further includes:

[0055] Based on the historical door grouping sequence, the leftmost horizontal coordinate of the left side of the center point of all door identification frames and the rightmost horizontal coordinate of the right side of the center point of all door identification frames are obtained. The distance between the leftmost and rightmost horizontal coordinates is evenly divided into several movement segments. The number of segments in which the center point of the historical door identification frame appears in the movement segment is counted as the movement smoothness indicator. The door movement smoothness indicator is determined to be less than the set indicator threshold. If so, the door opening and closing event is filtered out. Otherwise, it is a door opening and closing event. Because deep learning models occasionally misidentify non-door objects as doors, door opening and closing events caused by door misidentification are blocked by using door movement smoothness filtering.

[0056] Therefore, the advantages of the present invention are that it can automatically detect the behavior of personnel entering the dangerous area when the boxcar is operating on the side of the switch, and issue an alarm to remind, thereby achieving a safe and efficient response during the freight yard operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of a process of the present invention;

[0058] Figure 2 is a schematic diagram of the preset reference line of the image in the present invention;

[0059] Figure 3 Schematic diagram of calculating the inclination angle of the vertical reference line passing through the vertex of the door identification frame in the present invention;

[0060] Figure 4 is a schematic diagram of a vertical reference line passing through each vertex of the door identification frame in the present invention;

[0061] Figure 5 This is a first schematic diagram of the present invention in which the right side of the camera faces the physical frame of the door for recognition;

[0062] Figure 6 This is a second schematic diagram of the present invention in which the right side of the camera faces the physical frame of the door for recognition;

[0063] Figure 7 This is a first schematic diagram of the present invention in which the left side of the camera faces the physical frame of the door for recognition;

[0064] Figure 8 This is a second schematic diagram of the present invention in which the left side of the camera faces the physical frame of the door for recognition;

[0065] Figure 9 This is a schematic diagram of the estimated position of the physical frame of the door on the platform in the present invention;

[0066] Figure 10 It is a schematic diagram of the calculation of the danger zone in the present invention. DETAILED DESCRIPTION

[0067] The technical solution of the present invention will be further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0068] Example:

[0069] This embodiment is a method for detecting the safety of opening and closing doors of boxcars in railway freight yards. The method analyzes the images of boxcars parked on the platform taken by a camera. Figure 1 As shown, the following steps are included:

[0070] S1. Pre-set the reference line of the boxcar door in the camera monitoring image; the camera shoots the boxcar parked on the platform at a 30-degree pitch angle and a horizontal deflection angle of about 60 degrees to the left or right. Figure 2 As shown, the specific process includes:

[0071] Get the boxcar image captured by the camera and set an upper horizontal reference line L along the upper frame of the boxcar door. u , set a lower horizontal reference line L along the lower frame of the boxcar door d ;

[0072] Set multiple vertical reference lines L along the vertical direction of the door v1 ,L v2 ,…,L vn .

[0073] S2. Use a deep learning model to identify boxcar doors and personnel in the monitoring image, group the doors to obtain multiple historical door grouping sequences, and identify door opening and closing events based on the historical door grouping sequences;

[0074] S21. Real-time acquisition of a frame image;

[0075] S22. Use the trained deep learning model to identify the boxcar door and personnel in the image, and obtain the door recognition result [d 1k ,d 2k ,…,d ik ,…,d nk ] and person recognition results [p 1k ,p 2k ,…,p ik ,…p nk ], where the door recognition result d ik The door recognition frame of a car door identified by the deep learning model, including the coordinates of the upper left vertex and width and height information of the door recognition frame, and the recognition result of the person d ik A person identification box, including the coordinates of the upper left vertex and width and height information of the person identification box.

[0076] S23. Sort the identified doors according to the camera shooting direction;

[0077] If the camera is facing right, the doors are sorted from left to right according to the center coordinates of the door identification frame. If the camera is facing left, the doors are sorted from right to left according to the center coordinates of the door identification frame. The sorted door sequence [d′ 1k ,d′ 2k ,…,d′ 3k ,…,d′ nk ];

[0078] S24. Classify the doors that meet the first condition as the doors of the same carriage, and obtain the door grouping sequence [g 1k ,g 2k ,…,g ik ,…,g nk ], where g ik Indicates door grouping, g ik =[dlk ,d rk ], d lk Indicates the left door of the carriage, d rk Indicates the right door of the carriage;

[0079] The first condition is: any adjacent first door recognition frame d′ in the door sequence after sorting ik and the second door recognition frame d′ (i+1)k The center point distance is less than 3 times the width of the first door recognition frame, and the first door recognition frame d' ik area and the second door recognition frame d′ (i+1)k The area ratio is less than or equal to 8;

[0080] S25. Traverse the door group sequence [g 1k ,g 2k ,…,g ik ,…,g nk ] in each door group g ik ,

[0081] Determine the door grouping and the historical door grouping sequence G i =[g i1 ,g i2 ,…,g ik ,…,g in ] whether the last door group of G has geometric intersection, i represents the historical door grouping sequence of the i-th carriage,

[0082] If there is an intersection, the door group is added to the historical door group sequence. If there is no intersection or no historical door group sequence, a new historical door group sequence is created and the door group is added to the new historical door group sequence. After the traversal is completed, multiple historical door group sequences [G1, G2, ..., G k ,…,G n ].

[0083] S26. Based on the historical door grouping sequence, if a door moves, a door opening / closing event is determined. A door movement threshold of 0.4 is set, and the movement distance of each left door relative to the left door width and the movement distance of each right door relative to the right door width are calculated. A door opening / closing event includes the door opening / closing type, start time, and end time. The door width is the width of the door identification frame, and the door movement distance is the movement distance of the center point of the door identification frame calculated based on the historical door grouping sequence. The historical door grouping sequence is compared with the identified door. If a door moves, the start time is recorded based on the current frame image. If a door stops moving during movement, the end time is recorded based on the current frame image.

[0084] If the left door moves to the left more than the door movement threshold of 0.4, it is judged as opening the left door, and the start and end times are recorded.

[0085] If the distance the left door moves to the right exceeds the door movement threshold of 0.4, it is judged as closing the left door, and the start time and end time are recorded.

[0086] If the right door moves to the right more than the door movement threshold of 0.4, it is judged as opening the right door, and the start and end times are recorded.

[0087] If the distance the right door moves to the left exceeds the door movement threshold of 0.4, it is judged as closing the right door, and the start time and end time are recorded.

[0088] Since the doors of boxcars move when entering and exiting the platform, this step also requires screening door opening and closing events caused by the boxcar's movement by judging the gap distance. That is, filtering door opening and closing events where the gap distance is always less than the set door gap threshold during the door movement. Specifically, it includes:

[0089] Set a door gap threshold and calculate the gap distance between the left and right doors in the door group. Based on the historical door grouping sequence, if the door moves and the gap distance is greater than the door gap threshold, it is determined that it is not a door opening and closing event and returns to step S2. Otherwise, it is a door opening and closing event and proceeds to step S26.

[0090] In addition, because the deep learning model occasionally misidentifies non-door objects as doors, this step also uses door movement smoothness filtering to shield door opening and closing events caused by misidentification. Specifically,

[0091] According to the historical door grouping sequence, get the leftmost horizontal coordinate x of the left horizontal coordinate of the center point of all door recognition frames min , and the rightmost horizontal coordinate x in the right horizontal coordinates of the center points of all door recognition frames max , the distance between the leftmost horizontal coordinate and the rightmost horizontal coordinate [x min ,x max ] is evenly divided into 10 moving segments, and the number of segments in which the center point coordinates of the historical door recognition frame appear in the 10 moving segments is counted as the movement smoothness index, and it is judged whether the door movement smoothness index is less than the set index threshold 3. If so, the door opening and closing event is filtered and the process returns to step S2. If not, it is a door opening and closing event and the process proceeds to step S26.

[0092] S3. Calculate the physical door frame based on the preset reference line and determine the danger zone on the platform based on the physical door frame and door opening and closing events. This includes:

[0093] S31. Using an interpolation method, obtain the inclination angle of the vertical reference line of each vertex of the door identification frame passing through the door.

[0094]

[0095] like Figure 3 As shown, the dotted box is the door recognition box, where D l D is the distance from the vertex to the left preset vertical reference line. r is the distance from the vertex to the preset vertical reference line on the right, θ is the inclination angle of the vertical reference line relative to the horizontal line, θ l is the inclination angle of the preset vertical reference line on the left relative to the horizontal line, θ r It is the inclination angle of the preset vertical reference line on the right relative to the horizontal line;

[0096] like Figure 4 As shown, the inclination angles of the four vertices of the door recognition frame relative to the horizontal line are calculated by the above formula, which are:

[0097] The inclination angle θ1 of the vertical reference line L1 passing through the upper left vertex of the door recognition frame,

[0098] The inclination angle θ2 of the vertical reference line L2 passing through the upper right vertex of the door recognition frame,

[0099] The inclination angle θ3 of the vertical reference line L3 passing through the lower right vertex of the door recognition frame,

[0100] The inclination angle θ4 of the vertical reference line L4 passing through the lower left vertex of the door recognition frame.

[0101] S32. Calculate the intersection of the side physical frame and the lower horizontal reference line using the imaging deformation characteristics of the vehicle door physical frame in the monitoring image.

[0102] If the camera is shooting toward the right, in this case, the closer the door frame is to the right, the smaller its inclination angle is.

[0103] Determine whether the inclination angle θ2 of the vertical reference line passing through the upper right vertex of the door recognition frame is less than or equal to π / 2. If so, Figure 5 As shown, the intersection of the vertical reference line passing through the upper right vertex of the door identification frame and the horizontal reference line below is taken as the intersection point Pt of the right border of the door physical frame and the horizontal reference line below r If not, if Figure 6 As shown, the intersection point of the right border of the door identification frame and the lower horizontal reference line is taken as the intersection point Pt of the right border of the door physical frame and the lower horizontal reference line r ; The solid frame in the figure is the physical frame of the car door.

[0104] Determine whether the inclination angle θ4 of the vertical reference line passing through the lower left vertex of the door recognition frame is less than or equal to π / 2. If so, Figure 5 As shown, the intersection point of the left border of the door identification frame and the lower horizontal reference line is taken as the intersection point Pt of the left border of the door physical frame and the lower horizontal reference line l If not, if Figure 6 As shown, calculate the intersection point Pt′ of the upper horizontal line and the left border of the door identification frame l , take the intersection point Pt′ l The intersection of the vertical reference line and the horizontal reference line below is taken as the intersection point Pt of the left border of the door physical frame and the horizontal reference line below l ;

[0105] If the camera is shooting to the left, the door frame that is closer to the left will have a greater inclination angle.

[0106] Determine whether the inclination angle θ1 of the vertical reference line passing through the upper left vertex of the door recognition frame is greater than or equal to π / 2. If so, Figure 7 As shown, the intersection of the vertical reference line passing through the upper left vertex of the door identification frame and the lower horizontal reference line is taken as the intersection point Pt of the left border of the door physical frame and the lower horizontal reference line l If not, if Figure 8 As shown, the intersection point of the left border of the door identification frame and the lower horizontal reference line is taken as the intersection point Pt of the left border of the door physical frame and the lower horizontal reference line l ;

[0107] Determine whether the inclination angle θ3 of the vertical reference line passing through the lower right vertex of the door recognition frame is greater than or equal to π / 2. If so, Figure 7 As shown, the intersection point of the right border of the door identification frame and the lower horizontal reference line is taken as the intersection point Pt of the right border of the door physical frame and the lower horizontal reference line r If not, if Figure 8 As shown, calculate the intersection point Pt′ of the upper horizontal reference line and the right frame of the door identification frame r , take the intersection point Pt′ r The intersection of the vertical reference line and the horizontal reference line below is taken as the intersection point Pt of the right border of the door physical frame and the horizontal reference line below r ;

[0108] like Figure 10 As shown, the first intersection point Pt is obtained by the above judgment ll : The intersection of the left border of the left door physical frame and the lower horizontal reference line,

[0109] The second intersection point Pt lr : The intersection of the right border of the left door physical frame and the lower horizontal reference line,

[0110] The third intersection point Pt rl : The intersection of the left border of the right door physical frame and the lower horizontal reference line,

[0111] The fourth intersection point Pt rr : The intersection of the right border of the right door physical frame and the lower horizontal reference line.

[0112] S33. Calculate the estimated position of the top vertex of the door's physical frame on the platform when the door's physical frame is tilted toward the platform, such as Figure 9 As shown, including:

[0113] Get the vertex Pt below the physical border of the car door d The intersection point Pt of the vertical reference line and the horizontal reference line above u , recorded as the upper intersection point, and the horizontal reference line below is used as the symmetry axis to obtain the symmetry point Pt′ u , recorded as the symmetrical intersection point, the symmetrical intersection point is the estimated position;

[0114] Get the symmetrical points of the first, second, third and fourth intersections respectively, and record them as the first symmetrical intersection point Pt′ ll , the second symmetric intersection point Pt′ lr , the third symmetric intersection point Pt′ rl , the fourth symmetric intersection point Pt′ rr ;

[0115] S34. If the left door or the right door is opened, obtain the first intersection point Pt ll , the first symmetric intersection point Pt′ ll , the fourth intersection point Pt rr , the fourth symmetric intersection point Pt′ rr The enclosed area is regarded as a danger zone;

[0116] If the left door is closed, get the first intersection point Pt ll , the first symmetric intersection point Pt′ ll , the second intersection point Pt lr , the second symmetric intersection point Pt′ lr The enclosed area is regarded as a danger zone;

[0117] If the right door is closed, get the third intersection point Pt rl , the third symmetric intersection point Pt′ rl , the fourth intersection point Pt rr , the fourth symmetric intersection point Pt′ rr The enclosed area is designated as a danger zone.

[0118] S4. Make operational safety assessments based on personnel, door opening and closing events, and hazardous areas. This includes:

[0119] Determine whether any identified personnel appear in the danger zone between the start time and the end time of the door opening and closing event. If so, trigger an alarm and then return to step S2. If not, return to step S2.

[0120] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for detecting the safety of boxcar door opening and closing operations in a railway freight yard, which analyzes images of boxcar compartments parked on the platform captured by a camera, and is characterized by: The following steps are involved: S1. Pre-set the camera surveillance image boxcar door reference line, along the boxcar door lower frame set a lower horizontal reference line; S2. Use a deep learning model to identify boxcar doors and people in surveillance images, group the doors to obtain multiple historical door grouping sequences, and identify door opening and closing events based on these historical door grouping sequences. S3. Calculate the physical door frame based on the preset reference line, and determine the danger zone on the platform by the physical door frame and the door opening and closing events; The interpolation method is used to obtain the inclination angle of the vertical reference line passing through each vertex of the door recognition frame; Tilt angle = (distance from vertex to right preset vertical reference line × tilt angle of left preset vertical reference line relative to horizontal) / (distance from vertex to left preset vertical reference line + distance from vertex to right preset vertical reference line) + (distance from vertex to left preset vertical reference line × tilt angle of right preset vertical reference line relative to horizontal) / (distance from vertex to left preset vertical reference line + distance from vertex to right preset vertical reference line); Calculate the intersection of the door's physical frame and the lower horizontal reference line, and determine the door's physical frame based on the vertical reference line; The area where the door's physical frame falls toward the platform is obtained using an axisymmetric method; When the left or right door is opened, the area where the physical frame of the left or right door falls towards the platform is the danger zone; when the left or right door is closed, the area where the physical frame of the left or right door falls towards the platform is the danger zone; S4. Make operational safety assessments based on personnel, door opening and closing events, and hazardous areas.

2. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 1, characterized in that The pre-setting of the boxcar door reference line in the camera monitoring image includes: Obtain an image of the boxcar captured by the camera, set an upper horizontal reference line along the upper frame of the boxcar door, and set a lower horizontal reference line along the lower frame of the boxcar door; Set multiple vertical reference lines along the vertical direction of the door.

3. The method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 1 is characterized in that The identification of the boxcar door and the person in the surveillance image includes: Collect one frame of image in real time; A trained deep learning model is used to identify the boxcar doors and people in the image. The door recognition result is a door recognition frame, including the coordinates of the upper left vertex and the width and height information of the door recognition frame. The person recognition result is a person recognition frame, including the coordinates of the upper left vertex and the width and height information of the person recognition frame.

4. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 3, characterized in that The step of grouping the vehicle doors to obtain a plurality of historical vehicle door grouping sequences includes: Sort the identified doors according to the camera shooting direction. When the camera is shooting to the right, sort the identified doors from left to right according to the center coordinates of the door identification frames. When the camera is shooting to the left, sort the identified doors from right to left according to the center coordinates of the door identification frames to obtain a sorted door sequence. Classify the doors that meet the first condition as belonging to the same compartment to obtain a door group sequence, where each door group includes the left and right doors of the same compartment. The first condition is that the distance between the center points of any adjacent first door identification frame and the second door identification frame in the sorted door sequence is less than 3 times the width of the first door identification frame, and the ratio of the area of ​​the first door identification frame to the area of ​​the second door identification frame is less than or equal to 8. Traverse each door group in the door group sequence and determine whether the door group has a geometric intersection with the last door group in each historical door group sequence. If there is an intersection, add the door group to the historical door group sequence. If there is no intersection or no historical door group sequence, create a new historical door group sequence and add the door group to the newly created historical door group sequence. After the traversal is completed, multiple historical door group sequences are obtained.

5. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 4, characterized in that The identifying of the door opening and closing event includes: Based on the historical door grouping sequence, if a door moves, it is judged as a door opening and closing event. The door movement threshold is set and the movement distance of each left door relative to the left door width and the movement distance of each right door relative to the right door width are calculated. If the left door moves to the left beyond the door movement threshold, it is judged as opening the left door, and the start and end times are recorded. If the distance the left door moves to the right exceeds the door movement threshold, it is judged as closing the left door, and the start time and end time are recorded. If the right door moves to the right beyond the door movement threshold, it is judged as opening the right door, and the start and end times are recorded. If the distance the right door moves to the left exceeds the door movement threshold, it is judged as closing the right door, and the start time and end time are recorded.

6. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 5, characterized in that The calculated physical frame of the boxcar door includes: The interpolation method is used to obtain the inclination angle of the vertical reference line of each vertex of the door identification frame passing through the door. If the camera is facing right, Determine whether the inclination angle of the vertical reference line passing through the upper right vertex of the door identification frame is less than or equal to π / 2. If so, take the intersection of the vertical reference line passing through the upper right vertex of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line. If not, take the intersection of the right border of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line. Determine whether the inclination angle of the vertical reference line passing through the lower left vertex of the door identification frame is less than or equal to π / 2. If so, take the intersection of the left border of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line. If not, calculate the intersection of the upper horizontal line and the left border of the door identification frame, and take the intersection of the vertical reference line passing through the intersection and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line; If the camera is facing left, Determine whether the inclination angle of the vertical reference line passing through the upper left vertex of the door identification frame is greater than or equal to π / 2. If so, take the intersection of the vertical reference line passing through the upper left vertex of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line. If not, take the intersection of the left border of the door identification frame and the lower horizontal reference line as the intersection of the left border of the door physical frame and the lower horizontal reference line. Determine whether the inclination angle of the vertical reference line passing through the lower right vertex of the door identification frame is greater than or equal to π / 2. If so, take the intersection of the right border of the door identification frame and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line. If not, calculate the intersection of the upper horizontal reference line and the right border of the door identification frame, and take the intersection of the vertical reference line passing through the intersection and the lower horizontal reference line as the intersection of the right border of the door physical frame and the lower horizontal reference line; The first intersection point is obtained from the above judgment: the intersection of the left border of the left door physical frame and the lower horizontal reference line, The second intersection point: the intersection of the right frame of the left door physical frame and the lower horizontal reference line, The third intersection point: the intersection of the left border of the right door physical border and the lower horizontal reference line, Fourth intersection point: the intersection of the right border of the right door physical frame and the lower horizontal reference line.

7. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 6, characterized in that Determining the dangerous area on the platform based on the door physical frame and door opening and closing events includes: Calculate the estimated position of the vertex above the door's physical frame on the platform when the door's physical frame is tilted toward the platform, including: Obtain the intersection of the vertical reference line passing through the lower vertex of the physical door frame and the upper horizontal reference line, record it as the upper intersection point, and obtain the symmetrical point with the lower horizontal reference line as the symmetry axis, record it as the symmetrical intersection point. The symmetrical intersection point is the estimated position; Obtain the symmetrical points of the first, second, third, and fourth intersections respectively, and record them as the first symmetrical intersection point, the second symmetrical intersection point, the third symmetrical intersection point, and the fourth symmetrical intersection point; If the left door or the right door is opened, the area enclosed by the first intersection, the first symmetrical intersection, the fourth intersection, and the fourth symmetrical intersection is obtained as the danger zone; If the left door is closed, the area enclosed by the first intersection, the first symmetrical intersection, the second intersection, and the second symmetrical intersection is obtained as the danger zone; If the right door is closed, the area enclosed by the third intersection, the third symmetrical intersection, the fourth intersection, and the fourth symmetrical intersection is obtained as the danger zone.

8. A method for detecting door opening and closing safety of boxcars in railway freight yards according to any one of claims 1 to 7, characterized in that The aforementioned operational safety assessment based on personnel, door opening and closing events, and hazardous areas includes: Determine whether any identified personnel appear in the danger zone between the start time and the end time of the door opening and closing event. If so, trigger an alarm and then return to step S2. If not, return to step S2.

9. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 4 or 5, characterized in that The identifying door opening and closing events further includes: A door gap threshold is set and the gap distance between the left and right doors in the door group is calculated. Based on the historical door grouping sequence, if the door moves and the gap distance is greater than the door gap threshold, it is judged as a non-door opening and closing event; otherwise, it is a door opening and closing event.

10. A method for detecting door opening and closing safety of boxcars in railway freight yards according to claim 4 or 5, characterized in that The identifying door opening and closing events further includes: According to the historical door grouping sequence, the leftmost horizontal coordinate of the left side horizontal coordinates of all door recognition frame center points and the rightmost horizontal coordinate of the right side horizontal coordinates of all door recognition frame center points are obtained. The distance between the leftmost horizontal coordinate and the rightmost horizontal coordinate is evenly divided into several moving segments. The number of segments in which the historical door recognition frame center points appear in the moving segments is counted as the movement smoothness index. It is judged whether the door movement smoothness index is less than the set index threshold. If so, the door opening and closing event is filtered out. Otherwise, it is a door opening and closing event.

Images