Automatic track safety monitoring method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of rail traffic safety, in particular to an automatic rail safety monitoring method, electronic equipment and a storage medium, human eyes are replaced with a pre-trained image safety area recognition model, and a rail position in a monitored image and a foreign object target possibly influencing train operation safety are automatically recognized and extracted. By comparing the horizontal coordinate values of the lower left corner and the lower right corner of the foreign matter target with the horizontal coordinate values of the left side line and the right side line of the safe area range determined according to the track position, whether the foreign matter target can cause a foreign matter invasion event or not can be accurately judged, namely whether a foreign matter invasion phenomenon exists or not. And prompt information is sent out in time when foreign matter invasion is confirmed. The automatic monitoring process effectively avoids the disadvantage caused by the fact that the current monitoring work of the foreign matters along the rail mainly depends on manual on-site inspection, the efficiency, accuracy and timeliness of the monitoring of the foreign matters along the rail are remarkably improved, and the incidence rate of foreign matter invasion events is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of rail transit safety technology, and in particular to an automated rail safety monitoring method, electronic device, and computer-readable storage medium. Background Technology

[0002] As a backbone of my country's transportation system, railway transportation has always revolved around the operational philosophy of "safe transportation and efficient development." The railway track is the infrastructure of railway transportation, bearing the weight of train operation and transport tasks, ensuring the efficient operation of the railway system. However, in recent years, due to insufficient protection measures, especially in the monitoring and protection of foreign objects along the track, there are significant shortcomings, leading to frequent incidents of foreign object intrusion. Foreign object intrusion not only poses a safety hazard to train operation but can also trigger serious safety accidents, threatening the safety of passengers and goods. Therefore, effective foreign object monitoring along the track is of great significance to ensuring the safety of railway transportation.

[0003] Currently, foreign object monitoring along railway lines mainly relies on manual on-site inspections. While this method can detect foreign objects to some extent, it suffers from drawbacks such as being time-consuming, labor-intensive, and lacking real-time accuracy. Furthermore, its efficiency and accuracy are affected by human factors, making it difficult to guarantee effective identification of foreign objects along the track. Therefore, there is an urgent need for an automated track safety monitoring method that is compatible with railway operations to reduce the incidence of foreign object intrusion incidents. Summary of the Invention

[0004] In view of this, the present invention proposes an automated track safety monitoring method, electronic device and computer-readable storage medium, aiming to solve the technical problem that existing trackside foreign object monitoring methods are unable to guarantee the effective identification of foreign objects along the track.

[0005] To solve the above problems, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention proposes an automated track safety monitoring method, comprising:

[0007] In response to acquiring a monitoring image of the monitoring area captured by the camera, the monitoring image is input into a pre-trained image safety zone recognition model to extract the track position from the monitoring image and determine the safety zone range on both sides of the track based on the track position; wherein, the safety zone range includes a left side line and a right side line, and the positions of the left side line and the right side line on the monitoring image are determined by the horizontal coordinate values ​​of the image coordinate system constructed by the monitoring image;

[0008] While inputting the monitoring image into the pre-trained image safe zone recognition model, if a foreign object target is extracted from the monitoring image, the horizontal coordinates of the lower left corner and the lower right corner of the foreign object target on the monitoring image are obtained; wherein, the horizontal coordinates of the lower left corner of the foreign object target are recorded as the lower left horizontal coordinates, and the horizontal coordinates of the lower right corner of the foreign object target are recorded as the lower right horizontal coordinates.

[0009] The horizontal coordinates of the left edge of the safe zone intersect the corresponding horizontal line of the distance marker of the foreign object distance camera in the monitoring image, and the horizontal coordinates of the right edge of the safe zone intersect the corresponding horizontal line of the distance marker of the foreign object distance camera in the monitoring image are obtained; wherein, the horizontal coordinate value of the left edge of the safe zone is recorded as the left horizontal coordinate value, and the horizontal coordinate value of the right edge of the safe zone is recorded as the right horizontal coordinate value;

[0010] If the horizontal coordinate value of the lower left corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then a foreign object intrusion is confirmed and a foreign object intrusion warning message is issued.

[0011] If the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then a foreign object intrusion is confirmed and a foreign object intrusion warning message is issued.

[0012] If the horizontal coordinate value of the lower left corner is not greater than the horizontal coordinate value of the left side and the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the right side, then a large foreign object intrusion is confirmed and a warning message indicating the presence of a large foreign object intrusion is issued.

[0013] Furthermore, the foreign object includes ground-based foreign objects and airborne foreign objects;

[0014] If the foreign object is a ground object, then the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image is the vertical coordinate value of the lower left or lower right corner of the monitoring image.

[0015] If the foreign object is an airborne foreign object, the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image corresponding to the horizontal line is obtained by mapping the actual distance between the foreign object and the camera obtained by the radar device or depth acquisition device onto the vertical coordinate value of the horizontal line in the monitoring image.

[0016] Determining whether the foreign object is a ground-based or airborne object includes:

[0017] Determine the pixel distance between the horizontal line mapped to the actual distance in the monitoring image and the horizontal line corresponding to the vertical coordinate value of the lower left or lower right corner of the monitoring image;

[0018] If the pixel distance is less than or equal to a preset error threshold, it is confirmed as a foreign object on the ground;

[0019] If the pixel distance is greater than a preset error threshold, it is confirmed as an airborne foreign object;

[0020] If it is confirmed to be a foreign object on the ground, a ground foreign object intrusion is confirmed and a ground foreign object intrusion warning message is issued;

[0021] If the object is confirmed to be an airborne foreign object, the horizontal line mapped to the actual distance in the monitoring image is obtained, and the vertical coordinate value of the upper edge of the space occupied by the carriage in the monitoring image is obtained based on the vertical coordinate value of the horizontal line. The space occupied by the carriage is obtained based on the aspect ratio of the carriage entity, and the space occupied by the carriage includes the upper edge. The position of the upper edge on the monitoring image is determined by the vertical coordinate value of the image coordinate system. The vertical coordinate value of the lower boundary of the foreign object is recorded as the lower vertical coordinate value, and the vertical coordinate value of the upper edge of the space occupied by the carriage is recorded as the upper vertical coordinate value.

[0022] If the lower vertical coordinate value is not greater than the upper vertical coordinate value, then an airborne foreign object intrusion is confirmed and a warning message is issued.

[0023] Furthermore, the confirmation of an aerial foreign object intrusion includes:

[0024] The radar equipment is activated to measure the distance between the radar equipment and the aerial foreign object entity corresponding to the foreign object target. If the distance value is not greater than a preset distance threshold, an aerial foreign object intrusion is confirmed; or

[0025] The depth acquisition device is activated to obtain a depth image of the monitored area. Based on the depth image, the depth value between the depth acquisition device and the aerial foreign object entity corresponding to the foreign object target is obtained. If the depth value is not greater than a preset depth threshold, the presence of an aerial foreign object intrusion is confirmed.

[0026] Further, the step of inputting the monitoring image into a pre-trained image safety zone recognition model to extract the track position from the monitoring image and determine the safety zone range on both sides of the track based on the track position includes:

[0027] The monitoring image is preprocessed to obtain a corresponding mask image. The mask image is then input into a pre-trained image security area recognition model to extract the track image from the mask image and obtain the pixel values ​​of the left boundary and the right boundary of the track image on the mask image. The pixel value of the left boundary of the track image is denoted as the left pixel value, and the pixel value of the right boundary of the track image is denoted as the right pixel value.

[0028] The track pixel width of the track image is calculated based on the left pixel value and the right pixel value, and the proportional relationship coefficient between the track pixel width and the track physical width of the track entity is obtained.

[0029] Calculate the scaling factor and the car pixel width; wherein the scaling factor is equal to the ratio of the track physical width to the car physical width of the car entity, and the car pixel width is equal to the product of the ratio and the track pixel width;

[0030] Based on the pixel width of the carriage and the physical width of the carriage, obtain a scale for the pixel width of the carriage and the physical width of the carriage;

[0031] The pixel width is widened according to a preset algorithm based on the track pixel width;

[0032] Determine the position of the safe zone range on both sides of the track on the mask image; wherein, the horizontal coordinate value of the left edge of the safe zone range is equal to the left pixel value minus the widened pixel width, and the horizontal coordinate value of the right edge of the safe zone range is equal to the right pixel value plus the widened pixel width;

[0033] The physical size range of the security zone is determined based on its position on the mask image, the scaling factor, and the scale.

[0034] Furthermore, the preset algorithm includes: the widened pixel width is equal to two-thirds of the track pixel width, or the widened pixel width is equal to half the difference between the carriage pixel width and the track pixel width.

[0035] Further, calculating the track pixel width of the track image based on the left pixel value and the right pixel value includes:

[0036] In the longitudinally extended track image, a horizontal line segment is selected, and the track pixel width of the track image is calculated based on the pixel values ​​of the left endpoint and the right endpoint of the horizontal line segment on the mask image.

[0037] The track pixel width is collected once at a preset interval to determine whether the track pixel width continues to increase within a preset time period;

[0038] If so, the track pixel width of the track image is calculated based on the bottom horizontal line segment near the camera in the track image;

[0039] If not, then continue to calculate the track pixel width of the track image based on the selected horizontal line segment.

[0040] Further, the step of extracting the track position from the monitoring image and determining the safety zone range on both sides of the track based on the track position includes:

[0041] Extract the orbit image from the monitoring image and obtain the orbit position of the orbit image in the monitoring image;

[0042] At least one key point is set in the track image according to a preset rule, and the vertical coordinate value of the key point in the monitoring image is obtained;

[0043] Obtain the physical vertical height of the key point corresponding to the track entity, and confirm whether the physical vertical height and vertical coordinate value of the key point match according to the size ratio;

[0044] If so, the safety zone range on both sides of the track is determined based on the track position;

[0045] If not, the vertical coordinate value of the key point is modified to match the physical vertical height; while the vertical coordinate value of the key point is modified, the track position is automatically modified synchronously, and the safety zone range on both sides of the track is determined based on the synchronously modified track position.

[0046] The modification of the vertical coordinate value of the key point includes: modifying the vertical coordinate value of the key point by moving the key point in the monitoring image, or modifying the vertical coordinate value of the key point by inputting a vertical coordinate value that matches the physical vertical height of the key point.

[0047] Furthermore, the safe zone is displayed in the monitoring image in the form of a first rectangle; wherein, the left side of the first rectangle represents the left edge of the safe zone, and the right side of the first rectangle represents the right edge of the safe zone.

[0048] The safe zone includes three ranges, and the distance between the actual safe area represented by the three ranges and the camera increases sequentially. The color and character identifier of the first rectangle of each range are different from the color and character identifier of the first rectangle of the other ranges.

[0049] The foreign object target is displayed in the monitoring image in the form of a second rectangle; wherein, the second rectangle surrounds the foreign object target to the greatest extent, the lower left corner of the second rectangle represents the lower left corner of the foreign object target, and the lower right corner of the second rectangle represents the lower right corner of the foreign object target.

[0050] Secondly, the present invention also provides an electronic device comprising:

[0051] Memory, used to store program instructions; and

[0052] A processor for executing the program instructions to implement the steps of the automated track safety monitoring method as described above.

[0053] Thirdly, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the automated track safety monitoring method as described above.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows: an automated track safety monitoring method, electronic device, and computer-readable storage medium, comprising: in response to acquiring a monitoring image of a monitoring area captured by a camera, inputting the monitoring image into a pre-trained image safety zone recognition model, so as to extract the track position in the monitoring image and determine the safety zone range on both sides of the track based on the track position; while inputting the monitoring image into the pre-trained image safety zone recognition model, if a foreign object target is extracted in the monitoring image, obtaining the horizontal coordinate values ​​of the lower left corner and the lower right corner of the foreign object target in the monitoring image; obtaining the horizontal coordinate values ​​of the left edge and the right edge of the safety zone range in the monitoring image; if the horizontal coordinate value of the lower left corner is not less than the horizontal coordinate value of the left side and the horizontal coordinate value of the lower right corner is not greater than the horizontal coordinate value of the right side, then confirming the presence of a foreign object intrusion and issuing a warning message indicating the presence of a foreign object intrusion. As can be seen, this automated track safety monitoring method, electronic equipment, and computer-readable storage medium utilize a pre-trained image safety zone recognition model to replace the human eye, automatically identifying and extracting track locations and foreign object targets that may affect train operation safety from the monitored images. By comparing the horizontal coordinates of the lower left and lower right corners of the foreign object target with the horizontal coordinates of the left and right edges of the safety zone determined based on the track location, it can accurately determine whether the foreign object target will cause a foreign object intrusion event, i.e., whether a foreign object intrusion exists, and promptly issue a warning message when a foreign object intrusion is confirmed. This automated monitoring process effectively avoids the drawbacks of current trackside foreign object monitoring work, which mainly relies on manual on-site inspections. By introducing neural network algorithm technology, it can automatically extract track locations and foreign object targets, and use a simple coordinate comparison method to determine whether a foreign object intrusion exists. This not only significantly improves the efficiency and accuracy of trackside foreign object monitoring but also effectively enhances the timeliness of track safety monitoring, greatly reduces the incidence of foreign object intrusion events, and thus effectively ensures the safety of railway transportation. Attached Figure Description

[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0056] Figure 1 This is a flowchart illustrating the first embodiment of the automated track safety monitoring method of the present invention;

[0057] Figure 2This is a schematic diagram of the first structure of the automated track safety monitoring method of the present invention for foreign object intrusion;

[0058] Figure 3 This is a schematic diagram of the second structure of the automated track safety monitoring method of the present invention for foreign object intrusion;

[0059] Figure 4 This is a schematic diagram of the third structure of the automated track safety monitoring method of the present invention for foreign object intrusion;

[0060] Figure 5 This is a schematic diagram of the structure of the automated track safety monitoring method of the present invention for large foreign object intrusion;

[0061] Figure 6 This is a schematic diagram of the first structure of the automated track safety monitoring method of the present invention for non-foreign object intrusion;

[0062] Figure 7 This is a schematic diagram of the second structure of the automated track safety monitoring method of the present invention, which is designed to prevent foreign object intrusion.

[0063] Figure 8 This is a flowchart illustrating the second embodiment of the automated track safety monitoring method of the present invention;

[0064] Figure 9 This is a flowchart illustrating the third embodiment of the automated track safety monitoring method of the present invention;

[0065] Figure 10 This is a flowchart illustrating the fourth embodiment of the automated track safety monitoring method of the present invention;

[0066] Figure 11 This is a flowchart illustrating the fifth embodiment of the automated track safety monitoring method of the present invention;

[0067] Figure 12 This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention. Detailed Implementation

[0068] The solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0070] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0071] Please see Figure 1 The diagram shown is a flowchart of the first embodiment of the automated track safety monitoring method of the present invention.

[0072] In this embodiment, the automated track safety monitoring method includes:

[0073] Step S100: In response to acquiring a monitoring image of the monitoring area captured by the camera, the monitoring image is input into a pre-trained image safety zone recognition model so as to extract the track position from the monitoring image and determine the safety zone range on both sides of the track based on the track position.

[0074] The safe zone includes a left edge and a right edge, and the positions of the left edge and the right edge on the monitoring image are determined by the horizontal coordinate values ​​of the image coordinate system constructed by the monitoring image.

[0075] Specifically, cameras are typically installed at the front or front of the train to monitor track conditions in real time. The monitored area consists of images of the track and surrounding environment within a certain distance ahead of the train. The image safety zone recognition model is a neural network-based algorithm that has been pre-trained on a large amount of monitoring image data to learn how to accurately identify and extract track positions and foreign object targets from the monitoring images. This image safety zone recognition model can effectively distinguish between the track and foreign objects (including ground and airborne foreign objects) to ensure the accuracy of automated track safety monitoring. The safety zone includes the left and right sidelines (the horizontal coordinates of the entire left and right sidelines are equal). Once a foreign object enters the area between the left and right sidelines, it is considered an intrusion and an alarm is triggered immediately. Conversely, foreign objects outside the left and right sidelines are considered safe and will not affect train operation. The monitoring images are captured by a camera fixed at the front of the train. The safety zone range can be accurately pre-calibrated based on the camera's lens parameters, installation position, and elevation angle, thus establishing a correlation between the coordinate values ​​of the monitoring image and the precise distance values ​​of the safety zone range. The image coordinate system constructed from the monitoring images can be a common two-dimensional Cartesian coordinate system (with the upper left corner as the origin, the x-axis to the right, and the y-axis downwards), a coordinate system based on the pixel values ​​in the monitoring image, a polar coordinate system, or other custom coordinate systems. In practical applications, a suitable coordinate system can be selected according to specific needs to better facilitate automated track safety monitoring. By selecting an appropriate coordinate system for precise coordinate comparison and judgment, the accuracy and efficiency of automated track safety monitoring can be improved.

[0076] Adding a safety distance to the left boundary of the track image creates the safety line on the left side of the track (the left edge of the safety zone). This safety line represents the safe position for the train within that location. The width variation between the safety line and the left boundary of the track image is proportional to the variation in track pixel width. This means that as the track pixel width changes, the distance between the safety line and the left boundary of the track image will adjust proportionally.

[0077] Adding a safety distance to the right boundary of the track image creates the safety line on the right side of the track (the right edge of the safety zone). This safety line represents the safe position for the train within that location. The width variation between the safety line and the right boundary of the track image is proportional to the variation in track pixel width. This means that as the track pixel width changes, the distance between the safety line and the right boundary of the track image will adjust proportionally.

[0078] Step S200: While inputting the monitoring image into the pre-trained image safe zone recognition model, if a foreign object target is extracted from the monitoring image, the horizontal coordinates of the lower left corner and the lower right corner of the foreign object target on the monitoring image are obtained.

[0079] Wherein, the horizontal coordinate value of the lower left corner of the foreign object target is recorded as the lower left horizontal coordinate value, and the horizontal coordinate value of the lower right corner of the foreign object target is recorded as the lower right horizontal coordinate value.

[0080] Specifically, after inputting the monitoring image into a pre-trained image safe zone recognition model, in addition to extracting the track position, it is also necessary to identify other foreign objects outside the track. For each identified and extracted foreign object, the system automatically defines a bounding box, which can be a rectangle or a polygon. Specifically, a rectangle is the smallest rectangle used to enclose the foreign object, while a polygon is generated by identifying the boundary points of the foreign object, forming a more precise shape. If the boundary of the foreign object is a polygon, a rectangle can also be generated based on the outer boundary of this polygon. This rectangle must be large enough to completely enclose the polygon. The lower left corner of the foreign object represents the leftmost and bottommost points of the defined shape of the foreign object in the monitoring image. The lower right corner of the foreign object represents the rightmost and bottommost points of the defined shape of the foreign object in the monitoring image.

[0081] In some embodiments of this application, the safe zone is displayed in the monitoring image in the form of a first rectangle; wherein the left side of the first rectangle represents the left edge of the safe zone, and the right side of the first rectangle represents the right edge of the safe zone.

[0082] Specifically, the safety zone is displayed in the monitoring image as a first rectangle. The left side of the first rectangle represents the left edge of the train's safe travel position, the top side represents the top edge of the train's safe travel position, and the right side represents the right edge of the train's safe travel position. Using the obtained left, top, and right edges, the positional relationship between the train car and the safety zone can be represented when the train reaches the actual position corresponding to the safety zone. The right side of the left edge includes the space occupied by the train car and the reserved left-side extended safety zone, and the left side of the right edge includes the space occupied by the train car and the reserved right-side extended safety zone.

[0083] In some embodiments of this application, the safe zone includes three ranges, and the distance values ​​between the actual safe areas represented by the three ranges and the camera increase sequentially. The color and character identifier of the first rectangle of each range are different from the color and character identifier of the first rectangle of the other ranges.

[0084] Specifically, the actual safe zone is calculated at different distances from the camera and displayed accordingly in the monitoring image as safe zone ranges. Typically, three incremental distance values ​​are set, such as 10 meters, 20 meters, and 30 meters, or 20 meters, 40 meters, and 60 meters. The safe zone range is displayed on the monitoring image as a rectangular frame (or safety warning frame), where the left side of the rectangle represents the left edge of the safe zone, and the right side represents the right edge. The entire rectangle is drawn in a specific color. The colors of different rectangles can be defined using color combinations from far to near, such as rainbow colors or spectral colors, or using predefined color bars corresponding to distances. Character labels for the corresponding distance values ​​can also be drawn along with the rectangles to distinguish different safe zone ranges from the camera at different distances. In addition to drawing the three safe zone ranges on the monitoring image, a label text, such as "Rail," can be added and displayed at a set position (e.g., 5 meters) from the camera to help users more intuitively understand the track information displayed in the monitoring image.

[0085] Specifically, taking three increasing distance values ​​of 10 meters, 20 meters, and 30 meters as examples, the vertical height (height of the rectangle) of the lower edge line of the three ranges (the straight line containing the lower edge line is called the safety warning distance line) on the monitoring image can be defined as standard_10m, standard_20m, and standard_30m, respectively.

[0086] standard_30m = h*2 / / 3, where h is the height of the monitored image, h*2 / / 3 represents 2 / 3 of the height of the monitored image, which is the vertical height of the first rectangle, and / / is the integer division operator, which means rounding down.

[0087] standard_20m = standard_30m + h / / 3 / / 2 / / 3, which means that the vertical height of the second rectangle is increased by a distance of h / / 3 / / 2 / / 3 based on the vertical height of the first rectangle.

[0088] standard_10m = standard_30m + h / / 3 / / 2, which means that the vertical height of the third rectangle is increased by a distance of h / / 3 / / 2 based on the vertical height of the first rectangle.

[0089] In addition, the vertical height of the label text is defined as label_5m = standard_10m + 24, which means that the vertical height of the label text is shifted downward by 24 pixels based on the vertical height of the third rectangle. The label text, such as "Rail", is displayed on the monitoring image at a position corresponding to the actual distance of 5 meters from the camera.

[0090] In some embodiments of this application, the foreign object target is displayed in the monitoring image in the form of a second rectangle (or the second rectangle is hidden and replaced by a foreign object target outline); wherein, the second rectangle surrounds the foreign object target to the greatest extent, the lower left corner of the second rectangle represents the lower left corner of the foreign object target, and the lower right corner of the second rectangle represents the lower right corner of the foreign object target.

[0091] Specifically, using a rectangular bounding box to represent foreign objects is a common visual identification method. This method can visually display the location and size of foreign objects on the monitoring image, facilitating observation and analysis and improving the accuracy and efficiency of foreign object identification. The second rectangular bounding box encloses the foreign object to the greatest extent possible, meaning its size is adjusted to be as large as possible to ensure it completely covers all parts of the foreign object. Since the second rectangular bounding box includes all relevant information about the foreign object, it effectively reduces the possibility of missed detections during subsequent foreign object extraction. The lower left corner coordinates (horizontal and vertical coordinates) of the second rectangular bounding box are the same as or close to the lower left corner coordinates (horizontal and vertical coordinates) of the foreign object. The lower right corner coordinates of the second rectangular bounding box are the same as or close to the lower right corner coordinates of the foreign object. This correspondence helps to accurately locate and analyze the shape and position of the foreign object.

[0092] Specifically, the complete process of clearly displaying the safe zone range on the monitoring image is as follows: First, the camera in the perception layer acquires the monitoring image. Then, through neural network algorithm technology (i.e., the monitoring image is identified and extracted through the image safe zone recognition model), the processed data is transmitted to the host chassis via serial communication. The host system then processes and saves the data again. Finally, the data is transmitted to the host display screen for display via serial communication.

[0093] Step S300: Obtain the horizontal coordinate value of the left side line of the safe zone when it intersects with the corresponding horizontal line of the distance mark of the foreign object distance camera in the monitoring image, and the horizontal coordinate value of the right side line when it intersects with the corresponding horizontal line of the distance mark of the foreign object distance camera in the monitoring image.

[0094] The horizontal coordinate value of the left edge of the safe zone is denoted as the left horizontal coordinate value, and the horizontal coordinate value of the right edge of the safe zone is denoted as the right horizontal coordinate value.

[0095] Step S401: If the horizontal coordinate value of the lower left corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then it is confirmed that there is a foreign object intrusion and a prompt message indicating that there is a foreign object intrusion is issued.

[0096] Step S402: If the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then it is confirmed that there is a foreign object intrusion and a prompt message indicating that there is a foreign object intrusion is issued.

[0097] Step S403: If the horizontal coordinate value of the lower left corner is not greater than the horizontal coordinate value of the left side and the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the right side, then a large foreign object intrusion is confirmed and a prompt message indicating the presence of a large foreign object intrusion is issued.

[0098] It should be noted that the alert messages for foreign object intrusion and large foreign object intrusion can be the same, such as "Foreign Object Intrusion Alert," to promptly detect potential security threats. Alternatively, the alert messages for foreign object intrusion and large foreign object intrusion can differ, such as "Foreign object intrusion detected, please be aware" and "Large foreign object intrusion detected, please handle as soon as possible," to allow for timely implementation of appropriate security measures based on the different types of intruding objects, ensuring a swift response to potential security risks.

[0099] Specifically, please refer to Figures 2 to 4 As shown, if the horizontal coordinate value of the lower left corner falls between the horizontal coordinate values ​​of the left and right sides, it indicates that an intrusion has occurred on the right side of the lower left corner of track 20; if the horizontal coordinate value of the lower right corner falls between the horizontal coordinate values ​​of the left and right sides, it indicates that an intrusion has occurred on the left side of the lower right corner of track 20. If both of these situations occur simultaneously, it indicates that the foreign object 10 has completely intruded into the safe zone; if only one of these situations occurs, it indicates that the foreign object 10 is partially located within the safe zone. Please refer to [link / reference]. Figure 5 As shown, if the horizontal coordinate value of the lower left corner is less than the horizontal coordinate value of the left side and the horizontal coordinate value of the lower right corner is greater than the horizontal coordinate value of the right side, it indicates that a large foreign object has crossed the entire safety zone of track 20. Conversely, please refer to [link to relevant documentation]. Figure 6 and Figure 7 As shown, if the horizontal coordinate value of the lower right corner is less than the horizontal coordinate value of the left side, or the horizontal coordinate value of the lower left corner is greater than the horizontal coordinate value of the right side, it indicates that there is no foreign object 10 intrusion.

[0100] Specifically, when the monitored image is input into a pre-trained image security zone recognition model for foreign object target identification, multiple foreign object targets may be extracted simultaneously. Each foreign object target needs to be assessed for the presence of foreign object intrusion. This assessment includes foreign object target overlap parameter 1 and foreign object target overlap parameter 2. Wherein,

[0101] The foreign object overlap parameter 1 is as follows: The horizontal coordinate value of the lower right corner of the foreign object is compared with the horizontal coordinate value of the left edge of the safe zone (specifically, the coordinates of the intersection point of the left edge and the horizontal line where the bottom edge of the foreign object's rectangle is located can be selected. The vertical coordinate value of this intersection point is equal to the vertical coordinate value of the horizontal line where the bottom edge of the foreign object's rectangle is located, and the horizontal coordinate value of this intersection point is the horizontal coordinate value of the left edge of the safe zone). When the horizontal coordinate value of the lower right corner of the foreign object is located to the right of the left edge of the safe zone, the comparison is made.

[0102] The foreign object target overlap parameter 2 is: compare the horizontal coordinate value of the lower left corner of the foreign object target with the horizontal coordinate value of the right edge of the safe zone. When the horizontal coordinate value of the lower left corner of the foreign object target is located to the left of the right edge of the safe zone.

[0103] When both the foreign object overlap parameter 1 and the foreign object overlap parameter 2 are satisfied, it indicates that the foreign object has invaded the safe zone, and an alarm for the foreign object invading the safe zone should be triggered immediately.

[0104] Compared with existing technologies, the automated track safety monitoring method of this invention utilizes a pre-trained image safety zone recognition model to replace the human eye, automatically identifying and extracting track positions and foreign object targets that may affect train operation safety from the monitoring images. By comparing the horizontal coordinates of the lower left and lower right corners of the foreign object target with the horizontal coordinates of the left and right edges of the safety zone determined based on the track position, it can accurately determine whether the foreign object target will cause a foreign object intrusion event, i.e., whether a foreign object intrusion occurs, and promptly issue a warning message when a foreign object intrusion is confirmed. This automated monitoring process effectively avoids the drawbacks of current trackside foreign object monitoring work, which mainly relies on manual on-site inspections. By introducing neural network algorithm technology, it can automatically extract track positions and foreign object targets, and use a simple coordinate comparison method to determine whether a foreign object intrusion has occurred. This not only significantly improves the efficiency and accuracy of trackside foreign object monitoring, but also effectively improves the timeliness of track safety monitoring, greatly reduces the incidence of foreign object intrusion events, and thus effectively ensures the safety of railway transportation.

[0105] Please see Figure 8 The diagram shown is a flowchart of a second embodiment of the automated track safety monitoring method of the present invention. This embodiment, based on other embodiments of the automated track safety monitoring method, provides a detailed description of the steps for issuing alerts based on the type of intruding foreign object.

[0106] The foreign objects include ground-based foreign objects and airborne foreign objects;

[0107] If the foreign object is a ground object, then the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image is the vertical coordinate value of the lower left or lower right corner of the monitoring image.

[0108] If the foreign object is an airborne foreign object, the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image corresponding to the horizontal line is obtained by mapping the actual distance between the foreign object and the camera obtained by the radar device or depth acquisition device onto the vertical coordinate value of the horizontal line in the monitoring image.

[0109] Determining whether the foreign object is a ground-based or airborne object includes:

[0110] Determine the pixel distance between the horizontal line mapped to the actual distance in the monitoring image and the horizontal line corresponding to the vertical coordinate value of the lower left or lower right corner of the monitoring image;

[0111] If the pixel distance is less than or equal to a preset error threshold, it is confirmed as a foreign object on the ground;

[0112] If the pixel distance is greater than a preset error threshold, it is confirmed as an airborne foreign object;

[0113] Specifically, there are many methods to distinguish whether a foreign object is a ground object or an airborne object. For example, a pre-trained image-based safe zone recognition model can be used, trained on a large amount of image data of both ground and airborne objects, to learn how to accurately identify them. Alternatively, sensors (such as LiDAR or ultrasonic sensors) can be used to measure the height of the foreign object. If the height of the foreign object is below a certain threshold (e.g., a few centimeters above the ground), it can be identified as a ground object; if the height exceeds the threshold, it is identified as an airborne object. Examples of these methods will not be elaborated upon here.

[0114] In this embodiment, the process of confirming the presence of ground or airborne foreign objects and issuing corresponding alerts includes:

[0115] Step S411: If it is confirmed to be a foreign object on the ground, then confirm the presence of a foreign object intrusion and issue a prompt message indicating the presence of a foreign object intrusion.

[0116] Step S412: If it is confirmed to be an airborne foreign object, obtain the horizontal line that maps the actual distance to the monitoring image, and obtain the vertical coordinate value of the upper side line of the space occupied by the carriage on the monitoring image according to the vertical coordinate value of the horizontal line.

[0117] The space occupied by the carriage is obtained based on the length-to-width ratio of the carriage body. The space occupied by the carriage includes the upper side line. The position of the upper side line on the monitoring image is determined by the vertical coordinate value of the image coordinate system. The vertical coordinate value of the lower boundary of the foreign object target is recorded as the lower vertical coordinate value, and the vertical coordinate value of the upper side line of the space occupied by the carriage is recorded as the upper vertical coordinate value.

[0118] Specifically, the space occupied by the carriage represents the area occupied by the train carriage. This area can be represented on the monitoring image as a carriage space map, such as displaying the train's display frame line. Generally, the aspect ratio of the actual train (carriage entity) is fixed. Therefore, the aspect ratio of the carriage image displayed on the monitoring image is also fixed. When the line segment formed by the intersection of the left and right edges of the first rectangle and the safety warning distance line is used as the bottom line of the carriage, the train display frame line (rectangular frame) can be calculated based on the aspect ratio of the carriage entity. Of course, a corresponding polygonal frame can also be drawn based on the actual cross-sectional shape of the carriage. At the position of the nth safety warning distance line, drawing the train width and height indication areas will yield the train display frame line. This train display frame line can visually represent the space occupied by the train at that position.

[0119] Step S413a: If the lower vertical coordinate value is not greater than the upper vertical coordinate value, then the radar device is activated to measure the distance between the radar device and the aerial foreign object entity corresponding to the foreign object target. If the distance value is not greater than a preset distance threshold, then the presence of aerial foreign object intrusion is confirmed and a warning message of aerial foreign object intrusion is issued.

[0120] Step S413b: If the lower vertical coordinate value is not greater than the upper vertical coordinate value, then the depth acquisition device is activated to obtain the depth image of the monitoring area. Based on the depth image, the depth value between the depth acquisition device and the aerial foreign object entity corresponding to the foreign object target is obtained. If the depth value is not greater than a preset depth threshold, then the presence of aerial foreign object intrusion is confirmed and a warning message indicating the presence of aerial foreign object intrusion is issued.

[0121] It should be noted that there is no specific order between steps S413a and S413b; either step can be performed.

[0122] Specifically, after identifying the type of foreign object, if it is confirmed to be a ground-based foreign object, the conclusion of whether ground-based foreign object intrusion exists can be directly drawn. If it is confirmed to be an airborne foreign object, further airborne foreign object intrusion judgment is required. That is, the upper edge of the space occupied by the carriage is used as an auxiliary basis for airborne foreign object intrusion warning judgment. When the lower boundary vertical coordinate value of the airborne foreign object target is lower than the upper vertical coordinate value of the upper edge of the space occupied by the carriage, it indicates that airborne foreign object intrusion exists.

[0123] The automated track safety monitoring method of this invention achieves near 100% accuracy in detecting ground-based foreign object intrusions, regardless of the type of foreign object. However, for airborne foreign objects, the detection is less accurate due to the inability to precisely represent spatial distances in two-dimensional images, leading to a possibility of misjudgment, although early warning is still achievable. Therefore, after initial assessment of airborne foreign object intrusions, further manual visual inspection or secondary assessment using methods such as radar or depth-of-field acquisition equipment can improve the accuracy of airborne foreign object intrusion detection. This automated track safety monitoring method effectively solves the current problem of relying solely on manual processing of railway condition video images, allowing for greater utilization of video information.

[0124] Please see Figure 9 The diagram shown is a flowchart of the third embodiment of the automated track safety monitoring method of the present invention. This embodiment, based on other embodiments of the automated track safety monitoring method, provides a detailed description of the steps for extracting the track position and determining the safety zone range on both sides of the track based on the track position.

[0125] In this embodiment, step S100 includes:

[0126] Step S111: In response to acquiring a monitoring image of the monitoring area captured by the camera, the monitoring image is preprocessed to obtain a corresponding mask image. The mask image is then input into a pre-trained image security area recognition model to extract the track image from the mask image and obtain the pixel values ​​of the left boundary of the track image and the right boundary of the track image on the mask image.

[0127] The pixel value at the left boundary of the track image is denoted as the left pixel value, and the pixel value at the right boundary of the track image is denoted as the right pixel value.

[0128] Specifically, a mask image is a binary image where the value of each pixel indicates the state of the corresponding pixel in the original image. Processing a mask image significantly reduces the number of pixels that need to be processed compared to processing the original monitoring image, thus improving computational efficiency. This is because a mask image only retains pixel information of the region of interest, ignoring background or irrelevant areas, thereby reducing the computational burden. Focusing only on the target object (such as orbital position and foreign objects) avoids complex calculations on the entire monitoring image. Furthermore, mask images can reduce background noise interference, enhance feature extraction of target objects, and improve the accuracy of subsequent identification and analysis.

[0129] Specifically, this embodiment uses a pixel-value-based coordinate system. In this system, the top-left corner of the monitored image is considered the origin (0,0), the x-axis represents the horizontal pixel index (column index), and the y-axis represents the vertical pixel index (row index). The left pixel value is the x-coordinate value in the coordinate system, representing the horizontal position of the left boundary of the track image in the mask image. The right pixel value is also the x-coordinate value in the coordinate system, representing the horizontal position of the right boundary of the track image in the mask image. In this way, the position of the track image in the mask image can be accurately located, providing necessary information for subsequent analysis and processing of the safe zone.

[0130] Step S112: Calculate the track pixel width of the track image based on the left pixel value and the right pixel value, and obtain the proportional relationship coefficient between the track pixel width and the track physical width of the track entity.

[0131] Specifically, the track pixel width represents the pixel width occupied by the track image in the masked image. The physical width of the track entity refers to the actual width of the track, usually measured in meters (m). This value can be obtained by measurement or based on standard track widths (e.g., the standard railway track width is 1435 mm). The scaling factor = track pixel width / physical width of the track entity. By calculating and analyzing the track pixel width and the scaling factor, necessary data support is provided for subsequent analysis and processing of the safety zone range.

[0132] Step S113: Calculate the scaling factor and the carriage pixel width.

[0133] Wherein, the scaling factor is equal to the ratio of the physical width of the track to the physical width of the carriage entity, and the carriage pixel width is equal to the product of the ratio and the track pixel width.

[0134] Specifically, by calculating the pixel width of the carriage, an important parameter can be provided for drawing the train booth outline in the monitoring image, which helps to improve the accuracy and reliability of the drawing.

[0135] Step S114: Obtain the scale of the vehicle's pixel width and physical width based on the vehicle's pixel width and physical width.

[0136] Specifically, by calculating and analyzing the scale of the car's pixel width and physical width, necessary data support is provided for the subsequent analysis and processing of the safety zone range.

[0137] Step S115: Calculate the widened pixel width according to the preset algorithm based on the track pixel width.

[0138] Specifically, trains, such as locomotives, are relatively large, especially during operation, and the width and height of the carriages affect the safety zones on both sides of the track. Relying solely on track position and foreign object detection may not be sufficient to fully assess the safety of a moving train. Therefore, calculating a wider pixel width effectively takes into account the train's size, ensuring that potential safety hazards can be effectively identified during monitoring. This wider pixel width calculation allows for the establishment of reasonable safety boundaries on both sides of the track, ensuring that any possible obstacles or foreign objects can be promptly identified and addressed when a train passes, effectively reducing the risk of foreign object intrusion accidents and improving overall safety.

[0139] In some embodiments of this application, the preset algorithm includes: the widened pixel width is equal to two-thirds of the track pixel width, or the widened pixel width is equal to half the difference between the carriage pixel width and the track pixel width.

[0140] Specifically, the widened pixel width can be calculated based on the actual width of the train and track, such as a train width of 3.3 meters and a track width of 1.4 meters, i.e., 1.4*2 / 3+1.4+1.4*2 / 3≈3.3 meters, and the proportional relationship.

[0141] The reference standards for train width are as follows:

[0142] Passenger carriages: The width is generally 3.3 meters (at the door handrail) or 3.1 meters (vehicle width).

[0143] Freight car body: The width is generally 2.8 meters.

[0144] Special carriages: For example, there are two types of train mail carriages: Type 21 and Type 22. The Type 21 carriage is 22 meters long, and the Type 22 carriage is 23.6 meters long.

[0145] Diesel locomotives: For example, the width of the DF8B train is 3304mm (at the door handrail) and 3100mm (car width); the width of the DF4DK train is 3304mm (at the door handrail) and 3100mm (car width); the width of the DF7G train is 3100mm; the width of the ND5 train is 3276mm; the width of the HXN3 train is 3100mm; the width of the HXN5 train is 3119mm; and the width of the HXN5B train is 3284mm.

[0146] The international standard railway gauge is 1435 mm.

[0147] Specifically, using two-thirds of the track pixel width as the widening pixel width is a simple and uniform approach that ensures sufficient safety margins in most cases, effectively preventing potential safety hazards. However, different railway environments and train types may lead to varying safety requirements. An algorithm that sets the widening pixel width to half the difference between the carriage pixel width and the track pixel width is more flexible. It can be adjusted according to specific train and track conditions, more accurately reflecting actual safety requirements and ensuring that the safety zone on both sides of the track adapts to different situations when a train passes. For example, in some cases, the train's physical width is larger, potentially requiring a wider safety zone; while in other cases, the track's physical width is narrower, possibly requiring only a smaller widening pixel width. By flexibly using both algorithms and their combination, it is possible to better adapt to various train operating environments.

[0148] Step S116: Determine the location of the safe zone range on both sides of the track on the mask image.

[0149] Wherein, the horizontal coordinate value of the left edge of the safe zone is equal to the left pixel value minus the widened pixel width, and the horizontal coordinate value of the right edge of the safe zone is equal to the right pixel value plus the widened pixel width.

[0150] Specifically, based on the widened pixel width and the left and right boundaries of the track image, the positions of the left and right edges of the safety zone on the mask image can be determined. By reserving widened pixel widths at the left and right boundaries of the track image, it can be ensured that the safety zones on both sides of the track can effectively cover potential safety hazards during train operation.

[0151] Step S117: Determine the physical size range of the security area based on the position of the security area range on the mask image, the scaling factor, and the scale bar.

[0152] Specifically, after determining the location of the safe zone on the masked image, it is also necessary to understand the actual physical size of this zone to help railway operators make reasonable plans and designs to ensure sufficient safety margin during train operation.

[0153] The automated track safety monitoring method of this invention aims to overcome the shortcomings of existing track safety monitoring technologies. Most existing track safety monitoring technologies rely on manual methods to discover and handle problems, lacking sufficient intelligence. Although some technologies introduce neural network algorithms, most are limited to track identification and foreign object identification, failing to fully consider the unsafe zones existing on both sides of the track due to the large size of trains. Therefore, this automated track safety monitoring method integrates multiple technologies and improves the neural network algorithm, expanding the display of the safe zone range on both sides of the track. This improvement can better identify intrusive foreign objects that may affect train operation safety, improving the accuracy and efficiency of track safety monitoring and providing a more reliable safety guarantee for train operation.

[0154] Please see Figure 10 The diagram shown is a flowchart of the fourth embodiment of the automated track safety monitoring method of the present invention. This embodiment, compared to other embodiments of the automated track safety monitoring method, provides a detailed description of the step of calculating the track pixel width of the track image.

[0155] In this embodiment, step S112 includes:

[0156] Step S1121: Select a horizontal line segment in the longitudinally extended track image, and calculate the track pixel width of the track image based on the pixel values ​​of the left endpoint and the right endpoint of the horizontal line segment on the mask image.

[0157] Specifically, in the monitoring image, the track extends longitudinally, meaning the primary viewpoint of the monitoring image is along the direction of train travel. This longitudinal extension makes the track appear as a long strip in the monitoring image. The horizontal line segment is a line segment corresponding to the width direction of the track entity, typically horizontal. Its function is to measure the width of the track, i.e., the distance from the left edge to the right edge. In a longitudinally extended track image, a horizontal line segment can be selected. This horizontal line segment preferably passes through the central area of ​​the track image to ensure that it covers the entire width of the track entity. Specifically, a suitable height can be selected in the track image (e.g., half the overall height of the track image), and a horizontal line segment can be drawn at that height. By measuring the pixel values ​​of the left and right endpoints of this horizontal line segment on the mask image, the pixel width of the track can be calculated.

[0158] Step S1122: Collect the track pixel width once every preset period, and determine whether the track pixel width continues to increase within a preset time period.

[0159] Specifically, a fixed time interval (e.g., a preset period of one second) is set to collect track pixel width data. Each time data is collected, the current track pixel width data is recorded and stored in an array or list for subsequent analysis. At track junctions, there are usually branching points, causing the track pixel width to show a gradually increasing trend in the monitoring image. At non-junctions, the track width is usually relatively stable, and the track pixel width does not show significant changes in the monitoring image over a period of time. A preset time period (e.g., a preset duration of 20 seconds) is set, and the collected track pixel width data is continuously observed within this time period. If the track pixel width continuously increases within the preset duration, it indicates a junction; if the track pixel width does not continuously increase within the preset duration, it indicates a non-junction. It should be noted that the automated track safety monitoring method of this embodiment involves a continuous monitoring process to analyze and determine whether the track a train is passing through is a junction or a non-junction.

[0160] Step S1123a: If yes, then calculate the track pixel width of the track image based on the bottom horizontal line segment near the camera side in the track image.

[0161] Specifically, at level crossings, the shape and width of the track structure change. Monitoring images captured by cameras are often affected by perspective effects, especially when there are level crossings. The portion closer to the camera will appear wider, while the portion farther from the camera may appear narrower due to perspective. Therefore, selecting the bottom horizontal line segment closer to the camera can more accurately reflect the actual track width, as this portion of the track width is not affected by perspective distortion in the monitoring image.

[0162] Step S1123b: If not, continue to calculate the track pixel width of the track image based on the selected horizontal line segment.

[0163] Specifically, at non-intersection locations, the width of the track entity remains stable, and the previously selected horizontal line segment can continue to be used to calculate the track pixel width.

[0164] It should be noted that there is no specific order between steps S1123a and S1123b; either step can be performed.

[0165] Step S1124: Obtain the proportional relationship coefficient between the track pixel width and the track physical width of the track entity.

[0166] The automated track safety monitoring method in this embodiment determines whether a track is a junction by periodically collecting and analyzing the changes in track width and state characteristics. Based on the determination results, the horizontal line segment used to calculate the track pixel width is adjusted to better reflect the actual track changes, thereby obtaining a more accurate result for foreign object intrusion.

[0167] Please see Figure 11 The diagram shown is a flowchart of the fifth embodiment of the automated track safety monitoring method of the present invention. Compared with other embodiments of the automated track safety monitoring method, this embodiment adds a step of synchronously modifying the track position in the monitoring image according to the slope change of the track entity.

[0168] In this embodiment, step S100 includes:

[0169] Step S121: In response to acquiring the monitoring image of the monitoring area captured by the camera, the monitoring image is input into the pre-trained image security zone recognition model.

[0170] Step S122: Extract the track image from the monitoring image and obtain the track position of the track image in the monitoring image.

[0171] Specifically, the undulations of railway tracks generally follow industry standards, with relatively small changes in gradient. This means that within a certain distance, the changes in the monitoring images captured by the cameras will not be too drastic. This stability ensures a relatively consistent distance relationship between the monitoring images and the cameras, which helps improve the accuracy of analysis and processing.

[0172] Step S123: Set at least one key point in the track image according to a preset rule, and obtain the vertical coordinate value of the key point in the monitoring image.

[0173] Specifically, although the gradient of railway tracks varies relatively little, in certain situations, such as mountainous or uneven terrain, the width of track pixels in monitoring images may still be affected by changes in gradient. This means that under different gradient conditions, there may be an error between the actual physical width of the track and the width of track pixels displayed in the monitoring image. By using sensors, laser rangefinders, or other measuring tools to measure or calculate, the physical vertical height of key points corresponding to the track entity can be obtained. The relationship between the vertical coordinates of the key points in the monitoring image and their physical vertical height can be confirmed through a proportional relationship, ultimately allowing a judgment on whether the vertical coordinates of the key points match the physical vertical height.

[0174] Step S124: Obtain the physical vertical height of the key point corresponding to the track entity, and confirm whether the physical vertical height and vertical coordinate value of the key point match according to the size ratio.

[0175] Specifically, key points can be selected at track corners, as these locations typically exhibit significant track condition changes; alternatively, key points can be placed at track sharp angles, which helps to better capture track geometric variations; and track joints can also be chosen as key points, as they may affect train stability. Pre-calibration data using pixel values ​​from monitoring images is used to obtain baseline vertical coordinates for these key points at specific distances (e.g., 100 meters, 50 meters, and 25 meters from the camera), providing crucial reference data for subsequent track monitoring.

[0176] Step S125a: If so, determine the safety zone range on both sides of the track based on the track position.

[0177] Step S125b: If not, modify the vertical coordinate value of the key point to match the physical vertical height; while modifying the vertical coordinate value of the key point, the track position is automatically modified synchronously, and the safety zone range on both sides of the track is determined based on the synchronously modified track position.

[0178] The modification of the vertical coordinate value of the key point includes: modifying the vertical coordinate value of the key point by moving the key point in the monitoring image, or modifying the vertical coordinate value of the key point by inputting a vertical coordinate value that matches the physical vertical height of the key point.

[0179] It should be noted that there is no specific order between steps S125a and S125b; either step can be performed.

[0180] Specifically, if the physical vertical height and vertical coordinate value of the key point match, it means that the track slope change is within an acceptable range and the track condition is stable. The basic data for analysis and processing is reliable, and there is no need to modify the vertical coordinate value of the key point.

[0181] If the physical vertical height and vertical coordinate values ​​of a key point do not match, the vertical coordinate values ​​of the key point need to be modified. This mismatch can lead to incorrect delineation of the safety zone, increasing safety hazards. The vertical coordinate values ​​of the key point must be adjusted to ensure they match the physical vertical height. Only when both match, and based on the automatically synchronized and modified track position, can the safety zone boundaries on both sides of the track be reassessed and determined to guarantee the effectiveness of safety monitoring.

[0182] When the vertical coordinates of keypoints are modified, the track position is automatically updated synchronously. Specifically, the new track position is calculated using a preset formula based on the amount of change in the keypoints and their positional relationship on the track. After the new track position is calculated, all track position data related to the keypoints are updated in batches to ensure the consistency and accuracy of all related data.

[0183] The automated track safety monitoring method in this embodiment can automatically and synchronously modify the track position in the monitoring image based on changes in the vertical coordinate values ​​of key points, thereby ensuring the consistency and accuracy of the overall data. This synchronous modification mechanism not only improves the efficiency of data processing but also reduces the possibility of human error.

[0184] The following details the implementation process of the automated track safety monitoring method in this embodiment:

[0185] 1. Obtain the polygon of the orbital target by identifying it using a neural network algorithm.

[0186] By training an image safety zone recognition model to identify closed areas of railway tracks, the closed areas of tracks in the monitoring images are found and marked, that is, the track images and track locations are extracted.

[0187] 2. Fill the interior of closed vertices of the polygon with pixels.

[0188] First, by connecting the vertices of the polygon, a polygon image is drawn in the monitoring image. Then, the interior of the polygon image is filled with color to obtain a solid shape, which is a single color.

[0189] 3. Based on the track width in the image and the safety distance line in the pixel calibration width direction in the image.

[0190] The track image can be understood as two parallel lines. As the monitoring image extends longitudinally, its actual physical width (the physical width of the track entity) remains constant in non-intersection areas. Since a polygon is a polygonal region enclosing the left and right rails (tracks), the left boundary of the polygon can be understood as the left boundary of the left track, and the right boundary as the right boundary of the right track. The number of pixels on the left and right boundaries represents the track pixel width. The ratio coefficient between the track physical width and the track pixel width is the ratio coefficient between that horizontal pixel and the physical size.

[0191] 4. Determine whether it is a polygonal area of ​​a branch line region based on the extension trend of the track pixel width.

[0192] The polygon in a branching area may be a graphic consisting of multiple intersecting tracks. The pixel distance between the left and right boundary points on the same horizontal line is no longer represented by the track pixel width. Therefore, when the track pixel width shows an increasing trend and is continuous, it is represented as a branching section. In branching sections, the track pixel width must be recalculated based on the track extension trend.

[0193] 5.1 Branching area, obtaining track factor

[0194] Based on the increasing extension trend of the track, the track pixel width and the proportional relationship coefficient K are calculated based on the bottom horizontal line segment near the camera side (bottom of the monitored image).

[0195] 5.2 Non-branching area, obtaining track factor

[0196] The proportional relationship coefficient K is obtained by using the track pixel width W1 on the polygonal horizontal line and the track physical width W0.

[0197] 6. Based on the pixel width of the carriage and the pixel width of the track, obtain the carriage-track scaling factor M, and obtain the future left and right boundaries of the carriage on the track.

[0198] Based on the track object width W0 and the carriage physical width Wc, the scaling factor M = Wc / W0 is obtained. Based on the track pixel width W1, the carriage pixel width W2 = W1 * M is calculated. The extended pixel widths on the left and right sides W4 = (W2 - W1) / 2.

[0199] Specifically, the left edge line of the safety zone is plotted at position W4 to the left of the left boundary of the polygon, and the right edge line of the track safety zone is plotted at position W4 to the right of the right boundary of the polygon. The safety zone lines extending from the left and right sides of the polygon are calculated continuously, line by line or skipping lines in the monitoring image. Finally, the result can be displayed on the screen as a track polygon in the middle, with a safety line on each side.

[0200] 7. Using cameras installed according to fixed standards, monitor the relationship between the image and the actual distance, and calibrate the vertical pixel position at a specific distance in front.

[0201] The undulations ahead of the railway have industry standards, and the gradient changes very little. The distance relationship between the monitoring image and the camera fluctuates relatively little. The image pixel pre-calibration data can be used to obtain the corresponding distance values ​​of specific key points, such as the reference vertical coordinate values ​​of the 100-meter position, 50-meter position, 25-meter position, etc.

[0202] 8. Revise critical distance error based on track width (optional)

[0203] While industry standards stipulate that the gradient changes along the railway track should generally be small, variations in the gradient can still sometimes significantly impact the width of track pixels in the track image. To mitigate this, the distance accuracy of keypoints can be improved and optimized based on the ratio of their physical vertical height to their vertical coordinates. This optimization can be achieved by adjusting either the vertical coordinates of the keypoints or their position.

[0204] Safety frame of the carriage at 9 key safety distance positions

[0205] At the safety warning distance line position, the car pixel width is calculated based on the track pixel width. Then, based on the measured aspect ratio of the physical dimensions of the car entity, a car occupancy space diagram (car safety block diagram) is drawn at the safety warning distance line position. Rectangles or polygons can be used to simulate the future occupancy of the car at this position.

[0206] 10 key points with safety distance markings, explaining key information about the safety distance.

[0207] Draw label text near the safety warning distance line to explain relevant information.

[0208] 11 Identify ground intrusion objects

[0209] This automated track safety monitoring method focuses on identifying and issuing warnings of ground object intrusions. Foreign objects outside the track detected by the image safety zone recognition model are considered potential intruders, and their maximum bounding boxes (or polygonal boxes) are used to define their maximum bounding boxes. The purpose of this automated track safety monitoring method is to simplify calculations and facilitate rapid judgment; therefore, each foreign object has a maximum bounding box. If the line segment connecting the bottom two vertices of this maximum bounding box overlaps with the line segment connecting the vertices of the two sides of the safety zone at the same vertical coordinate values, it indicates that the foreign object has intruded into the track's safety zone.

[0210] 12 Warning labels for identifying ground intrusion objects

[0211] Upon detecting an intrusion of a foreign object on the ground, the system annotates the image with color changes to generate an early warning, which can be combined with various warning prompts such as photoelectric and audible alerts. Furthermore, differentiated warning alerts can be provided for vehicles inside and outside based on the type of intruding object. For example, if the intruding object is a person, an external audible warning alert is used to remind the person to move away; if the intruding object is an animal, a specific scare frequency or audio frequency is used to scare it away; when the intruding object is an inanimate object such as a stone, the system specifically alerts occupants to take precautions. Additionally, if there are staff members nearby, a voice warning can be provided to prompt them to move the intruding object. The type of intruding object can also be identified and extracted by training an image-based safe zone recognition model.

[0212] 13 Identify aerial intrusion objects

[0213] When this automated track safety monitoring method identifies aerial object intrusions, it needs to be combined with depth cameras / radar for joint judgment because distance is not a sufficient criterion. That is, the depth value of the aerial object is obtained by using a depth camera or the distance value of the aerial object is obtained by using radar. When the depth value or distance value is too short, it is judged that there is an aerial object intrusion.

[0214] The automated track safety monitoring method of this invention uses the lower boundary of the foreign object target as the intrusion baseline. Two intersection points are obtained by intersecting this baseline with the left and right edges of the safety zone. By utilizing the horizontal coordinates of the lower left and lower right corners of the foreign object target and their positional relationship with these two intersection points, the presence of a foreign object intrusion can be accurately determined. This automated track safety monitoring method can directly use the horizontal coordinates of the monitored image for foreign object intrusion detection, without requiring calculations of the shape's area or other complex calculations. The calculation algorithm is simple and fast, effectively reducing the computational load and improving the calculation speed, making it particularly suitable for detecting foreign object intrusions from ground objects.

[0215] The following are embodiments of the electronic device provided by the present invention. The embodiments of the electronic device and the embodiments of the automated track safety monitoring method described above belong to the same concept. Details not described in detail in the embodiments of the electronic device can be found in the embodiments of the automated track safety monitoring method described above.

[0216] Please see Figure 12 As shown, it is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiment of the present invention.

[0217] In this embodiment, an electronic device includes:

[0218] Memory 1005 is used to store program instructions; and

[0219] Processor 1001 is used to execute the program instructions to implement the steps of the automated track safety monitoring method as described above.

[0220] The electronic device in this embodiment of the invention can be a desktop computer, laptop, handheld computer, server, or other computing device. Figure 12 As shown, the electronic device may include: a processor 1001 (e.g., CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit, such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0221] Those skilled in the art will understand that Figure 12 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0222] like Figure 12 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0223] exist Figure 12 In the electronic device shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user terminal) and communicate with the client; and the processor 1001 can be used to call the computer program stored in the memory 1005. When the computer program is called and executed by the processor 1001, it implements the steps of the above-mentioned automated track safety monitoring method.

[0224] The following are embodiments of the computer-readable storage medium provided by the present invention. The embodiments of the computer-readable storage medium belong to the same concept as the embodiments of the automated track safety monitoring method and electronic device described above. Details not described exhaustively in the embodiments of the computer-readable storage medium can be found in the embodiments of the automated track safety monitoring method and electronic device described above.

[0225] In this embodiment, a computer-readable storage medium stores computer-executable instructions thereon, which, when executed by a processor, implement the steps of the automated track safety monitoring method as described above.

[0226] The electronic device and computer-readable storage medium of the present invention are both capable of implementing the steps of the above-described automated track safety monitoring method, and therefore possess at least all the beneficial effects brought about by the technical solutions of the above-described automated track safety monitoring method step embodiments, which will not be elaborated here.

[0227] The above description is only a part or preferred embodiment of the present invention. Neither the text nor the drawings should limit the scope of protection of the present invention. All equivalent structural transformations made using the content of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

[0228] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0229] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0230] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0231] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0232] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An automated track safety monitoring method, characterized in that, include: In response to acquiring a monitoring image of the monitoring area captured by the camera, the monitoring image is input into a pre-trained image safety zone recognition model to extract the track position from the monitoring image and determine the safety zone range on both sides of the track based on the track position; wherein, the safety zone range includes a left side line and a right side line, and the positions of the left side line and the right side line on the monitoring image are determined by the horizontal coordinate values ​​of the image coordinate system constructed by the monitoring image; While inputting the monitoring image into the pre-trained image safe zone recognition model, if a foreign object target is extracted from the monitoring image, the horizontal coordinates of the lower left corner and the lower right corner of the foreign object target on the monitoring image are obtained; wherein, the horizontal coordinates of the lower left corner of the foreign object target are recorded as the lower left horizontal coordinates, and the horizontal coordinates of the lower right corner of the foreign object target are recorded as the lower right horizontal coordinates. The horizontal coordinates of the left edge of the safe zone intersect the corresponding horizontal line of the distance marker of the foreign object distance camera in the monitoring image, and the horizontal coordinates of the right edge of the safe zone intersect the corresponding horizontal line of the distance marker of the foreign object distance camera in the monitoring image are obtained; wherein, the horizontal coordinate value of the left edge of the safe zone is recorded as the left horizontal coordinate value, and the horizontal coordinate value of the right edge of the safe zone is recorded as the right horizontal coordinate value; If the horizontal coordinate value of the lower left corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then a foreign object intrusion is confirmed and a foreign object intrusion warning message is issued. If the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the left side and not greater than the horizontal coordinate value of the right side, then a foreign object intrusion is confirmed and a foreign object intrusion warning message is issued. If the horizontal coordinate value of the lower left corner is not greater than the horizontal coordinate value of the left side and the horizontal coordinate value of the lower right corner is not less than the horizontal coordinate value of the right side, then a large foreign object intrusion is confirmed and a warning message indicating the presence of a large foreign object intrusion is issued.

2. The automated track safety monitoring method according to claim 1, characterized in that, The foreign objects include ground-based foreign objects and airborne foreign objects; If the foreign object is a ground object, then the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image is the vertical coordinate value of the lower left or lower right corner of the monitoring image. If the foreign object is an airborne foreign object, the vertical coordinate value of the distance marker between the foreign object and the camera on the monitoring image corresponding to the horizontal line is obtained by mapping the actual distance between the foreign object and the camera obtained by the radar device or depth acquisition device onto the vertical coordinate value of the horizontal line in the monitoring image. Determining whether the foreign object is a ground-based or airborne object includes: Determine the pixel distance between the horizontal line mapped to the actual distance in the monitoring image and the horizontal line corresponding to the vertical coordinate value of the lower left or lower right corner of the monitoring image; If the pixel distance is less than or equal to a preset error threshold, it is confirmed as a foreign object on the ground; If the pixel distance is greater than a preset error threshold, it is confirmed as an airborne foreign object; If it is confirmed to be a foreign object on the ground, a ground foreign object intrusion is confirmed and a ground foreign object intrusion warning message is issued; If the object is confirmed to be an airborne foreign object, the horizontal line mapped to the actual distance in the monitoring image is obtained, and the vertical coordinate value of the upper edge of the space occupied by the carriage in the monitoring image is obtained based on the vertical coordinate value of the horizontal line. The space occupied by the carriage is obtained based on the aspect ratio of the carriage entity, and the space occupied by the carriage includes the upper edge. The position of the upper edge on the monitoring image is determined by the vertical coordinate value of the image coordinate system. The vertical coordinate value of the lower boundary of the foreign object is recorded as the lower vertical coordinate value, and the vertical coordinate value of the upper edge of the space occupied by the carriage is recorded as the upper vertical coordinate value. If the lower vertical coordinate value is not greater than the upper vertical coordinate value, then an airborne foreign object intrusion is confirmed and a warning message is issued.

3. The automated track safety monitoring method according to claim 2, characterized in that, The confirmation of an aerial foreign object intrusion includes: The radar equipment is activated to measure the distance between the radar equipment and the aerial foreign object entity corresponding to the foreign object target. If the distance value is not greater than a preset distance threshold, an aerial foreign object intrusion is confirmed; or The depth acquisition device is activated to obtain a depth image of the monitored area. Based on the depth image, the depth value between the depth acquisition device and the aerial foreign object entity corresponding to the foreign object target is obtained. If the depth value is not greater than a preset depth threshold, the presence of an aerial foreign object intrusion is confirmed.

4. The automated track safety monitoring method according to claim 1, characterized in that, The step of inputting the monitoring image into a pre-trained image safety zone recognition model to extract the track position from the monitoring image and determine the safety zone range on both sides of the track based on the track position includes: The monitoring image is preprocessed to obtain a corresponding mask image. The mask image is then input into a pre-trained image security area recognition model to extract the track image from the mask image and obtain the pixel values ​​of the left boundary and the right boundary of the track image on the mask image. The pixel value of the left boundary of the track image is denoted as the left pixel value, and the pixel value of the right boundary of the track image is denoted as the right pixel value. The track pixel width of the track image is calculated based on the left pixel value and the right pixel value, and the proportional relationship coefficient between the track pixel width and the track physical width of the track entity is obtained. Calculate the scaling factor and the car pixel width; wherein the scaling factor is equal to the ratio of the track physical width to the car physical width of the car entity, and the car pixel width is equal to the product of the ratio and the track pixel width; Based on the pixel width of the carriage and the physical width of the carriage, obtain a scale for the pixel width of the carriage and the physical width of the carriage; The pixel width is widened according to a preset algorithm based on the track pixel width; Determine the position of the safe zone range on both sides of the track on the mask image; wherein, the horizontal coordinate value of the left edge of the safe zone range is equal to the left pixel value minus the widened pixel width, and the horizontal coordinate value of the right edge of the safe zone range is equal to the right pixel value plus the widened pixel width; The physical size range of the security zone is determined based on its position on the mask image, the scaling factor, and the scale.

5. The automated track safety monitoring method according to claim 4, characterized in that, The preset algorithm includes: the widened pixel width is equal to two-thirds of the track pixel width, or the widened pixel width is equal to half the difference between the carriage pixel width and the track pixel width.

6. The automated track safety monitoring method according to claim 4, characterized in that, The step of calculating the track pixel width of the track image based on the left pixel value and the right pixel value includes: In the longitudinally extended track image, a horizontal line segment is selected, and the track pixel width of the track image is calculated based on the pixel values ​​of the left endpoint and the right endpoint of the horizontal line segment on the mask image. The track pixel width is collected once at a preset interval to determine whether the track pixel width continues to increase within a preset time period; If so, the track pixel width of the track image is calculated based on the bottom horizontal line segment near the camera in the track image; If not, then continue to calculate the track pixel width of the track image based on the selected horizontal line segment.

7. The automated track safety monitoring method according to claim 1, characterized in that, The step of extracting the track position from the monitoring image and determining the safety zone range on both sides of the track based on the track position includes: Extract the orbit image from the monitoring image and obtain the orbit position of the orbit image in the monitoring image; At least one key point is set in the track image according to a preset rule, and the vertical coordinate value of the key point in the monitoring image is obtained; Obtain the physical vertical height of the key point corresponding to the track entity, and confirm whether the physical vertical height and vertical coordinate value of the key point match according to the size ratio; If so, the safety zone range on both sides of the track is determined based on the track position; If not, the vertical coordinate value of the key point is modified to match the physical vertical height; while the vertical coordinate value of the key point is modified, the track position is automatically modified synchronously, and the safety zone range on both sides of the track is determined based on the synchronously modified track position. The modification of the vertical coordinate value of the key point includes: modifying the vertical coordinate value of the key point by moving the key point in the monitoring image, or modifying the vertical coordinate value of the key point by inputting a vertical coordinate value that matches the physical vertical height of the key point.

8. The automated track safety monitoring method according to claim 1, characterized in that: The safe zone is displayed in the monitoring image in the form of a first rectangle; wherein, the left side of the first rectangle represents the left edge of the safe zone, and the right side of the first rectangle represents the right edge of the safe zone. The safe zone includes three ranges, and the distance between the actual safe area represented by the three ranges and the camera increases sequentially. The color and character identifier of the first rectangle of each range are different from the color and character identifier of the first rectangle of the other ranges. The foreign object target is displayed in the monitoring image in the form of a second rectangle; wherein, the second rectangle surrounds the foreign object target to the greatest extent, the lower left corner of the second rectangle represents the lower left corner of the foreign object target, and the lower right corner of the second rectangle represents the lower right corner of the foreign object target.

9. An electronic device, characterized in that, include: Memory, used to store program instructions; as well as A processor for executing the program instructions to implement the steps of the automated track safety monitoring method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the automated track safety monitoring method as described in any one of claims 1 to 8.

Images