Railway construction area intelligent conflict early warning method based on multi-modal information fusion

Abstract

The invention discloses a railway construction area intelligent conflict early warning method based on multi-modal information fusion, and belongs to the technical field of railway construction monitoring, and the method comprises the steps: collecting a railway construction scene image, and carrying out the recognition of a construction area and the determination of a construction boundary; positioning the position of the locomotive by combining Beidou positioning and a visual semantic matching method; and carrying out locomotive conflict detection and risk assessment according to the locomotive position and the construction area and the construction boundary of the railway. By dynamically and intelligently identifying the construction operation area and deeply fusing the locomotive positioning information, the active and dynamic collaborative early warning of the railway construction operation safety risk is realized. Meanwhile, when it is detected that the distance between the locomotive and the construction operation area is a safe distance, early warning is conducted on the locomotive and the construction operation area at the same time, and the safety of constructors and the locomotive in the complex and changeable railway construction environment can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of railway construction monitoring, and particularly relates to an intelligent conflict early warning method for a railway construction area based on multi-modal information fusion. BACKGROUND

[0002] In recent years, with the continuous growth of railway mileage and the continuous improvement of transportation density, maintenance, expansion and reconstruction operations along the line are becoming more and more frequent, and the conflict risk between the construction area and train operation has become a core hidden danger restricting train safety.

[0003] In actual operation, the safety protection of the railway construction operation area has long relied on manual warning, fixed signs and traditional signal systems, and there are problems such as response lag, insufficient positioning accuracy, and weak risk prediction ability. This mode has significant lag and uncertainty, especially in the case of temporary expansion of the construction area, variable weather and special conditions, the information transmission is not timely or the driver's observation is blocked, which can easily cause serious accidents such as mistaken entry and collision, causing casualties and line interruption. At the same time, a fundamental deficiency is that an integrated intelligent early warning system that closely integrates real-time dynamic positioning of locomotives / trains with intelligent identification and dynamic definition of construction operation areas has not been established. The existing scheme focuses on risk monitoring and early warning in known areas, and cannot effectively solve the problem of active and cooperative safety protection in the case where the position and range of the construction area may change or need to be identified in real time, limiting the universality and intelligence level of the early warning system.

[0004] Technical scheme of prior art one; The railway construction site safety protection system mainly generates a virtual electronic fence covering the uplink and downlink lines of the railway (each 4 meters wide) and divided into multiple intervals based on Beidou positioning and GIS system through an electronic fence module, supports dynamic boundary adjustment; a human-computer interaction module is used to set the "open / closed" state of the electronic fence, associate the construction plan containing the geographic range and time period, and automatically set the corresponding area fence to an open state during the construction period, while visualizing the fence range, state and construction unit position; a site positioning module obtains the real-time geographic position of construction personnel and equipment with the help of Beidou positioning, filters abnormal data with a correction algorithm, and improves the positioning accuracy to sub-meter to centimeter level through nearest matching reference station; a safety risk judgment module compares the construction unit position and the electronic fence range in real time, and determines that there is a safety risk and triggers a prompt when the construction unit is located in the closed fence or the distance from the closed fence boundary is less than a threshold value; it can also include a site alarm module that alarms to the construction personnel through sound and light or vibration after receiving the risk signal, and has the characteristics of dynamic fence control, accurate positioning and early warning, visualization and linkage, etc.

[0005] The disadvantages of prior art one are: (1) Only for the safety protection of construction units, without conflict detection between locomotives and construction areas, unable to issue early warning to locomotive drivers; (2) The electronic fence range is fixed, which is difficult to adapt to the dynamic adjustment demand of the construction area; (3) The risk judgment depends on static geographic range, without combining dynamic factors such as locomotive speed and construction type, and the early warning precision is insufficient; (4) Without visual recognition ability, unable to automatically identify the boundary and key target of the construction area, and the electronic fence parameters need to be manually maintained.

[0006] The technical solution of the prior art two; A railway worker safety warning device realizes closed-loop protection through a triple linkage system. The construction site warning device includes a construction area main device (integrated control module, sound and light alarm and display screen) and a boundary auxiliary device (located 2km in the uplink and downlink directions, including a camera and an alarm), which monitors the train dynamics in real time and triggers the warning. The mechanical equipment warning device (vehicle-mounted terminal) monitors the mechanical state and alarms locally when an abnormality is found. The worker warning device (handheld terminal) receives the warning signal and reminds the personnel through sound and vibration combination. The three are interconnected through a wireless module: the auxiliary device captures the train information → the main device integrates the data and triggers the sound and light alarm → synchronously pushes to the handheld / vehicle-mounted terminal → the station monitoring center remotely schedules, forming a "monitoring-alarm-protection" full-link real-time protection system.

[0007] Disadvantages of the prior art two (1) Mainly configured manually, the monitoring range of the construction area and the alarm logic need to be manually set.

[0008] (2) The video is only used for manual viewing, although there is a camera module, but it does not integrate intelligent recognition algorithm, and cannot automatically determine the construction boundary and coordinates.

[0009] (3) The locomotive positioning and construction detection are not integrated, and the dynamic conflict detection and bidirectional early warning between the train and the construction area cannot be realized.

[0010] (4) The boundary deployment is difficult, the auxiliary device needs to be erected at 2km in the uplink / downlink direction of the construction area boundary, but the terrain along the railway is complex (bridge, rural area), the implementation cost of power supply, network and fixed support frame is high, and it is easy to be stolen or damaged. Summary of the invention

[0011] The purpose of the present application is to solve or improve the above-mentioned problems by providing a railway construction area intelligent conflict early warning method based on multi-modal information fusion.

[0012] To achieve the above-mentioned purpose, the technical solution adopted by the present application is: A railway construction zone intelligent conflict early warning method based on multi-modal information fusion, comprising the following steps: S1, collecting railway construction scene images, and performing construction area identification and construction boundary determination; S2, combining Beidou positioning and visual semantic matching method to position the locomotive position; S3, according to the locomotive position and the construction area and construction boundary of the railway, performing locomotive conflict detection and risk assessment.

[0013] Further, the S1 comprises the following steps: S11, collecting railway construction scene images and identifying the construction area thereof; S12, converting the identified construction area into world coordinates; S13, determining the construction area boundary by using an alpha-shape algorithm; S14, determining the construction zone occupied track based on the construction area boundary determined in S13.

[0014] Further, in S12, converting the identified construction area into world coordinates comprises:

[0015] In the formula, is a scale factor; , is the coordinate in the image pixel coordinate system; , , is the coordinate in the world coordinate system; K is the camera calibration matrix; [R∣t] is the extrinsic matrix.

[0016] Further, S14 specifically comprises: intersecting the construction area boundary polygon with the track center line data, and outputting the construction occupied track number set and the occupied interval.

[0017] Further, the S2 comprises the following steps: S21, when the accuracy parameter , wherein, is a preset accuracy threshold, and the Beidou positioning is used to obtain the locomotive position; if , then S22 is executed; S22, using a visual semantic matching method to calculate the optimal pose of the locomotive; S23, based on the locomotive position by Beidou positioning and the optimal pose of the locomotive, obtaining the final positioning position of the locomotive.

[0018] Further, in S22, using a visual semantic matching method to calculate the optimal pose of the locomotive comprises: The front trackside visual mark image is collected by using the on-board camera on the locomotive, the extracted semantic features of turnout geometry, sleeper texture and signal position are matched with the local track image template by using the weighted feature similarity, and the optimal pose of the locomotive is obtained, wherein the weighted feature similarity is expressed as:

[0019] In the formula, is the overall similarity score; is a feature similarity function, is a weight coefficient; is the kth semantic feature extracted from the current locomotive camera image; is the kth semantic feature extracted from the template image.

[0020] Further, in the S23, the final positioning position of the locomotive is obtained based on the Beidou positioning of the locomotive and the optimal pose of the locomotive, comprising:

[0021] In the formula,

[0022] In the formula, is the final positioning position of the locomotive; is the Beidou positioning of the locomotive; is the optimal pose of the locomotive; is the adaptive fusion weight based on the Beidou accuracy.

[0023] Further, the S3 comprises the following steps: S31, judging whether there is an intersection between the track number of the locomotive and the set of construction occupied track numbers, if there is an intersection, then entering S32; S32, calculating the minimum Euclidean distance between the locomotive and the construction area boundary:

[0024] In the formula, is the minimum Euclidean distance between the locomotive and the construction area boundary; G is the set of geographic coordinates of the construction area boundary points; S33, calculating the time to collision TTC based on the minimum Euclidean distance and the relative speed of the locomotive, and triggering the warning based on the time to collision TTC.

[0025] Further, in the S33, the time to collision TTC is calculated, comprising:

[0026] In the formula, is the relative speed of the locomotive; When Or If the distance threshold is exceeded, a locomotive warning is triggered; The time threshold is a time threshold; The distance threshold is a distance threshold.

[0027] Further, in the S33, based on the minimum Euclidean distance between the locomotive and the construction area boundary And the hierarchical warning strategy of the collision time TTC synchronously drives the alarm of the locomotive end and the construction end, which is specifically:

[0028] In the formula, The warning level is a warning level.

[0029] The railway construction area intelligent conflict warning method based on multi-modal information fusion provided by the application has the following beneficial effects: (1) Solve the problem of dynamic identification of the construction area: through multi-modal perception and computer vision technology, automatically adapt to the change of the construction area boundary, eliminate the dependence on the preset fixed electronic fence, and overcome the blind area caused by the temporary expansion or position change of the construction area in the prior art.

[0030] (2) Solve the positioning reliability problem: the fusion mechanism of Beidou and visual semantic matching provides positioning redundancy in signal shielding scenes such as tunnels and mountainous areas, ensures continuous and reliable output of the locomotive position, and overcomes the risk of pure Beidou positioning failure; realize the positioning ability of track level accuracy, and provide accurate space basis for conflict detection.

[0031] (3) Solve the problem of conflict detection accuracy: combine the real-time position, speed and dynamic boundary of the construction area, establish a double-parameter risk assessment model based on the minimum distance and collision time (TTC), which is more scientific than a single static threshold to reflect the actual risk level. Accurately determine the spatial conflict relationship between the construction area and the track where the locomotive is located through GIS mapping, and avoid misjudgment of the risk of adjacent track construction.

[0032] (4) Solve the problem of warning coordination: synchronously trigger the two-way warning of the locomotive end and the construction end, eliminate the limitation of only single-end protection in the prior art, and significantly improve the response efficiency of risk avoidance. The hierarchical warning mechanism dynamically adjusts the warning intensity according to the risk, optimizes the effectiveness of human-computer interaction, and avoids alert fatigue caused by excessive alarm.

[0033] (5) Accurate fitting of construction boundary: generate a non-convex polygon boundary through an alpha-shape algorithm, avoid the problem of virtual expansion of the construction range caused by the traditional convex hull algorithm, and reduce the risk of false alarm (especially suitable for complex construction scenes of bypass track facilities). BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1A flow chart of the intelligent conflict early warning method for a railway construction zone based on multi-modal information fusion in Embodiment 1 of the present application.

[0035] Figure 2 A convex hull algorithm is used to identify the construction zone boundary in Embodiment 1 of the present application.

[0036] Figure 3 An alpha-shape algorithm is used to identify the construction zone boundary in Embodiment 1 of the present application.

[0037] Figure 4 A flow chart of the intelligent conflict early warning system for a railway construction zone based on multi-modal information fusion in Embodiment 2 of the present application. DETAILED DESCRIPTION

[0038] The specific embodiments of the present application are described below to facilitate understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited in scope to the specific embodiments, and that all applications utilizing the concept of the present application are within the scope of the present application as defined and determined by the appended claims, as long as various changes are obvious to those skilled in the art within the spirit and scope of the present application.

[0039] Embodiment 1 The intelligent conflict early warning method for a railway construction zone based on multi-modal information fusion of the present embodiment overcomes the blind area of protection caused by temporary expansion or position change of the construction zone in the prior art, achieves positioning capability at track level, provides accurate spatial basis for conflict detection, accurately determines the spatial conflict relationship between the construction zone and the track where the locomotive is located through GIS mapping, avoids misjudgment of the risk of adjacent track construction, at the same time, the hierarchical warning mechanism dynamically adjusts the warning intensity according to the risk, optimizes the effectiveness of human-computer interaction, avoids alert fatigue caused by excessive alarm, and the like. Figure 1 In particular, the following contents are included: S1, collecting railway construction scene images and identifying the construction area and determining the construction boundary, which specifically includes the following contents: S11, collecting railway construction scene images and identifying the construction area; In some embodiments, railway construction scene images are collected in real time by using railway line video monitoring equipment, and a deep learning target detection network is used to identify key targets such as construction signboards, engineering machinery, construction personnel, and temporary isolation facilities. The identification process is prior art, and therefore the detailed process will not be described again.

[0040] S12, converting the identified construction area into world coordinates; The identified construction area is output in the form of a pixel contour, and based on the camera calibration matrix K and the external parameter matrix [R∣t], the pixel coordinates are converted into world coordinates, which include:

[0041] where, is a scale factor. It is a non-zero constant to convert homogeneous coordinates back to three-dimensional coordinates; , is the coordinate in the image pixel coordinate system. It is a two-dimensional coordinate, representing the specific position of the target point in the image (u-th column, v-th row); , , is the coordinate in the world coordinate system. It is a three-dimensional coordinate, representing the specific position of the target point in the actual geographical space; K is the camera calibration matrix; [R|t] is the extrinsic matrix, camera intrinsic / extrinsic: intrinsic matrix K (focal length, principal point, radial distortion, etc.), extrinsic R, t (camera pose and position in the world coordinate system).

[0042] Mapping the construction area boundary points to a set of geographic coordinates by combining GIS maps and high-precision track center line data of railway lines G :

[0043] S13, determining the construction area boundary by using the α-Shape algorithm; In some embodiments, the railway construction area often presents a non-convex polygon (such as bypassing a signal machine, a turnout, or a temporary avoidance device), and the traditional convex hull algorithm will be excessively expanded to a non-construction area, as shown in FIG. 2B. The present application preferentially uses the α-Shape algorithm (the α value is adaptively adjusted according to the distance between track facilities), to generate a compact boundary that can contain concave structures, and is more consistent with the actual construction range; in the absence of complex shielding scenarios, the convex hull algorithm can be used in a degraded manner to reduce the computational overhead, as shown in FIG. 2C. Figure 2 Figure 3

[0044] S14, determining the occupied track of the construction area based on the construction area boundary determined in S13; Intersecting the construction area boundary polygon with the track center line data to output a set of construction-occupied track numbers and occupied intervals (such as “No. 3 track, K201+300 to K201+700”).

[0045] S2, introducing a fusion mechanism of Beidou high-precision positioning and visual semantic matching in locomotive positioning, to ensure that a stable and reliable locomotive position can still be obtained in a complex road-following environment (such as a tunnel, a bridge, a curve, and a mountainous area), which includes the following steps: S21, Beidou positioning; The locomotive end is equipped with a Beidou high-precision positioning module, which obtains the locomotive position and the precision parameter , when the precision parameter​​ At that time, among them The system uses a preset accuracy threshold, a critical value to determine whether the BeiDou positioning result is usable, to directly obtain the locomotive's position using BeiDou positioning; if... If so, then execute S22; S22, Visual semantic matching; In one specific embodiment, an onboard camera on the locomotive captures images of visual markers along the track ahead. Weighted feature similarity is then used to extract semantic features of the turnout geometry, sleeper texture, and signal positions. and with local orbital image templates Matching is performed to obtain the optimal pose of the locomotive. The weighted feature similarity is expressed as:

[0046] In the formula, This is the overall similarity score. This value is the sum of weighted similarities for all features and measures the degree of matching between the current image and the template image. A higher value indicates a higher degree of matching and more reliable visual localization results. For feature similarity function, These are the weighting coefficients; This refers to the k-th semantic feature extracted from the current locomotive camera image; Let be the k-th semantic feature extracted from the template image.

[0047] S23. Based on the locomotive's position and optimal pose determined by BeiDou positioning, the final positioning of the locomotive is obtained, including:

[0048] in:

[0049] In the formula, This refers to the final location of the locomotive; The locomotive's location as determined by BeiDou positioning; To achieve the optimal locomotive position; To achieve a dynamic balance between positioning accuracy and robustness, adaptive fusion weights based on BeiDou accuracy are adaptively adjusted according to visual matching confidence. This results in a precise positioning module that primarily uses BeiDou and secondarily uses visual technology, supporting positioning output at the track number level.

[0050] S3. Based on the locomotive location and the railway construction area and boundary, conduct locomotive conflict detection and risk assessment, which specifically includes the following steps: S31. Determine the locomotive track number Set of track numbers occupied by construction If there is intersection, go to S32; Where the intersection of two numbers is expressed as:

[0051] S32, the minimum Euclidean distance between the computer car and the construction area boundary; In some embodiments, the conflict detection is based on the locomotive position With the construction area boundary G as input, the minimum distance of the computer car to the polygon boundary is calculated:

[0052] In the formula, D_min is the minimum Euclidean distance between the locomotive and the construction area boundary; G is the set of geographic coordinates of the construction area boundary points.

[0053] S33, according to the minimum Euclidean distance and the relative speed of the locomotive, the time to collision TTC is calculated, and the pre-warning is triggered based on the time to collision TTC; Wherein, the time to collision TTC is calculated, including:

[0054] In the formula, V is the relative speed of the locomotive; When Or The locomotive pre-warning is triggered; T is the time threshold, which is a preset safety time threshold. If the calculated time to collision TTC is less than this threshold, it means that the remaining reaction time is insufficient, and the system should immediately trigger the pre-warning; D is the distance threshold, which is a preset safety distance threshold. If the minimum distance D_min between the locomotive and the construction area is less than this threshold, it means that the physical space is too close, and the pre-warning should be triggered immediately regardless of the time.

[0055] In S33, the minimum Euclidean distance between the locomotive and the construction area boundary And the hierarchical pre-warning strategy of the time to collision TTC synchronously drives the alarm of the locomotive end and the construction end, which is specifically:

[0056] In the formula, W is the pre-warning level.

[0057] The pre-warning level and the trigger condition are shown in the following table:

[0058] The alarm signal is synchronized in real time to the locomotive end, the construction end and the dispatching center through pre-warning and communication, ensuring the whole chain response. The threshold support of TTC is dynamically adjusted according to the line speed limit and the construction type. When the communication is interrupted, the locomotive end and the construction end can trigger the alarm based on the local calculation and TTC.

[0059] Embodiment 2 The intelligent conflict early warning system for railway construction area based on multi-modal information fusion in this embodiment refers to Figure 4 which comprises: a video monitoring subsystem for real-time acquisition of railway construction scene images; a construction area identification subsystem for acquisition of railway construction scene images and identification of construction area and determination of construction boundary; a locomotive positioning subsystem for fusion of Beidou positioning and visual semantic matching method to position the locomotive; a conflict detection subsystem for locomotive conflict detection according to the locomotive position and the construction area and construction boundary of the railway; a warning and communication subsystem for synchronous driving of alarm of the locomotive end and the construction end based on the minimum Euclidean distance and the hierarchical warning strategy of time to collision TTC.

[0060] Although the specific embodiments of the invention are described in detail with reference to the accompanying drawings, it should not be understood as limiting the protection scope of the patent. Various modifications and variations made by those skilled in the art within the scope described in the claims are still within the protection scope of the patent.

Claims

1. A method for intelligent conflict early warning in railway construction areas based on multimodal information fusion, characterized in that, Includes the following steps: S1. Collect images of railway construction scenes and identify construction areas and determine construction boundaries; S2. Locate the locomotive position by combining BeiDou positioning and visual semantic matching methods; S3. Conduct locomotive conflict detection and risk assessment based on locomotive location and the railway construction area and boundary.

2. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 1, characterized in that, S1 includes the following steps: S11. Collect images of railway construction scenes and identify their construction areas; S12. Convert the identified construction area into world coordinates; S13. Use the α-Shape algorithm to determine the boundary of the construction area; S14. Based on the construction area boundary determined in S13, determine the tracks occupied by the construction area.

3. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 2, characterized in that, In step S12, converting the identified construction area into world coordinates includes: In the formula, Scale factor; , These are the coordinates in the image pixel coordinate system; , , The coordinates are in the world coordinate system; K is the camera calibration matrix; [R|t] is the extrinsic parameter matrix.

4. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 2, characterized in that, S14 specifically includes: finding the intersection of the construction area boundary polygon with the track centerline data, and outputting the set of track numbers occupied by construction and the occupied interval.

5. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 1, characterized in that, S2 includes the following steps: S21, When the accuracy parameter At that time, among them, The locomotive position is obtained using BeiDou positioning, based on a preset accuracy threshold; if If so, then execute S22; S22. Using a visual semantic matching method, the optimal pose of the computer vehicle is determined. S23. Based on the locomotive position and optimal locomotive pose determined by BeiDou positioning, the final positioning position of the locomotive is obtained.

6. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 5, characterized in that, In step S22, a visual semantic matching method is used to determine the optimal pose of the computer vehicle, including: Using an onboard camera on the locomotive to capture visual marker images along the track ahead, a weighted feature similarity method is employed to match the extracted semantic features of the turnout geometry, sleeper texture, and signal position with a local track image template, thereby obtaining the optimal locomotive pose. The weighted feature similarity is expressed as: In the formula, The overall similarity score; For feature similarity function, These are the weighting coefficients; This refers to the k-th semantic feature extracted from the current locomotive camera image; Let be the k-th semantic feature extracted from the template image.

7. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 6, characterized in that, In step S23, based on the locomotive's position and optimal pose determined by BeiDou positioning, the final positioning location of the locomotive is obtained, including: in: In the formula, This refers to the final location of the locomotive; The locomotive's location as determined by BeiDou positioning; To achieve the optimal locomotive position; This is an adaptive fusion weight based on BeiDou accuracy.

8. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 7, wherein step S3 includes the following sub-steps: S31. Determine whether there is an intersection between the locomotive track number and the set of track numbers occupied by construction. If there is an intersection, proceed to S32. S32. Minimum Euclidean distance between the computer vehicle and the boundary of the construction area: In the formula, G represents the minimum Euclidean distance between the locomotive and the boundary of the construction area; G is the set of geographic coordinates of the boundary points of the construction area. S33. Calculate the collision time TTC based on the minimum Euclidean distance and the relative speed of the locomotive, and trigger an early warning based on the collision time TTC.

9. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 8, wherein in S33, calculating the collision time TTC includes: In the formula, The relative speed between the locomotive and the train; when or When this happens, a locomotive warning is triggered; This is a time threshold; This is the distance threshold.

10. The intelligent conflict early warning method for railway construction areas based on multimodal information fusion according to claim 9, wherein in step S33, the minimum Euclidean distance between the locomotive and the boundary of the construction area is used. The time-of-collision (TTC) collision warning strategy synchronously drives alarms on both the locomotive and construction ends, specifically as follows: In the formula, It is at the warning level.

Images