Railway gauge intrusion detection method based on multi-sensor fusion

Abstract

The invention discloses a railway gauge intrusion detection method based on multi-sensor fusion, and the method comprises the steps: reducing the number of to-be-processed point clouds through a preset point cloud preprocessing algorithm, accelerating the algorithm operation speed, carrying out the down-sampling of point cloud data through employing voxel grids, only retaining the average position point cloud in each grid, further reducing the number of point clouds, and carrying out the down-sampling of the point cloud data through employing the voxel grids. Through the preset rail surface fitting method, the invasion and limit foreign matter is segmented from the rail surface background, and laser radar three-dimensional data detection and video detection are fused, so that the missing report resistance and the false report resistance are improved, and the result is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of railway foreign body intrusion detection technology, and in particular to a railway limit intrusion detection method based on multi-sensor fusion. Background Art

[0002] With economic development, railway construction has entered a period of rapid development. As the mileage of railway construction increases, the safety along the railway has received more and more attention. In recent years, the potential safety hazards along the railway have increased, especially safety accidents caused by natural disasters such as mudslides and falling rocks. Railway construction is prone to natural disasters when crossing mountainous areas, plateaus and hilly areas. Once these disasters occur, they may cause huge harm and losses to the railway department.

[0003] Chinese patent document CN111564015A discloses a "rail transit perimeter intrusion monitoring method and device". The steps of the method include: A, obtaining a video sequence image; B, extracting one frame of the video sequence image, extracting the background image from the frame image, and extracting the pixel coordinate matrix of the edge line of the rail; C, dividing the background image into multiple grids, and determining the center coordinates of each grid; D, dividing the multiple grids into sensitive areas and non-sensitive areas according to the Euclidean distance between the center coordinates of the grid and the edge line of the rail; E, using the target detection algorithm to determine whether there is a moving object in each grid, and if there is a moving object, execute step F; F, based on the rail transit perimeter intrusion recognition model, make a judgment, and when the moving object is not a train and is in a sensitive area, an alarm is issued. The above technical solution lacks initial detection, the detection form is single, and there is a risk of missed reports and false alarms. Summary of the invention

[0004] The present invention mainly solves the technical problems that the original technical solution lacks initial detection, has a single detection form, and has the risk of missed reports and false alarms. It provides a railway limit intrusion detection method based on multi-sensor fusion, reduces the number of point clouds to be processed through a preset point cloud preprocessing algorithm, accelerates the algorithm operation speed, and then uses a voxel grid to downsample the point cloud data, retaining only the average position point cloud in each grid, further reducing the number of point clouds, and segmenting the intruding objects from the rail surface background through a preset rail surface fitting method, and integrates the laser radar three-dimensional data detection with the video detection, thereby improving the ability to resist missed reports and false alarms and making the results more accurate.

[0005] The above technical problem of the present invention is mainly solved by the following technical solution: The present invention comprises the following steps: S1 constructs a railway clearance intrusion detection system; S2 monitors foreign objects invading the railway perimeter under 3D point cloud; S3 reduces the number of point clouds to be processed through a preset point cloud preprocessing algorithm; S4 uses a preset clustering algorithm to obtain the location of intruding foreign objects and the three-dimensional box and issue a radar alarm; S5 combines radar alarm information and video alarm information and outputs alarm information.

[0006] Preferably, the railway clearance intrusion detection system in step S1 comprises a laser radar monitoring unit, a 3D2D mapping unit and a radar-visual fusion unit connected in sequence, and the radar-visual fusion unit is also connected to a video detection unit and an alarm unit.

[0007] Preferably, the step S2 specifically includes: after the laser wave emitted by the laser radar is scanned onto the object, the three-dimensional spatial position coordinates and reflection intensity of the object are calculated based on the received reflected wave.

[0008] Preferably, the step S3 includes filtering of noise points and isolated points and cutting of redundant point clouds outside the defense zone, thereby reducing the number of point clouds to be processed and accelerating the algorithm operation speed.

[0009] Preferably, the step S3 presets a rail surface fitting method to segment the intruding foreign object and the rail surface to obtain a point cloud of the intruding foreign object, specifically comprising: S3.1 Obtain the normal vector from the point cloud and obtain the vector field of the unknown object M S3.2 uses voxel grids to downsample the point cloud data, retaining only the average position point cloud in each grid; S3.3 uses the fitted rail surface as the boundary between the rail surface and the intruding foreign matter, and separates the point cloud of the fitted rail surface and below. The remaining point cloud is the target point cloud containing the intruding foreign matter.

[0010] Preferably, the step S3.1 specifically includes, assuming that all points are located on or near the surface of the unknown object M, there exists an indicator function χM:R 3 →R: Where p = p(x, y, z) is a point in space that satisfies the gradient of the indicator function Approximate vector field Right now: Two-side divergence operator Transformed into Poisson's equation: The Poisson equation is solved by the finite element method, and the implicit function expression χ is obtained.

[0011] Preferably, the step S3.2 specifically includes extracting isosurfaces from voxels using the MC algorithm based on the voxel corner value χ(p) and the isovalue γ of the unknown object M, and outputting a triangular mesh.

[0012] Preferably, the step S4 obtains the position and size of the intruding foreign body through a preset clustering algorithm, and finally outputs the position of the intruding foreign body and a three-dimensional box, specifically including: S4.1 Select a certain intrusion point as the target point, and randomly select another point to calculate the distance between the two points; S4.2 If the distance between the two points is within the preset distance threshold, the random point is marked as the target point; if the distance between the two points exceeds the distance threshold, another point is selected to calculate the distance between the two points; S4.3 When the utilization rate of the intrusive foreign object points reaches the preset utilization rate, select other unmarked intrusive foreign object points and repeat the above steps. After repeated multiple times, the size of the intrusive foreign object is reached and the classification of different intrusive foreign objects is completed.

[0013] Preferably, the step S5 specifically includes obtaining the three-dimensional point cloud detection information and the whole-image target detection information of the video frame at the same time, respectively, first determining whether the three-dimensional laser radar alarm information has an intrusion box that is mistakenly detected, and if an intrusion box is detected, entering a false alarm elimination mode; if no intrusion box is detected, entering a false alarm prevention mode.

[0014] Preferably, in the false alarm elimination mode, the 2D coordinates of the intruding foreign object and the size of the alarm box are first obtained, and the intruding foreign object box is mapped to the video frame image; the ROI region of interest is intercepted with the mapped intruding foreign object box as the center and the multiples of the length and width of the mapped intruding foreign object box, and the intercepted ROI region of interest is input into the video detection unit; if the detection information output by the video detection unit does not contain intruding foreign object information, no alarm is required; if there is an intruding foreign object box, the intersection and union ratio IOU between the video intruding foreign object box frame and the radar mapping intruding foreign object box frame is calculated: Among them, inersectionArea represents the intersection of the video intrusion limit foreign object box frame and the radar mapping intrusion limit foreign object box frame, and UnionArea represents the union of the video intrusion limit foreign object box frame and the radar mapping intrusion limit foreign object box frame. If the IOU is greater than the set intersection-union ratio threshold iou_threshold, an alarm message is issued, otherwise no alarm is issued.

[0015] The beneficial effects of the present invention are as follows: the number of point clouds to be processed is reduced through a preset point cloud preprocessing algorithm, the algorithm operation speed is accelerated, and then the point cloud data is downsampled using a voxel grid, only the average position point cloud in each grid is retained, and the number of point clouds is further reduced. Through a preset track surface fitting method, the intruding object is segmented from the rail surface background, and the lidar three-dimensional data detection is integrated with the video detection, thereby improving the ability to resist missed alarms and false alarms and making the result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a principle connection structure diagram of the present invention.

[0017] Figure 2 It is a flow chart of radar-vision fusion of the present invention.

[0018] Figure 3 It is a schematic diagram of redundant point cloud cutting outside the defense zone of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution of the present invention is further described in detail below through embodiments and in combination with the accompanying drawings. It should be understood that the specific implementation method described here is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] The safety detection of trains has always been an important research in related fields. When a train collides with an object that invades the rails (such as pedestrians, animals, vehicles, etc.), it will cause huge property losses and negative impacts to individuals and society. At present, some video surveillance equipment will be installed at the turnout intersections, and alarms will be issued when abnormalities occur around the railway. The methods involved in these existing monitoring devices are generally traditional image recognition methods or target detection algorithms combined with deep learning. The accuracy of traditional image recognition methods in detecting anomalies is not very good, and a missed detection may cause significant losses. If the target detection method involving deep learning is to achieve a real-time detection speed that can be applied, the requirements for equipment resources are relatively high.

[0021] Two-dimensional laser radar has been introduced in the field of rail transit safety protection and has achieved certain results. However, due to the problems of false alarms and missed alarms in the actual detection application of two-dimensional laser radar, it has been gradually replaced by three-dimensional laser radar. In some foreign body intrusion monitoring systems, some have introduced cameras to detect intrusion behaviors in a purely visual way. However, both laser radar and cameras have the following defects: Detection that relies solely on lidar is prone to false alarms (interference from rain, fog, dust, etc.) and missed alarms (the target is obscured).

[0022] Detection that relies solely on cameras is prone to problems such as missed detections at night.

[0023] Therefore, in order to solve the problem of false alarms and missed alarms caused by a single sensor in rail transit intrusion detection, a new and effective track perimeter intrusion detection method is needed to solve the above problems.

[0024] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the drawings; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0026] Embodiment: A railway clearance intrusion detection method based on multi-sensor fusion in this embodiment, such as Figure 1 As shown, the following steps are included: S1 constructs a railway clearance intrusion detection system. The railway clearance intrusion detection system includes a laser radar monitoring unit, a 3D2D mapping unit and a laser vision fusion unit connected in sequence, and the laser vision fusion unit is also connected to a video detection unit and an alarm unit.

[0027] The laser radar monitoring unit is mainly responsible for monitoring the intrusive foreign objects on the railway perimeter under the 3D point cloud, and outputting the location and size of the intrusive objects when they are found. Specifically, the laser wave emitted by the laser radar is scanned to the object, and then the three-dimensional spatial position coordinates (x, y, z) and reflection intensity and other properties of the object are calculated based on the received reflected wave. Through the preset point cloud preprocessing algorithm: noise and isolated point filtering, redundant point cloud cutting outside the defense zone, the number of point clouds to be processed is reduced and the algorithm operation speed is accelerated. Then the voxel grid is used to downsample the point cloud data, and only the average position point cloud in each grid is retained to further reduce the number of point clouds. Through the preset track surface fitting method, the intrusive object and the rail surface background are segmented to segment the intrusive foreign objects.

[0028] The 3D2D mapping unit is mainly responsible for mapping the three-dimensional position and size of the target detected by the radar to the 2D image of the camera, and the lidar and camera realize the spatial coordinate system 1. The specific implementation first uses Zhang Youding's camera intrinsic calibration method to calculate the camera intrinsic parameter matrix. Then, the point cloud feature detection method and image features are used to establish 3D-2D feature point pairs, solve the PnP problem, estimate the external parameter matrix of the lidar and camera, and realize the three-dimensional and two-dimensional coordinate system 1.

[0029] The video detection unit is mainly responsible for detecting targets in the video frames input by the camera. Specifically, it trains common intrusion objects such as people, small animals, falling rocks, mudslides, branches, etc. through the target detection neural network. Through the trained neural network model, the input image is inferred and the target category position and size with different confidence rates are output.

[0030] The radar-visual fusion unit is mainly responsible for the decision-making fusion of radar alarm information and video alarm information, and outputs high-reliability alarm information. Figure 2 shown.

[0031] The laser-visual fusion unit obtains the 3D point cloud detection information and the whole-image target detection information of the video frame at the same time. The confidence threshold of the whole-image target detection can be set high, generally between 0.7-0.9. First, determine whether the 3D laser radar alarm information has a misdetected box. If an intrusion box is detected, enter the false alarm elimination mode; if no intrusion box is detected, enter the false alarm prevention mode.

[0032] S2 monitors foreign objects that intrude the railway perimeter under 3D point cloud. The laser waves emitted by the laser radar are scanned onto the objects, and then the three-dimensional spatial position coordinates and reflection intensity of the objects are calculated based on the received reflected waves.

[0033] S3 reduces the number of point clouds to be processed through a preset point cloud preprocessing algorithm, including noise and isolated point filtering and redundant point cloud cutting outside the defense zone, reducing the number of point clouds to be processed and accelerating the algorithm calculation speed.

[0034] The redundant point cloud cutting method outside the defense zone pre-specifies the rail plane defense zone, which is composed of multiple spatial points, and the multiple spatial points can fit a plane. All point clouds are mapped to the plane, and then each point is calculated whether it is within the closed polygonal area composed of the spatial points, and the point clouds that are not within the space are deleted, and only the points within the polygonal area are retained, and then the vertical distance h of each point cloud from the plane is calculated.

[0035] In the vertical direction, the preset height range is [h0,h1], and the height values ​​h,h0,h1 are directional, with negative values ​​below the plane and positive values ​​above the plane. Then the formula is: The point clouds that do not satisfy the formula are deleted, and finally the segmentation of the point clouds in the defense zone space is realized. Figure 3 shown.

[0036] It also includes a preset track surface fitting method to segment the intruding foreign object and the track surface to obtain the intruding foreign object point cloud, specifically including: S3.1 point cloud to obtain the normal vector, and obtain the vector field of the unknown object M Assuming that all points are located on or near the surface of the unknown object M, there exists an indicator function χM:R 3 →R: Where p = p(x, y, z) is a point in space. It satisfies: the indicator function gradient Approximate vector field Right now: Two-side divergence operator Transformed into Poisson's equation: The Poisson equation is solved by the finite element method, and the implicit function expression χ is obtained.

[0037] S3.2 extracts isosurfaces from voxels using the MC algorithm based on the voxel corner value χ(p) and the isovalue γ of the unknown object M, and outputs a triangular mesh.

[0038] S3.3 uses the fitted rail surface as the boundary between the rail surface and the intruding foreign matter, and separates the point cloud of the fitted rail surface and below. The remaining point cloud is the target point cloud containing the intruding foreign matter.

[0039] S4 uses a preset clustering algorithm to obtain the location and 3D box of intruding foreign objects and issues a radar alarm. The preset clustering algorithm is used to obtain the location and size of intruding foreign objects, and finally outputs the location and 3D box of intruding foreign objects, including: S4.1 Select a certain intrusion point as the target point, and randomly select another point to calculate the distance between the two points; S4.2 If the distance between the two points is within the preset distance threshold, the random point is marked as the target point; if the distance between the two points exceeds the distance threshold, another point is selected to calculate the distance between the two points; S4.3 When the utilization rate of the intrusive foreign object points reaches the preset utilization rate, select other unmarked intrusive foreign object points and repeat the above steps. After repeated multiple times, the size of the intrusive foreign object is reached and the classification of different intrusive foreign objects is completed.

[0040] S5 combines radar alarm information and video alarm information and outputs alarm information.

[0041] The radar-visual fusion unit fuses radar alarm information and video alarm information at the decision-making level and outputs high-reliability alarm information. Figure 2 As shown: The laser-visual fusion unit obtains the 3D point cloud detection information and the whole-image target detection information of the video frame at the same time. The confidence threshold of the whole-image target detection can be set high, generally between 0.7-0.9. First, determine whether the 3D laser radar alarm information has a misdetected box. If an intrusion box is detected, enter the false alarm elimination mode; if no intrusion box is detected, enter the false alarm prevention mode.

[0042] In the false alarm elimination mode, firstly, the position and size of the 3D intrusion box are obtained through the 3D2D unit to obtain the 2D coordinates of the intrusion and the size of the alarm box, and the intrusion box is mapped to the video frame image. On the video frame image, the ROI region of interest is intercepted with the center of the mapped box as the center and a certain multiple of the length and width of the mapped box size, generally 5-10 times, and not exceeding the original image size. The intercepted ROI region of interest is input into the video detection unit, and the confidence threshold is set relatively low, generally between 0.05-0.3.

[0043] If the detection information output by the video detection unit does not contain any information about an intruding foreign object, no alarm is required. If there is an intruding foreign object box, the intersection-over-union (IOU) ratio between the intruding foreign object box in the video and the intruding foreign object box in the radar mapping is calculated.

[0044] inersectionArea: represents the intersection of the video intrusion limit foreign object box and the radar mapping intrusion limit foreign object box; UnionArea: represents the union of the video intrusion limit box and the radar mapping intrusion limit box; If the IOU is greater than the set intersection-union ratio threshold iou_threshold (usually 0.4-0.9), an alarm message is issued, otherwise no alarm is issued.

[0045] In the mode of preventing missed alarms, first determine whether an intrusion box is identified in the target detection information of the entire video frame. If there is a box, an alarm is output directly, otherwise no alarm is output.

[0046] Example like Figure 1 As shown, the method of the present invention includes a total of 5 processing units, namely, a laser radar detection unit, a 3D2D mapping unit, a video detection unit, a radar and vision fusion unit, and an alarm unit.

[0047] The laser radar monitoring unit is mainly responsible for monitoring the intrusive foreign objects on the railway perimeter under the 3D point cloud, and outputting the location and size of the intrusive objects when they are found. Specifically, the laser wave emitted by the laser radar is scanned to the object, and then the three-dimensional spatial position coordinates (x, y, z) and reflection intensity and other properties of the object are calculated based on the received reflected wave. Through the preset point cloud preprocessing algorithm: noise and isolated point filtering, redundant point cloud cutting outside the defense zone, the number of point clouds to be processed is reduced and the algorithm operation speed is accelerated. Then the voxel grid is used to downsample the point cloud data, and only the average position point cloud in each grid is retained to further reduce the number of point clouds. Through the preset track surface fitting method, the intrusive object and the rail surface background are segmented to segment the intrusive foreign objects.

[0048] The redundant point cloud cutting method outside the defense zone pre-specifies the rail plane defense zone, which is composed of multiple spatial points, and the multiple spatial points can fit a plane. All point clouds are mapped to the plane, and then each point is calculated whether it is within the closed polygonal area composed of the spatial points, and the point clouds that are not within the space are deleted, and only the points within the polygonal area are retained, and then the vertical distance h of each point cloud from the plane is calculated.

[0049] In the vertical direction, the preset height range is [h0,h1], and the height values ​​h,h0,h1 are directional, with negative values ​​below the plane and positive values ​​above the plane. Then the formula is: The point clouds that do not satisfy the formula are deleted, and finally the segmentation of the point clouds in the defense zone space is realized. Figure 3 shown.

[0050] Further: the specific steps of using the preset rail surface fitting method to segment the intrusion object and the rail surface to obtain the intrusion foreign object point cloud are as follows: (1) Obtain the normal vector from the point cloud and obtain the vector field of the unknown object M Assuming that all points are located on or near the surface of the unknown object M, there exists an indicator function χM:R 3 →R: Where p = p(x, y, z) is a point in space. It satisfies: the indicator function gradient Approximate vector field Right now: Two-side divergence operator Transformed into Poisson's equation: The Poisson equation is solved by the finite element method to obtain the implicit function expression χ. Based on the voxel corner value χ(p) and the isovalue γ of the unknown object M, the MC algorithm is used to extract the isosurface from the voxel and output the triangular mesh.

[0051] (2) The fitted rail surface is used as the boundary between the rail surface and the intruding foreign object, and the point cloud of the fitted rail surface and below is segmented. The remaining point cloud is the target point cloud containing the intruding foreign object. The location and size of the intruding object are obtained through the preset clustering algorithm. The trajectory of the intruding object is then obtained through the target tracking algorithm. Through trajectory analysis, some flying birds and interference clusters are filtered out. Finally, the location and 3D box of the intruding object are output.

[0052] It is further obtained that the specific steps of using the preset clustering algorithm to separate the intruding foreign matter and the rail surface and then clustering them into a cluster are as follows: (1) Select a certain intrusion point as the target point and randomly select another point to calculate the distance between the two points; (2) If the distance of the randomly selected target point is within the preset distance threshold, the random point is marked as the target point; (3) If the distance between the random point and the target point exceeds the distance threshold, then take the next point to calculate the distance to the target point; (4) When the utilization rate of the intrusion foreign body points reaches the preset utilization rate, other unmarked intrusion foreign body points are selected and the above process (1)-(3) is repeated. After multiple repetitions, the classification of different intrusion foreign bodies can be completed.

[0053] The 3D2D mapping unit is mainly responsible for mapping the three-dimensional position and size of the target detected by the radar to the 2D image of the camera, and the lidar and camera realize the spatial coordinate system 1. The specific implementation first uses Zhang Youding's camera intrinsic calibration method to calculate the camera intrinsic parameter matrix. Then, the point cloud feature detection method and image features are used to establish 3D-2D feature point pairs, solve the PnP problem, estimate the external parameter matrix of the lidar and camera, and realize the three-dimensional and two-dimensional coordinate system 1.

[0054] The video detection unit is mainly responsible for detecting targets in the video frames input by the camera. Specifically, it trains common intrusion objects such as people, small animals, falling rocks, mudslides, branches, etc. through the target detection neural network. Through the trained neural network model, the input image is inferred and the target category position and size with different confidence rates are output.

[0055] The radar-visual fusion unit is mainly responsible for the decision-making fusion of radar alarm information and video alarm information, and outputs high-reliability alarm information. Figure 2 As shown: The laser-visual fusion unit obtains the 3D point cloud detection information and the whole-image target detection information of the video frame at the same time. The confidence threshold of the whole-image target detection can be set high, generally between 0.7-0.9. First, determine whether the 3D laser radar alarm information has a misdetected box. If an intrusion box is detected, enter the false alarm elimination mode; if no intrusion box is detected, enter the false alarm prevention mode.

[0056] In the false alarm elimination mode, firstly, the position and size of the 3D intrusion box are obtained through the 3D2D unit to obtain the 2D coordinates of the intrusion and the size of the alarm box, and the intrusion box is mapped to the video frame image. On the video frame image, the ROI region of interest is intercepted with the center of the mapped box as the center and a certain multiple of the length and width of the mapped box size, generally 5-10 times, and not exceeding the original image size. The intercepted ROI region of interest is input into the video detection unit, and the confidence threshold is set relatively low, generally between 0.05-0.3.

[0057] If the detection information output by the video detection unit does not contain any information about an intruding foreign object, no alarm is required. If there is an intruding foreign object box, the intersection-over-union (IOU) ratio between the intruding foreign object box in the video and the intruding foreign object box in the radar mapping is calculated.

[0058] inersectionArea: represents the intersection of the video intrusion limit foreign object box and the radar mapping intrusion limit foreign object box; UnionArea: represents the union of the video intrusion limit box and the radar mapping intrusion limit box; If the IOU is greater than the set intersection-union ratio threshold iou_threshold (usually 0.4-0.9), an alarm message is issued, otherwise no alarm is issued.

[0059] In the mode of preventing missed alarms, first determine whether an intrusion box is identified in the target detection information of the entire video frame. If there is a box, an alarm is output directly, otherwise no alarm is output.

[0060] The specific embodiments described herein are merely examples of the spirit of the present invention. The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that technicians in the technical field to which the present invention belongs can make various modifications or supplements to the described specific embodiments or replace them in a similar manner, but will not deviate from the spirit of the present invention or exceed the scope defined in the attached claims. For ordinary technicians in this field, multiple deformations and improvements can also be made without departing from the concept of the present invention. Therefore, the scope of protection of the patent of the present invention shall be based on the attached claims.

Claims

1. A railway clearance intrusion detection method based on multi-sensor fusion, characterized in that: The following steps are involved: S1 builds a railway clearance intrusion detection system; S2 monitors foreign objects invading the railway perimeter under 3D point cloud; S3 reduces the number of point clouds to be processed through a preset point cloud preprocessing algorithm; S4 uses a preset clustering algorithm to obtain the location of intruding foreign objects and the three-dimensional box and issue a radar alarm; S5 combines radar alarm information and video alarm information and outputs alarm information.

2. A railway clearance intrusion detection method based on multi-sensor fusion according to claim 1, characterized in that: The railway clearance intrusion detection system in step S1 includes a laser radar monitoring unit, a 3D2D mapping unit and a radar and vision fusion unit connected in sequence, and the radar and vision fusion unit is also connected to a video detection unit and an alarm unit.

3. A railway clearance intrusion detection method based on multi-sensor fusion according to claim 1 or 2, characterized in that: The step S2 specifically includes: after the laser wave emitted by the laser radar is scanned onto the object, the three-dimensional spatial position coordinates and reflection intensity of the object are calculated based on the received reflected waves.

4. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 1 is characterized in that: The step S3 includes filtering noise points and isolated points and cutting redundant point clouds outside the defense zone, thereby reducing the number of point clouds to be processed and accelerating the algorithm operation speed.

5. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 1 is characterized in that: The step S3 presets a rail surface fitting method to segment the intruding foreign object and the rail surface to obtain the intruding foreign object point cloud, specifically including: S3.1 Obtain the normal vector from the point cloud and obtain the vector field of the unknown object M S3.2 uses voxel grids to downsample the point cloud data, retaining only the average position point cloud in each grid; S3.3 uses the fitted track surface as the boundary between the track surface and the intruding foreign objects, and separates the point cloud on and below the fitted track surface. The remaining point cloud is the target point cloud containing the intruding foreign objects.

6. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 5 is characterized in that: The step S3.1 specifically includes assuming that all points are located on or near the surface of the unknown object M, and there exists an indicator function χM:R 3 →R: Where p = p(x, y, z) is a point in space that satisfies the gradient of the indicator function Approximate vector field Right now: Two-side divergence operator Transformed into Poisson's equation: The Poisson equation is solved by the finite element method, and the implicit function expression χ is obtained.

7. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 6 is characterized in that: The step S3.2 specifically includes extracting isosurfaces from voxels using the MC algorithm based on the voxel corner value χ(p) and the isovalue γ of the unknown object M, and outputting a triangular mesh.

8. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 1 or 5, characterized in that: The step S4 obtains the position and size of the intruding foreign body through a preset clustering algorithm, and finally outputs the position of the intruding foreign body and a three-dimensional box, which specifically includes: S4.1 Select a certain intrusion point as the target point, and randomly select another point to calculate the distance between the two points; S4.2 If the distance between the two points is within the preset distance threshold, mark the random point as the target point; if the distance between the two points exceeds the distance threshold, select another point to calculate the distance between the two points; S4.3 When the utilization rate of the intrusive foreign object points reaches the preset utilization rate, select other unmarked intrusive foreign object points and repeat the above steps. After repeated multiple times, the size of the intrusive foreign object is reached and the classification of different intrusive foreign objects is completed.

9. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 1 is characterized in that: The step S5 specifically includes obtaining the three-dimensional point cloud detection information and the whole-image target detection information of the video frame at the same time, first determining whether the three-dimensional laser radar alarm information has an intrusion box that is mistakenly detected, and if an intrusion box is detected, entering a false alarm elimination mode; if no intrusion box is detected, entering a false alarm prevention mode.

10. The railway clearance intrusion detection method based on multi-sensor fusion according to claim 9 is characterized in that: In the false alarm elimination mode, the 2D coordinates of the intruding foreign object and the size of the alarm box are first obtained, and the intruding foreign object box is mapped to the video frame image; the ROI region of interest is intercepted with the mapped intruding foreign object box as the center and the multiple of the length and width of the mapped intruding foreign object box, and the intercepted ROI region of interest is input into the video detection unit; if the detection information output by the video detection unit does not contain intruding foreign object information, no alarm is required; if there is an intruding foreign object box, the intersection and union ratio (IOU) between the video intruding foreign object box and the radar mapped intruding foreign object box is calculated: Among them, inersectionArea represents the intersection of the video intrusion limit foreign object box frame and the radar mapping intrusion limit foreign object box frame, and UnionArea represents the union of the video intrusion limit foreign object box frame and the radar mapping intrusion limit foreign object box frame. If the IOU is greater than the set intersection-union ratio threshold iou_threshold, an alarm message is issued, otherwise no alarm is issued.

Images