Method and system for detecting target object in urban rail defense area

Abstract

The invention relates to a method and a system for detecting a target object in an urban rail defense area. The method comprises the following steps: acquiring laser radar point cloud data and image data in the urban rail defense area; segmenting the laser radar point cloud data to obtain point cloud data of a region of interest; calibrating the position of the target object based on the point cloud data of the region of interest to obtain a point cloud data training sample; segmenting the image data, and calibrating the position of a target object on the segmented image data to obtain an image data training sample; inputting the point cloud data training sample and the image data training sample into a double-branch network for training to obtain a target object detection model; and detecting the target object in the target urban rail defense area by using the target object detection model. According to the method, the image sample and the laser radar point cloud sample are trained by using the double-branch network structure, and the advantages of the two samples can be fully utilized, so that the learning effect of the target object detection model is enhanced, and the possibility of missing detection and false detection is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of target object detection, and in particular to a method and system for detecting a target object in an urban rail defense zone. Background Art

[0002] With the rapid development of urban rail transit, the importance of urban rail transit systems has become increasingly prominent, and safety has become an urgent issue to be solved. In the operation of urban rail transit, there are many potential safety hazards, among which the detection of obstacles is particularly critical. The existence of these obstacles may pose a threat to the safe operation of trains. Therefore, timely and accurate detection of targets in the defense zone is of great significance to ensure driving safety and reduce accidents.

[0003] The existing obstacle detection method is mainly manual inspection, but manual inspection cannot achieve all-weather real-time monitoring. On high-load, high-efficiency urban rail, if obstacles cannot be found in time, it may endanger driving safety. In addition, obstacle identification through image recognition technology is greatly affected by the environment and cannot obtain accurate obstacle location information all day long. Summary of the invention

[0004] In order to solve the above problems, an object of the embodiments of the present invention is to provide a method and system for detecting targets in a defense zone of an urban rail transit.

[0005] A method for detecting a target object in a city rail defense zone, comprising:

[0006] Step 1: Obtain the laser radar point cloud data and image data within the urban rail defense area;

[0007] Step 2: Segment the laser radar point cloud data to obtain point cloud data of the area of ​​interest;

[0008] Step 3: Calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample;

[0009] Step 4: Segment the image data, and calibrate the position of the target object on the segmented image data to obtain image data training samples;

[0010] Step 5: Input the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model;

[0011] Step 6: Use the target object detection model to complete the detection of the target object in the target urban rail defense zone.

[0012] Preferably, when the target object is detected, the method further comprises:

[0013] Step 6.1: Use the inertial navigation device to obtain the absolute speed information of the target at each moment;

[0014] Step 6.2: Use the absolute speed information of the target object at each moment to correct the point cloud data to obtain the corrected point cloud data;

[0015] Step 6.3: Use the corrected point cloud data to repair the motion distortion of the target object.

[0016] Preferably, in step 6.1, the absolute speed calculation formula of the target at each moment is:

[0017] front_scale = (Ti-T1) / (T2-T1)

[0018] back_scale = (T2-Ti) / (T2-T1)

[0019] Vi=front_scale*V2+back_scale*V1

[0020] Wi=front_scale*W2+back_scale*W1

[0021] Among them, Vi represents the linear velocity of the target object at time Ti, Wi represents the angular velocity of the target object at time Ti, V1 represents the linear velocity output by the inertial navigation device at time T1, V2 represents the linear velocity output by the inertial navigation device at time T2, W2 represents the angular velocity output by the inertial navigation device at time T2, and W1 represents the angular velocity output by the inertial navigation device at time T1.

[0022] Preferably, in step 6.2, the correction formula of the point cloud data is:

[0023] delta_T=Ti-T0

[0024] Angle=Wi*delta_T

[0025] Pi_adjust=rotate(Pi,Angle)+Vi*delta_T

[0026] Among them, Angle represents the relative motion of the i-th point with respect to the 0-th point, Wi represents the angular velocity of the target object at time Ti, T0 represents the time of the first point of the point cloud, Ti represents the time of the i-th point in the point cloud data, Pi_adjust represents the corrected point, rotate(Pi,Angle) represents converting Angle into a rotation matrix and then multiplying it by Pi, and Pi represents the position of the i-th original point cloud.

[0027] Preferably, the step 6.3: using the corrected point cloud data to repair the motion distortion of the target object comprises:

[0028] Step 6.3.1: Calculate the curvature information of adjacent points under the same line bundle in the current frame point cloud data;

[0029] Step 6.3.2: Find the matching point with the closest curvature information in the adjacent line bundle in the point cloud of the previous frame;

[0030] Step 6.3.3: Calculate the speed information of the target object based on the distance information between adjacent matching point pairs;

[0031] Step 6.3.4: Use the velocity information of the target object to repair the motion distortion of the target object.

[0032] Preferably, in step 6.3.1, the curvature information calculation formula of adjacent points under the same line bundle is:

[0033] Curve_i=sqrt(sum(j=-5to+5)(Pj-Pi))

[0034] Among them, Curve_i represents the curvature information of the i-th point, sqrt represents the square root, sum represents the sum, Pj represents the position of the adjacent points, and Pi represents the position of the i-th point.

[0035] Preferably, in step 6.3.3, the target object speed information calculation formula is:

[0036] delta_T=Ti-Ti'

[0037] Vi'=(Pi-P') / delta_T

[0038] Among them, delta_T represents the time between two adjacent frames, Ti represents the i-th frame, Ti' represents the previous frame, Vi' represents the linear velocity of the target object calculated from the point cloud data at time Ti, Pi represents the position of the i-th point in the current frame, and P' represents the position of the i-th point in the previous frame.

[0039] Preferably, in step 6.3.4, the formula is used:

[0040] delta_T=Ti-T0

[0041] P_adjust=Pi+Vi'*delta_T

[0042] Repair the motion distortion of the target object; where delta_T represents the time difference between the first point and the i-th point on the target object, P_adjust represents the point after dedistortion, and Pi represents the position of the i-th point in the current frame.

[0043] The present invention also provides a detection system for a target object in a city rail defense zone, comprising:

[0044] Data acquisition module, used to obtain laser radar point cloud data and image data within the urban rail defense zone;

[0045] A point cloud data segmentation module, used to segment the laser radar point cloud data to obtain point cloud data of an area of ​​interest;

[0046] A calibration module is used to calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample;

[0047] An image data segmentation module is used to segment the image data and calibrate the position of the target object on the segmented image data to obtain image data training samples;

[0048] A training module, used for inputting the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model;

[0049] The target object detection module is used to complete the detection of the target object in the target urban rail defense zone using the target object detection model.

[0050] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0051] The present invention relates to a method and system for detecting targets in an urban rail defense zone. Compared with the prior art, the present invention utilizes a dual-branch network structure to train image samples and laser radar point cloud samples, which can fully utilize the advantages of both, thereby enhancing the learning effect of the target detection model and reducing the possibility of missed detection and false detection.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A flow chart of a method for detecting a target object in a city rail defense zone provided by the present invention;

[0055] Figure 2 A schematic diagram of the structure of a detection device for a target object in a defense zone of a urban rail transit provided by the present invention;

[0056] Figure 3 This is a schematic diagram of extracting a region of interest provided by the present invention. DETAILED DESCRIPTION

[0057] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0059] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] See also Figure 1-2 The present invention provides a method for detecting a target object in a city rail defense zone, which is completed on the basis of a detection device for a target object in a city rail defense zone, and mainly consists of four parts, namely, a data acquisition unit, a field service unit, a central management unit, and a foreign object detection platform.

[0061] (1) Front-end acquisition system: The 3D laser radar and video camera constitute the front-end monitoring part of the scene, which is mainly responsible for the real-time acquisition and processing of the scene data and uploading it to the scene server. In practical applications, the present invention can set up a monitoring column next to the railroad track, with two laser radars installed on one column, with a height of 3 meters and a distance of 2 meters from the track.

[0062] (2) On-site service system: mainly responsible for data analysis and processing results. It extracts characteristic points of the roadbed, pillows, and rails from the point cloud data collected by the radar, and performs real-time comparison. It determines whether there are obstacles based on the comparison results and executes alarm operations. It also processes the alarm information sent by each radar in real time, uploads the alarm information to the monitoring center, or directly triggers the alarm unit to issue an alarm.

[0063] (3) Central management system: responsible for monitoring the entire monitored road section, centralizing the data of field service systems scattered across the country, forwarding and storing data between the field service system and the client, and real-time monitoring of the operating status of the on-site host cabinet and each radar.

[0064] (4) Foreign object detection platform: It provides a user interface for the line obstacle three-dimensional detection system, displays the operation information of the on-site equipment, real-time point cloud information, on-site video information, and equipment alarms, fault information, and equipment deactivation and activation information of each section of the road.

[0065] Furthermore, the method for detecting a target object in a city rail defense zone of the present invention comprises:

[0066] Step 1: Obtain the laser radar point cloud data and image data within the urban rail defense area;

[0067] Step 2: Segment the laser radar point cloud data to obtain point cloud data of the area of ​​interest;

[0068] Since the range of 3D laser radar can reach 300m, and the field of view can reach 90° horizontally and 30° vertically, the field of view can not only detect track point cloud data, but also cover the surrounding scenes. This requires us to set the ROI (region of interest) of the track information in the field of view and extract the track point cloud data as the monitoring area of ​​the present invention. Figure 3 As mentioned above, the red area is the set ROI, including the track bed, pillow stone, and rail point cloud data.

[0069] Step 3: Calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample;

[0070] In step 3, the present invention needs to perform data background sampling on the roadbed, pillow stones, rails or target objects in the ROI area, establish a background sample map, and extract characteristic values ​​and the collection of reflectivity information of different materials, and store them in a database to mark the position of the target objects in different situations.

[0071] Step 4: Segment the image data, and calibrate the position of the target object on the segmented image data to obtain image data training samples;

[0072] Since the image data will also capture the surrounding scenes, the present invention needs to use seed points to complete the image segmentation in a breadth-first search manner.

[0073] Step 5: Input the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model;

[0074] Compared with the three-dimensional point cloud information of the lidar, the two-dimensional image loses the depth information. However, due to the rich semantic information contained in the image, more comprehensive semantic information mining can achieve better results. The present invention adopts a dual-branch network structure, one branch network structure is used to extract detail information, and the other branch is used to extract semantic information. The features output by the two branches are fused together through a fusion module for feature extraction to complete the training process of the target detection model. In this way, the information of the two can be combined to improve the accuracy of target detection.

[0075] Step 6: Use the target object detection model to complete the detection of the target object in the target urban rail defense zone.

[0076] Furthermore, when a target object is detected, it also includes: using an inertial navigation device to perform self-motion compensation on the laser radar point cloud to obtain accurate laser radar point cloud data.

[0077] Furthermore, the step includes:

[0078] Step 6.1: Use the inertial navigation device to obtain the absolute speed information of the target at each moment;

[0079] In practical applications, the present invention first needs to synchronize the sensor time and use GPS to synchronize the time between multiple sensors. This is a prerequisite, and then obtain the absolute speed information of the moving target. In the present invention, the absolute speed information of each time period of a frame of point cloud can be obtained by interpolation;

[0080] Since the output frequency of the inertial navigation device is much higher than the frequency of a frame of the laser radar point cloud, the speed information of each point can be obtained by interpolation calculation. For example, a point Pi falling between T1 and T2, with a timestamp of Ti, then the speed information at Ti is:

[0081] front_scale = (Ti-T1) / (T2-T1)

[0082] back_scale = (T2-Ti) / (T2-T1)

[0083] Vi=front_scale*V2+back_scale*V1

[0084] Wi=front_scale*W2+back_scale*W1

[0085] Among them, Angle represents the relative motion of the i-th point with respect to the 0-th point, Wi represents the angular velocity of the target object at time Ti, T0 represents the time of the first point of the point cloud, Ti represents the time of the i-th point in the point cloud data, Pi_adjust represents the corrected point, rotate(Pi,Angle) represents converting Angle into a rotation matrix and then multiplying it by Pi, and Pi represents the position of the i-th original point cloud.

[0086] Step 6.2: Use the absolute speed information of the target object at each moment to correct the point cloud data to obtain the corrected point cloud data;

[0087] In step 6.2, the motion repair of point i is as follows, where the repaired point cloud is aligned to the first point of a frame of point cloud, which is time T0.

[0088] delta_T=Ti-T0

[0089] Angle=Wi*delta_T

[0090] Pi_adjust=rotate(Pi,Angle)+Vi*delta_T

[0091] Among them, Angle represents the relative motion of the i-th point with respect to the 0-th point, Wi represents the angular velocity of the target object at time Ti, T0 represents the time of the first point of the point cloud, Ti represents the time of the i-th point in the point cloud data, Pi_adjust represents the corrected point, rotate(Pi,Angle) represents converting Angle into a rotation matrix and then multiplying it by Pi, and Pi represents the position of the i-th original point cloud.

[0092] Step 6.3: Use the corrected point cloud data to repair the motion distortion of the target object.

[0093] Further, step 6.3 includes:

[0094] Step 6.3.1: Calculate the curvature information of adjacent points under the same line bundle in the current frame point cloud data;

[0095] In step 6.3.1, the curvature information of adjacent points under the same line bundle is calculated as follows:

[0096] Curve_i=sqrt(sum(j=-5to+5)(Pj-Pi))

[0097] Among them, Curve_i represents the curvature information of the i-th point, sqrt represents the square root, sum represents the sum, Pj represents the position of the adjacent points, and Pi represents the position of the i-th point.

[0098] Step 6.3.2: Find the matching point with the closest curvature information in the adjacent line bundles in the previous frame point cloud; in step 6.3.2, when finding the matching point of the previous frame, use the curvature information of the 5 adjacent points in the current frame, and find some closest Curve points within ±10 points of the same position of the same line bundle in the previous frame and within the range of ±3 lines. The L1 distance can be used directly.

[0099] Step 6.3.3: Calculate the speed information of the target object based on the distance information between adjacent matching point pairs;

[0100] In step 6.3.3, the target object speed information calculation formula is:

[0101] delta_T=Ti-Ti'

[0102] Vi'=(Pi-P') / delta_T

[0103] Among them, delta_T represents the time between two adjacent frames, Ti represents the i-th frame, Ti' represents the previous frame, Vi' represents the linear velocity of the target object calculated from the point cloud data at time Ti, Pi represents the position of the i-th point in the current frame, and P' represents the position of the i-th point in the previous frame.

[0104] Step 6.3.4: Repair the motion distortion of the target object by using the velocity information of the target object;

[0105] In step 6.3.4, the formula is used:

[0106] delta_T=Ti-T0

[0107] P_adjust=Pi+Vi*delta_T

[0108] Repair the motion distortion of the target object; where delta_T represents the time difference between the first point and the i-th point on the target object, P_adjust represents the point after dedistortion, and Pi represents the position of the i-th point in the current frame.

[0109] The present invention utilizes a dual-branch network structure to train image samples and lidar point cloud samples, which can fully utilize the advantages of both, thereby enhancing the learning effect of the target detection model and reducing the possibility of missed detection and false detection.

[0110] The present invention also provides a detection system for a target object in a city rail defense zone, comprising:

[0111] Data acquisition module, used to obtain laser radar point cloud data and image data within the urban rail defense zone;

[0112] A point cloud data segmentation module, used to segment the laser radar point cloud data to obtain point cloud data of an area of ​​interest;

[0113] A calibration module is used to calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample;

[0114] An image data segmentation module is used to segment the image data and calibrate the position of the target object on the segmented image data to obtain image data training samples;

[0115] A training module, used for inputting the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model;

[0116] The target object detection module is used to complete the detection of the target object in the target urban rail defense zone using the target object detection model.

[0117] Compared with the prior art, the beneficial effects of the system for detecting targets within a city rail defense zone provided by the present invention are the same as the beneficial effects of the method for detecting targets within a city rail defense zone described in the above technical solution, and will not be elaborated here.

[0118] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technical solution that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting a target object in a city rail defense zone, characterized in that: include: Step 1: Obtain the laser radar point cloud data and image data within the urban rail defense area; Step 2: Segment the laser radar point cloud data to obtain point cloud data of the area of ​​interest; Step 3: Calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample; Step 4: Segment the image data, and calibrate the position of the target object on the segmented image data to obtain image data training samples; Step 5: Input the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model; Step 6: Use the target object detection model to complete the detection of the target object in the target urban rail defense zone.

2. A method for detecting a target object in a city rail defense zone according to claim 1, characterized in that: When a target is detected, it also includes: Step 6.1: Use the inertial navigation device to obtain the absolute speed information of the target at each moment; Step 6.2: Use the absolute speed information of the target object at each moment to correct the point cloud data to obtain the corrected point cloud data; Step 6.3: Use the corrected point cloud data to repair the motion distortion of the target object.

3. A method for detecting a target object in a city rail defense zone according to claim 2, characterized in that: In step 6.1, the absolute speed calculation formula of the target at each moment is: front_scale = (Ti-T1) / (T2-T1) back_scale = (T2-Ti) / (T2-T1) Vi=front_scale*V2+back_scale*V1 Wi=front_scale*W2+back_scale*W1 Among them, Vi represents the linear velocity of the target object at time Ti, Wi represents the angular velocity of the target object at time Ti, V1 represents the linear velocity output by the inertial navigation device at time T1, V2 represents the linear velocity output by the inertial navigation device at time T2, W2 represents the angular velocity output by the inertial navigation device at time T2, and W1 represents the angular velocity output by the inertial navigation device at time T1.

4. A method for detecting a target object in a city rail defense zone according to claim 3, characterized in that: In step 6.2, the correction formula of the point cloud data is: delta_T=Ti-T0 Angle=Wi*delta_T Pi_adjust=rotate(Pi,Angle)+Vi*delta_T Among them, Angle represents the relative motion of the i-th point with respect to the 0-th point, Wi represents the angular velocity of the target object at time Ti, T0 represents the time of the first point of the point cloud, Ti represents the time of the i-th point in the point cloud data, Pi_adjust represents the corrected point, rotate(Pi,Angle) represents converting Angle into a rotation matrix and then multiplying it by Pi, and Pi represents the position of the i-th original point cloud.

5. A method for detecting a target object in a city rail defense zone according to claim 4, characterized in that: The step 6.3: using the corrected point cloud data to repair the motion distortion of the target object includes: Step 6.3.1: Calculate the curvature information of adjacent points under the same line bundle in the current frame point cloud data; Step 6.3.2: Find the matching point with the closest curvature information in the adjacent line bundle in the point cloud of the previous frame; Step 6.3.3: Calculate the speed information of the target object based on the distance information between adjacent matching point pairs; Step 6.3.4: Use the velocity information of the target object to repair the motion distortion of the target object.

6. A method for detecting a target object in a city rail defense zone according to claim 5, characterized in that: In step 6.3.1, the curvature information of adjacent points under the same line bundle is calculated as follows: Curve_i=sqrt(sum(j=-5to+5)(Pj-Pi)) Among them, Curve_i represents the curvature information of the i-th point, sqrt represents the square root, sum represents the sum, Pj represents the position of the adjacent points, and Pi represents the position of the i-th point.

7. A method for detecting a target object in a city rail defense zone according to claim 6, characterized in that: In step 6.3.3, the target object speed information calculation formula is: delta_T=Ti-Ti' Vi'=(Pi-P') / delta_T Among them, delta_T represents the time between two adjacent frames, Ti represents the i-th frame, Ti' represents the previous frame, Vi' represents the linear velocity of the target object calculated from the point cloud data at time Ti, Pi represents the position of the i-th point in the current frame, and P' represents the position of the i-th point in the previous frame.

8. A method for detecting a target object in a city rail defense zone according to claim 5, characterized in that: In step 6.3.4, the formula is used: delta_T=Ti-T0 P_adjust=Pi+Vi'*delta_T Repair the motion distortion of the target object; where delta_T represents the time difference between the first point and the i-th point on the target object, P_adjust represents the point after dedistortion, and Pi represents the position of the i-th point in the current frame.

9. A detection system for targets in urban rail defense areas, characterized in that: include: Data acquisition module, used to obtain laser radar point cloud data and image data within the urban rail defense zone; A point cloud data segmentation module, used to segment the laser radar point cloud data to obtain point cloud data of an area of ​​interest; A calibration module is used to calibrate the position of the target object based on the point cloud data of the area of ​​interest to obtain a point cloud data training sample; An image data segmentation module is used to segment the image data and calibrate the position of the target object on the segmented image data to obtain image data training samples; A training module, used for inputting the point cloud data training samples and the image data training samples into a dual-branch network for training to obtain a target object detection model; The target object detection module is used to complete the detection of the target object in the target urban rail defense zone using the target object detection model.

Images