Railway safety protection method and system based on unmanned aerial vehicle

Abstract

The invention discloses a railway safety protection method and system based on an unmanned aerial vehicle, and the method comprises the steps: firstly, obtaining the video stream data of a construction scene in real time based on the unmanned aerial vehicle; secondly, segmenting a railway in the obtained video stream data, extracting an uplink track mask and a downlink track mask, calculating a fitting straight line of an uplink track and a downlink track, and obtaining constructors in the video stream data and the coordinate position of each constructor; then, based on the fitting straight line of the uplink track and the downlink track, an electronic fence is automatically generated; and finally, setting hyper-parameters, and judging whether the constructor breaks into the dangerous area or not based on the coordinate position of the constructor and the electronic fence. According to the method, an electronic fence does not need to be drawn manually, and only the unmanned aerial vehicle needs to be carried to a designated construction site without external cooperation. In addition, the position of the constructor is obtained in real time, whether the constructor breaks into the dangerous area or not is monitored in real time, and safety and reliability of railway construction are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of rail transit technology, and in particular relates to a railway safety protection method and system based on unmanned aerial vehicle. Background Art

[0002] With the increasing density of railway transportation, railway construction, electrical and engineering personnel need to construct, inspect and maintain equipment related to railway lines. In the case of heavy construction and transportation tasks, trains often run on the line during construction and inspection and maintenance. Railways are generally divided into up and down tracks. When workers are working on the up track, the down track is normally open. Many workers lack safety awareness. When they take a break, they will run to another track for rest, which will cause safety hazards. In addition, the railway construction process has the characteristics of many construction points, long lines, and poor working environment. Personnel face various risks, especially most railway construction sites are constructed in the field, which makes personnel management difficult and personnel safety cannot be guaranteed. Therefore, when the staff are working on site, they need to enclose the on-site work site and dangerous areas. At present, the safety protection methods for construction site personnel are still traditional methods such as setting up warning lines and relying on personnel work experience. Therefore, the safety management of railway construction personnel is relatively rough, lacking effective safety management methods, and cannot effectively guarantee the personal safety of personnel.

[0003] Under the complex and harsh conditions of railway construction, personnel safety is greatly threatened. Figure 1 As shown, an existing management method is introduced, which is divided into two steps. One is to obtain the GIS database along the line, manually mark the dangerous area in the GIS map library, and delineate the electronic fence information. The other is to install the foundation enhancement system along the line, which requires construction workers to wear positioning devices and install positioning devices on vehicles. Then the location information of the vehicle and construction workers can be displayed on the map, and finally the operation is determined whether it is in violation of the rules based on the area of ​​the electronic fence. However, the above method is cumbersome. When constructing in different places, it is necessary to update the GIS map information and manually delineate the electronic fence. Furthermore, it is necessary to install positioning devices on construction workers and engineering machinery vehicles. Construction workers are prone to lose positioning devices or do not wear positioning devices. There are many uncontrollable factors, and when construction workers enter dangerous areas, they cannot be prompted with danger signals.

[0004] How to avoid safety accidents among construction workers and how to better ensure the personal safety of workers has become a practical problem that needs to be solved urgently. Summary of the invention

[0005] In response to the above problems, the present invention provides a railway safety protection method and system based on drones, which have strong timeliness, safety and reliability.

[0006] The purpose of the present invention is to provide a railway safety protection method based on unmanned aerial vehicle, comprising: Obtain video stream data of construction scenes in real time based on drones; Segment the railway in the acquired video stream data, extract the up track mask and the down track mask, calculate the fitting straight lines of the up track and the down track, and acquire the construction workers in the video stream data and the coordinate position of each construction worker; Automatically generate electronic fence based on the fitting straight line of the up track and down track; Set hyperparameters to determine whether construction workers have entered dangerous areas based on their coordinate positions and electronic fences.

[0007] Further, the railway in the acquired video stream data is segmented, and the uplink track mask and the downlink track mask are extracted, including: The acquired video stream data is input into the semantic segmentation algorithm deeplabv3 model, and a black and white image is output, where the white part represents the railway; Based on the output black and white image, obtain the pixel coordinates of the railway; The density clustering algorithm is used to classify the obtained pixel coordinate positions of the railway, and the pixel coordinates of the up track and the down track are classified. The pixel coordinates of the up track constitute the up track mask, and the pixel coordinates of the down track constitute the down track mask.

[0008] Further, the calculation of the fitting straight lines of the upward and downward tracks includes, The least square method is used to perform straight line fitting on the pixel coordinates of each corresponding track to obtain the fitting straight lines of the uplink track and the downlink track.

[0009] Furthermore, the least square method is used to perform straight line fitting on the pixel coordinates of each corresponding track, including: Prepare data sets of pixel point coordinates of the uplink track and the downlink track respectively; Establish a fitted straight line model; Based on the fitted straight line model, the objective function of the fitted straight line is established; Based on the established fitting straight line objective function, the target equation group is obtained by differentiating the slope and intercept in the fitting straight line model; Substitute the data in the data set of pixel point coordinates of the uplink track and the downlink track into the target equation group to obtain the slope and intercept of the fitting line of the uplink track and the downlink track; The slope and intercept of the upward track fitting line and the downward track fitting line are respectively substituted into the fitting line model to obtain the upward track fitting line and the downward track fitting line.

[0010] Furthermore, based on the fitting straight line of the up track and the down track, the electronic fence is automatically generated including: Determine the slope and intercept of the electronic fence, wherein the slope of the electronic fence is obtained according to a preset algorithm based on the slope of the fitting straight line of the upward track and the slope of the fitting straight line of the downward track, and the intercept of the electronic fence is obtained according to a preset algorithm based on the intercept of the fitting straight line of the upward track and the intercept of the fitting straight line of the downward track; Based on the determined slope and intercept of the electronic fence, a first straight line is formed, and the first straight line is the electronic fence.

[0011] Further, obtaining the coordinate position of the construction workers and each construction worker in the video stream data includes: The trained neural network-based target detection model is used to identify the construction workers in the video image, and the coordinate positions of the construction workers in the video image are obtained.

[0012] Furthermore, the training process of the neural network-based target detection model includes: Collect construction scene video stream data at the construction site; Label the construction workers in all the collected video stream data and create a data set; The selected neural network model is iteratively trained using the data set, the selected neural network model is optimized, and finally the optimized neural network model is used as the neural network-based target detection model.

[0013] Further, setting hyperparameters includes determining dangerous areas and safe areas of the railway based on the location of the construction work, including, Determine whether the construction work location is in the up track area or the down track area, where: If the construction work is located in the up track area, the down track area is the dangerous area and the up track area is the safe area; If the construction work location is in the down track area, the up track area is the dangerous area and the down track area is the safe area.

[0014] Furthermore, it also includes determining the track area where the construction personnel are located, including, The Y axis in the video image in the video stream data to be detected is preset to have the downward direction in the image as the positive direction; When the Y coordinate in any construction worker's coordinate position is less than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the up track area; When the Y coordinate in any construction worker's coordinate position is greater than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the down track area.

[0015] Further, based on the coordinate position of the construction personnel and the electronic fence, judging whether the construction personnel have entered the dangerous area includes: If the up-track area is a dangerous area, when the coordinate position of any construction worker is in the up-track area, it means that the construction worker has entered the dangerous area; If the down track area is a dangerous area, when the coordinate position of any construction worker is in the down track area, it means that the construction worker has entered the dangerous area; When construction workers enter a dangerous area, drones are used to issue warnings.

[0016] Another object of the present invention is to provide a railway safety protection system based on a drone, comprising a drone, wherein the drone comprises: The acquisition module is used to obtain video stream data of the construction scene in real time based on the drone; A processing module, used for segmenting the railway in the acquired video stream data, extracting the up track mask and the down track mask, calculating the fitting straight lines of the up track and the down track, and acquiring the construction workers in the video stream data and the coordinate position of each construction worker; A generation module, used to automatically generate an electronic fence based on the fitted straight line of the up track and the down track; The monitoring module is used to set hyperparameters and determine whether construction workers have entered dangerous areas based on their coordinate positions and electronic fences.

[0017] The protection method of the present invention uses a drone to monitor the track construction scene from high altitude and generates an electronic fence in real time. The above method can be applied to any construction location beside the railway. There is no need to manually draw the electronic fence. When the construction site is changed, it only needs to carry the drone to the designated construction site without the need for external cooperation.

[0018] In addition, the safety and reliability of railway construction are guaranteed by simultaneously obtaining the real-time location of construction workers and monitoring whether they have entered dangerous areas.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flow chart of a railway construction safety protection method in the prior art is shown; Figure 2 A schematic diagram of a flow chart of a railway safety protection method based on a drone in an embodiment of the present invention is shown; Figure 3 A schematic diagram of another railway safety protection method based on drones in an embodiment of the present invention is shown; Figure 4 An image of a railway construction scene acquired in real time based on a drone in an embodiment of the present invention is shown; Figure 5 A schematic diagram of a deeplabv3 algorithm flow in an embodiment of the present invention is shown; Figure 6 A schematic diagram of a railway after semantic segmentation by deeplabv3 in an embodiment of the present invention is shown; Figure 7 A schematic diagram of fitting straight lines of an upward track and a downward track in an embodiment of the present invention is shown; Figure 8 A schematic diagram of an electronic fence in an embodiment of the present invention is shown; Fig. 9 A flowchart of a training process of a target detection model based on a neural network in an embodiment of the present invention is shown; Fig.10 A schematic structural diagram of a railway safety protection system based on an unmanned aerial vehicle in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] like Figure 2As shown, in an embodiment of the present invention, a method for railway safety protection based on a drone is introduced, and the method includes, first, obtaining video stream data of a construction scene in real time based on a drone; second, segmenting the railway in the obtained video stream data, extracting the up track mask and the down track mask, and calculating the fitting straight line of the up track and the down track, and obtaining the coordinate position of the construction personnel and each construction personnel in the video stream data; then, automatically generating an electronic fence based on the fitting straight line of the up track and the down track; finally, setting hyper parameters, and judging whether the construction personnel have entered a dangerous area based on the coordinate position of the construction personnel and the electronic fence. The construction scene is monitored from a high altitude by a drone, and an electronic fence is generated in real time. The above method can be applied to any construction location beside the railway, without manually drawing the electronic fence, changing the construction site, and only carrying the drone to the designated construction site, without external cooperation. In addition, by simultaneously obtaining the location of the construction personnel in real time and monitoring whether the construction personnel have entered a dangerous area in real time, the safety and reliability of railway construction are guaranteed.

[0024] Specifically, Figure 3 As shown, the railway safety protection method based on drones includes two parts, one is to generate electronic fences in real time, and the other is to monitor whether construction workers enter dangerous areas, which specifically includes the following steps: Step S1: The drone acquires video stream data in real time, wherein the video stream data includes video images and pictures, wherein the video images are superimposed on the pictures. Figure 4 Shown is a video image in the video stream data.

[0025] Step S11: adopt Figure 5 The deeplabv3 (semantic segmentation model based on convolutional neural network) image semantic segmentation shown in FIG. segments the railway in the acquired video stream data; first, the acquired video stream data (including video images and pictures) is input into the semantic segmentation algorithm deeplabv3 model, the railway in the acquired video stream data is segmented, and a black and white image is output, where the white part represents two tracks, namely the up track and the down track. The specific segmentation effect is shown in FIG. Figure 6 As shown in Figure 1, the semantic segmentation algorithm deeplabv3 is used to extract the uplink and downlink tracks. Figure 5As shown in the figure, Deeplabv3 is a semantic segmentation model based on convolutional neural networks. Deeplabv3 also introduces a new encoder-decoder structure called "ASPP with decoder". Among them, the ASPP (Atrous Spatial Pyramid Pooling) part is similar to the ASPP module in Deeplabv3, but adds multiple convolution operations with different void rates, and adds a 1x1 convolution layer (Conv) after the last layer for dimensionality reduction. The decoder part upsamples the low-resolution feature map to the same resolution as the original image through deconvolution operations, and combines the feature map output by the upsampling encoder with the high-resolution feature map output by the ASPP part to finally generate the prediction result. In addition, by4 represents a 4-fold magnification of the spatial dimension, and Concat is used to output and merge the different features of the captured multiple branches. Therefore, the Deeplabv3 network structure is mainly divided into two parts: encoding and decoding. In the encoder, DCNN is a deep convolutional neural network, Atrous Conv is a hole convolution, 3×3 represents the convolution kernel size, and rate n represents that there are n-1 hole values ​​between adjacent convolution values. Semantic segmentation is a classification at the pixel level. Pixels belonging to the same category are classified into one category. Therefore, semantic segmentation understands the image at the pixel level, making the segmentation of video images in the video stream more accurate.

[0026] Step S12: Obtain pixel coordinate positions of the uplink track mask and the downlink track mask, that is, obtain the pixel coordinate positions of the railway based on the black and white image, such as Figure 4 In the example above, all pixels belonging to the railroad tracks are classified into one category. Figure 5 The left side is the input image of deeplabv3, which is also the picture in the video stream data obtained by the drone, that is, Figure 4 pass Figure 5 After convolution of the deeplabv3 semantic segmentation neural network, we get Figure 5 The output image on the right is a black and white image, in which the pixel value of the black part is 0 and the pixel value of the white part is 255. The result obtained through semantic segmentation records the coordinates of the pixel points with a pixel value of 255, that is, the coordinates of the pixel points with a pixel value of 255 are all the pixel points of the railroad, that is, the coordinate positions of the pixel points of the railway are obtained. The railway includes the up track and the down track, but at this time, it is not distinguished whether it is the pixel point coordinates of the up track or the down track.

[0027] Step S13: using a density clustering algorithm to classify the obtained railway pixel coordinates, and classify the pixel coordinates of the up track and the pixel coordinates of the down track. The density clustering algorithm includes: Input: Sample set , neighborhood parameters , sample distance metric, where , etc. are the pixel coordinates of the railway, that is, all ( )and( ), is the distance threshold, which is used to determine whether two data points are close to each other. The minimum number of data points required to consider a region dense.

[0028] Output: cluster division C, i.e., the result of dividing the pixel point coordinate positions into several clusters, each cluster is a subset of the pixel point coordinate positions, and the embodiment of the present invention includes two clusters, i.e., the pixel point coordinate positions of the uplink track and the pixel point coordinate positions of the downlink track.

[0029] The specific steps include: (1) Initialize the core object collection , initialize the number of clusters k=0, initialize the unvisited sample set =D, cluster partition C=

[0030] (2) For j = 1, 2, ... m, (j represents the number of pixel coordinates. For example, if there are 1000 mask pixel coordinates for the uplink and downlink tracks, then j needs to be traversed from 1 to 1000, and each time a pixel coordinate needs to be clustered, that is, divided into the corresponding class), follow the steps below to find all the core objects: a) Find samples by distance measurement of -Neighborhood subsample set .

[0031] b) If the number of samples in the subsample set satisfies , the sample Added core object sample collection: .

[0032] (3) If the core object collection , the algorithm ends, otherwise it goes to step 4.

[0033] (4) In the core object collection In the example above, randomly select a core object , initialize the current cluster core object queue , Initialize the category number , initialize the current cluster sample set , Update the unvisited sample set .

[0034] (5) If the current cluster core object queue , then the current cluster Generation completed, update cluster division , Update the core object collection , go to step 3. Otherwise update the core object set .

[0035] (6) In the current cluster core object queue Extract a core object , through the neighborhood distance threshold Find all -Neighborhood subsample set ,make , update the current cluster sample set , update the unvisited sample set ,renew , go to step 5.

[0036] The output result is: cluster division In the embodiment of the present invention, according to the above density clustering steps, there are only two categories in the end, namely , C 1 Represents the uplink orbit mask, C 2 Represents the downlink track mask.

[0037] Step S14: Use the least square method to perform straight line fitting on the corresponding pixel coordinates on each track to obtain the fitting straight lines of the uplink track and the downlink track. Specifically, The pixel point coordinate data sets of the uplink track and the downlink track are respectively prepared. For example, the pixel point coordinate data sets are shown in Table 1. Table 1 Dataset of pixel coordinates

[0038] Establish a fitted straight line model. Specifically, assume that the fitted straight line models of the upward and downward tracks are . Still using the data in Table 1 as an example, there are n data in the pixel point coordinate data set of any track. , where n represents one or more data, but is not limited to this. In certain scenarios, n can also be 0, and i satisfies , then for each , the straight line model of the track is established as:

[0039] Based on the fitted straight line model, the objective function of the fitted straight line is established, where the objective function satisfies:

[0040] Based on the established fitting straight line objective function, the target equation group is obtained by differentiating the slope and intercept in the fitting straight line model, which includes: first, based on formula (2) , respectively, for the slope and the intercept Derivation obtains formulas (3) and (4):

[0041] Then, let the partial derivatives in formulas (3) and (4) be 0, and we get the following system of linear equations with two variables:

[0042] The set of two-variable linear equations shown in formulas (5) and (6) is the target set of equations. By solving the corresponding data in the pixel point coordinate data set of the uplink track and the pixel point coordinate data of the downlink track through formulas (5) and (6), the slope k of the fitting line of the uplink track can be obtained respectively. 1 and intercept b 1 , and the slope k of the fitting line of the downward trajectory 2 and intercept b 2 , thus fitting two straight lines such as Figure 7 As shown, one of them represents the running direction of the up track and the other represents the running direction of the down track. Further, the formulas for the fitting straight lines of the up track and the down track are obtained as formula (7) and formula (8):

[0043] Step S15, automatically generating an electronic fence based on the fitting straight line of the upward track and the downward track includes: Determine the slope and intercept of the electronic fence, wherein the slope of the electronic fence is obtained according to a preset algorithm based on the slope of the fitting line of the upward track and the slope of the fitting line of the downward track, and the intercept of the electronic fence is obtained according to a preset algorithm based on the intercept of the fitting line of the upward track and the intercept of the fitting line of the downward track. Specifically, the preset algorithm may be that the slope of the electronic fence is the average of the slope of the fitting line of the upward track and the slope of the fitting line of the downward track, and the intercept of the electronic fence is the average of the intercept of the fitting line of the upward track and the intercept of the fitting line of the downward track. Still using formulas (7) and (8) for exemplary explanation, the slope of the electronic fence is and the intercept satisfy:

[0044] But not limited to this, preferably, the preset algorithm can also be set according to the actual scene a value between the uplink and downlink track intercepts, preferably, the preset algorithm includes setting a ratio , the slope of the electronic fence and the intercept satisfy:

[0045] Therefore, the average value in the above formulas (9) and (10) can be considered as the ratio The value of is 0.5.

[0046] Based on the determined slope and intercept of the electronic fence, a first straight line is formed, and the first straight line is the electronic fence, wherein the formula of the electronic fence satisfies:

[0047] Specifically, the electronic fence obtained is as follows: Figure 8 As shown, , They are the pixel coordinates on the uplink and downlink tracks respectively.

[0048] like Figure 3 In the process, while obtaining the electronic fence, the construction workers in the video stream data and the coordinate position of each construction worker are also obtained, which specifically includes the following steps: Step S21, calling the trained neural network-based target detection model; wherein the coordinate position of the construction personnel is obtained through a target detection algorithm such as a yolo series neural network model. Specifically, it is necessary to obtain a detection model through offline training of a target detection algorithm such as a yolo (you only look once, a convolution-based neural network target detection network structure) series neural network model, so as to Fig. 9 As shown, the training process of the neural network-based target detection model includes: Collect construction scene video stream data at the construction site; Label the construction workers in all the collected video stream data and create a data set; The selected neural network model is iteratively trained using the data set, the selected neural network model is optimized, and finally the optimized neural network model is used as the target detection model based on the neural network. Among them, the selected neural network module is optimized to obtain a neural network model that meets the requirements of detection accuracy, detection speed, etc.

[0049] Step S22, obtaining the construction workers in the video image and the coordinate position of each construction worker, includes using a trained neural network-based target detection model to identify the construction workers in the video image and obtaining the coordinate position of the construction workers in the video image.

[0050] In the embodiment of the present invention, the coordinate positions of n construction workers in the image are obtained according to the target detection model based on the neural network. , and z=0, 1, ..., n-1. Among them, and It can be any position in the worker detection frame. Preferably, and is the center point of the detection box.

[0051] In an embodiment of the present invention, setting the hyperparameters includes determining the dangerous area and the safe area of ​​the railway based on the construction work location, including determining whether the construction work location is in the up track area or the down track area, wherein if the construction work location is in the up track area, the down track area is the dangerous area, and the up track area is the safe area; if the construction work location is in the down track area, the up track area is the dangerous area, and the down track area is the safe area. Wherein, the construction work location is known location data, and the hyperparameters are determined according to the on-site construction personnel. If the construction personnel are constructing in the up track area, then the down track is a dangerous area. Similarly, if the construction personnel are constructing in the down track, then the up track is set as a dangerous area. For example, Figure 4 In the construction scenario shown, the construction work location is in the down track area, so it is necessary to set the down track area as a safe area and the up track area as a dangerous area in advance.

[0052] In the embodiment of the present invention, the method further includes determining the track area where the construction personnel are located, which includes presetting the Y axis in the video image in the detected video stream data to take the downward direction in the image as the positive direction, such as Figure 8 As shown. Among them, when the Y coordinate of any construction worker's coordinate position is less than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the upward track area; when the Y coordinate of any construction worker's coordinate position is greater than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the downward track area. That is, with the electronic fence as the dividing line, the area above the electronic fence is the upward track area, and the area below the electronic fence is the downward track area, so that the coordinate positions of the construction workers in the upward and downward track areas respectively meet the following requirements:

[0053] Formula (14) indicates that the coordinate position of the construction personnel is in the up-track area, and formula (15) indicates that the coordinate position of the construction personnel is in the down-track area.

[0054] Step S23, determine whether n is greater than z. If it is greater than z, there are still construction workers, and it is necessary to determine whether the construction workers have entered the dangerous area, and execute step S24; if it is less than z, end the determination.

[0055] Step S24: Based on the obtained electronic fence, determine y z > x z + Is it true, and at the same time let z accumulate 1, and return to step S23, wherein if it is true, execute step S2411, if not, execute step S2421; S2411, prove that the coordinates of the construction personnel are in the down track area; S2412. If the downlink track area is set as a dangerous area, the drone will shout a warning; S2421, prove that the coordinates of the construction personnel are in the up track area; S2422: If the uplink track area is set as a dangerous area, the drone will issue a warning.

[0056] The above method is applied to the track construction site, and can automatically generate an electronic fence without manual animation, and is applicable to all railway operation scenarios. Especially for the construction of one side of the track, it is easy to deploy, does not require external conditions, and does not require construction personnel to install electronic equipment. It only needs to move the drone, and finally the drone can sound an alarm to remind construction personnel in the dangerous area in time. In addition, by simultaneously identifying and obtaining the construction personnel and the location of the construction personnel, and real-time monitoring of whether the construction personnel enter the dangerous area, the above method improves the reliability of railway safety protection and the safety of construction.

[0057] like Fig.10As shown, in the embodiment of the present invention, a railway safety protection system based on a drone capable of executing the above method is also introduced, the system includes a drone, the drone includes a collection module, a processing module, a generation module and a monitoring module, wherein the collection module is used to obtain video stream data of the construction scene in real time based on the drone; the processing module is used to segment the railway in the acquired video stream data, extract the up track mask and the down track mask, and calculate the fitting straight line of the up track and the down track, and obtain the coordinate position of the construction personnel and each construction personnel in the video stream data; the generation module is used to automatically generate an electronic fence based on the fitting straight line of the up track and the down track; the monitoring module is used to set hyperparameters, based on the coordinate position of the construction personnel and the electronic fence, to determine whether the construction personnel have entered the dangerous area. The above system uses a drone to monitor the construction scene from a high altitude and generates an electronic fence in real time. The above system can be applied to any construction location beside the railway, without manually drawing the electronic fence, and also monitors whether the construction personnel have entered the dangerous area in real time. Finally, when the construction site is changed, it only needs to carry the drone to the designated construction site, without the need for external cooperation, which greatly improves the safety and reliability of railway safety protection.

[0058] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A railway safety protection method based on drones, characterized in that: include, Obtain video stream data of construction scenes in real time based on drones; Segment the railway in the acquired video stream data, extract the up track mask and the down track mask, calculate the fitting straight lines of the up track and the down track, and acquire the construction workers in the video stream data and the coordinate position of each construction worker; Automatically generate electronic fence based on the fitting straight line of the up track and down track; Set hyperparameters to determine whether construction workers have entered dangerous areas based on their coordinate positions and electronic fences.

2. The railway safety protection method based on drone according to claim 1 is characterized in that: Segment the railway in the acquired video stream data, extract the uplink track mask and the downlink track mask, including: The acquired video stream data is input into the semantic segmentation algorithm deeplabv3 model, and a black and white image is output, where the white part represents the railway; Based on the output black and white image, obtain the pixel coordinates of the railway; The density clustering algorithm is used to classify the obtained pixel coordinate positions of the railway, and the pixel coordinates of the up track and the down track are classified. The pixel coordinates of the up track constitute the up track mask, and the pixel coordinates of the down track constitute the down track mask.

3. The railway safety protection method based on drone according to claim 2 is characterized in that: The fitting straight lines for calculating the upward and downward orbits include, The least square method is used to perform straight line fitting on the pixel coordinates of each corresponding track to obtain the fitting straight lines of the uplink track and the downlink track.

4. The railway safety protection method based on drone according to claim 3 is characterized in that: The least square method is used to fit the pixel coordinates of each corresponding track into a straight line, including: Prepare data sets of pixel point coordinates of the uplink track and the downlink track respectively; Establish a fitted straight line model; Based on the fitted straight line model, the objective function of the fitted straight line is established; Based on the established fitting straight line objective function, the target equation group is obtained by differentiating the slope and intercept in the fitting straight line model; Substitute the data in the data set of pixel point coordinates of the uplink track and the downlink track into the target equation group to obtain the slope and intercept of the fitting line of the uplink track and the downlink track; The slope and intercept of the upward track fitting line and the downward track fitting line are respectively substituted into the fitting line model to obtain the upward track fitting line and the downward track fitting line.

5. The railway safety protection method based on drone according to claim 4 is characterized in that: Based on the fitting straight line of the up and down tracks, the electronic fence is automatically generated including: Determine the slope and intercept of the electronic fence, wherein the slope of the electronic fence is obtained according to a preset algorithm based on the slope of the fitting straight line of the upward track and the slope of the fitting straight line of the downward track, and the intercept of the electronic fence is obtained according to a preset algorithm based on the intercept of the fitting straight line of the upward track and the intercept of the fitting straight line of the downward track; Based on the determined slope and intercept of the electronic fence, a first straight line is formed, and the first straight line is the electronic fence.

6. The method for railway safety protection based on drone according to claim 1, characterized in that: Obtaining the construction workers in the video stream data and the coordinate position of each construction worker includes: The trained neural network-based target detection model is used to identify the construction workers in the video image, and the coordinate positions of the construction workers in the video image are obtained.

7. The method for railway safety protection based on drone according to claim 6 is characterized in that: The training process of the neural network-based target detection model includes: Collect construction scene video stream data at the construction site; Label the construction workers in all the collected video stream data and create a data set; The selected neural network model is iteratively trained using the data set, the selected neural network model is optimized, and finally the optimized neural network model is used as the neural network-based target detection model.

8. The method for railway safety protection based on drone according to any one of claims 1 to 7, characterized in that: Setting hyperparameters includes determining dangerous and safe areas of the railway based on the location of the construction work, including, Determine whether the construction work location is in the up track area or the down track area, where: If the construction work is located in the up track area, the down track area is the dangerous area and the up track area is the safe area; If the construction work location is in the down track area, the up track area is the dangerous area and the down track area is the safe area.

9. The method for railway safety protection based on drone according to claim 8, characterized in that: It also includes determining the track area where the construction personnel are located, including, The Y axis in the video image in the video stream data to be detected is preset to have the downward direction in the image as the positive direction; When the Y coordinate in any construction worker's coordinate position is smaller than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the up track area; When the Y coordinate in any construction worker's coordinate position is greater than the Y coordinate on the electronic fence corresponding to the corresponding horizontal coordinate, it means that the construction worker is in the down track area.

10. The method for railway safety protection based on drone according to claim 9, characterized in that: Based on the coordinate position of the construction workers and the electronic fence, it is judged whether the construction workers have entered the dangerous area, including: If the up-track area is a dangerous area, when the coordinate position of any construction worker is in the up-track area, it means that the construction worker has entered the dangerous area; If the down track area is a dangerous area, when the coordinate position of any construction worker is in the down track area, it means that the construction worker has entered the dangerous area; When construction workers enter a dangerous area, drones are used to issue warnings.

11. A railway safety protection system based on drones, characterized in that: Comprising a drone, the drone comprising, The acquisition module is used to obtain video stream data of the construction scene in real time based on the drone; A processing module, used for segmenting the railway in the acquired video stream data, extracting the up track mask and the down track mask, calculating the fitting straight lines of the up track and the down track, and acquiring the construction workers in the video stream data and the coordinate position of each construction worker; A generation module, used to automatically generate an electronic fence based on the fitted straight line of the up track and the down track; The monitoring module is used to set hyperparameters and determine whether construction workers have entered dangerous areas based on their coordinate positions and electronic fences.

Images