Work depot operation bidirectional early warning method and system, processing equipment and storage medium

Abstract

The invention relates to a two-way early warning method and system for work depot operation, processing equipment and a storage medium, and the method comprises the steps: arranging a vibration sensor on each track, and enabling the vibration sensors of all tracks in a certain region to form a corresponding vibration sensor array; the signals collected by the vibration sensors are processed, the vibration signals, collected by the vibration sensors, of the train passing through the track are extracted, and the track of the coming train is determined; a computer vision algorithm is adopted, based on the video monitoring data of the track of the coming train, the behavior state of a working worker in operation in the track is detected, and the behavior state is sent to a train center controller of the coming train; the train center controller of the coming train judges whether the working personnel in operation have potential threats or not according to the detected behavior state of the working personnel in operation in the track, if yes, early warning is carried out, and the method can be widely applied to the field of operation scenes of the railway service depot.

Description

Technical Field

[0001] The present invention relates to the field of railway maintenance section operation scenarios, and in particular to a two-way early warning method, system, processing equipment and storage medium for maintenance section operations. Background Art

[0002] Safety and reliability are of vital importance in modern rail transit systems. At present, accidental casualties occur frequently during construction and maintenance operations in railway maintenance sections. This is because the current maintenance operation protection scheme is relatively old and traditional, and the information transmission is lengthy. When the warning information is transmitted, the operators have no time to dodge. In addition, some areas have complex terrain and many continuous curves, which results in obstructed safety protection vision, high information omission rate, busy intercom channels, unstable transmission chain, and some operators illegally transfer to other sites and cross the line, which will pose a great threat to the personal safety of railway maintenance workers.

[0003] At present, the existing technology generally adopts manual operation, and carries out early warning through manual lookout method and crossing alarm method: However, the manual lookout method requires real-time communication with the site and notifies the site situation through walkie-talkies; the crossing alarm method requires laying cables, the equipment is complex and not easy to move, and the installation is cumbersome, which increases the workload and labor intensity of the staff. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a two-way early warning method, system, processing equipment and storage medium for engineering section operations, which can fully ensure the personal and property safety of operators and reduce workload.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions: In the first aspect, a two-way early warning method for engineering section operations is provided, comprising: A vibration sensor is arranged on each track, and the vibration sensors of all tracks in a certain area respectively constitute corresponding vibration sensor arrays; Process the signals collected by each vibration sensor, extract the vibration signals of the train passing the track collected by each vibration sensor, and determine the track of the oncoming train; Computer vision algorithms are used to detect the behavior of workers working on the track based on video surveillance data of the oncoming train's track, and the data is sent to the train control center of the oncoming train. The train control center of the oncoming train determines whether there is a potential threat to the workers working on the track based on the behavior status of the workers working on the track. If so, an early warning is issued, and a response feedback is sent to the train control center when the workers take risk avoidance actions.

[0006] Furthermore, the vibration sensors are arranged on each track, and the vibration sensors of all tracks in a certain area constitute a vibration sensor array, including: Arrange each vibration sensor at the railside of the corresponding track with the same d and r, where d is the width of the vibration sensor from the railside and r is the length of the vibration sensor from the track bifurcation.

[0007] Furthermore, the processing of the signals collected by each vibration sensor, extracting the vibration signals of the train passing the track collected by each vibration sensor, and determining the track of the oncoming train includes: A machine learning algorithm is used to extract the vibration signals of the train passing through the track collected by each vibration sensor; Performing Fourier transform on the vibration signals of the train passing the track collected by each vibration sensor, respectively, to obtain the vibration signals of the train passing the track after Fourier transform, as well as information in the time domain and frequency domain; The vibration signal characteristics of the train passing the track after Fourier transformation obtained by the vibration sensor array are combined into a vibration vector matrix; A machine learning algorithm is used to make vibration decisions based on the distance between the vibration sensor and the train as the distance boundary and the vibration threshold, and based on the corresponding vibration vector matrix, it is determined whether it is a train coming on a specific track.

[0008] Furthermore, the computer vision algorithm is used to detect the behavior status of the workers working on the track based on the video surveillance data of the track of the oncoming train, and sends it to the train control center of the oncoming train, including: Collect video surveillance data of the track of the oncoming train; Using target detection algorithms, workers working on the track can be identified based on video surveillance data of the track where the train is approaching; The spatiotemporal graph convolutional network model is used to perform human behavior recognition on the identified staff and obtain the behavioral status of the staff.

[0009] Furthermore, the construction process of the spatiotemporal graph convolutional network model is as follows: The human skeleton of the staff member is labeled according to key nodes to obtain a labeled skeleton diagram; The labeled skeleton image is used for algorithm training, the pixel information in the video surveillance data is analyzed, the key points and skeleton information of the human body are extracted, and the human body posture is modeled and recognized; Correspond skeleton actions to time series one by one to construct a one-to-one dataset; Based on the constructed data set, the spatiotemporal graph convolutional network model is trained to obtain a trained spatiotemporal graph convolutional network model, wherein the output of the spatiotemporal graph convolutional network model is the recognized behavior state.

[0010] Furthermore, the spatiotemporal graph convolutional network model is trained based on the constructed data set to obtain a trained spatiotemporal graph convolutional network model, including: The temporal attention mechanism and the spatial attention mechanism are introduced into the training of the spatiotemporal graph convolutional network model. The temporal attention mechanism and the spatial attention mechanism are used to capture the temporal relationship and spatial features between video frames.

[0011] Furthermore, the train control of the oncoming train determines whether there is a potential threat to the workers on the track according to the behavior of the workers on the track. If there is a potential threat, an early warning is issued. When the workers take risk avoidance actions, the train control sends a response feedback, including: The train control center of the approaching train determines whether there is a potential threat to the staff working on the track based on the behavior status of the staff working on the track and the behavior status of the pre-stored threat information; If there is a potential threat, the train central control will send the potential danger signal to the staff through the wearable device worn by the staff who are working; When the staff receives a potential danger signal and takes evasive action, they send response feedback to the oncoming train through a wearable device.

[0012] In the second aspect, a two-way early warning system for engineering section operations is provided, comprising: Vibration sensors are used to collect vibration signals from trains passing on corresponding tracks. The vibration sensors of all tracks in a certain area constitute corresponding vibration sensor arrays. A data acquisition module, used to process the signals collected by each of the vibration sensors, extract the vibration signals collected by each of the vibration sensors, and determine the track of the oncoming train; A behavior status detection module is used to detect the behavior status of workers working on the track based on the video surveillance data of the track of the oncoming train using a computer vision algorithm, and send the data to the train control center of the oncoming train; The train control center is used to determine whether there is a potential threat to the workers working on the track based on the behavioral status of the workers working on the track. If so, an early warning will be issued, and response feedback will be sent to the train control center when the workers take risk avoidance actions.

[0013] According to a third aspect, a processing device is provided, comprising computer program instructions, wherein the computer program instructions, when executed by the processing device, are used to implement the steps corresponding to the above-mentioned two-way early warning method for engineering section operations.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, are used to implement the steps corresponding to the above-mentioned two-way early warning method for maintenance section operations.

[0015] The present invention adopts the above technical solution, which has the following advantages: 1. The present invention integrates dynamic signals in both vehicle-to-person and person-to-vehicle directions, bidirectionally identifies the potential threat level to the workers who are working when the train approaches, and ensures that no danger information is missed. Vehicle-to-person means arranging vibration sensors on each track to transmit vibration signals; person-to-vehicle means using computer vision algorithms to detect the behavior information of the workers, and transmit the information to the train, and transmit and receive risk avoidance signals through the train central control and wearable devices worn by the workers.

[0016] 2. The present invention arranges the vibration sensor array around the rails and combines it with a machine learning algorithm to improve the accuracy of identifying the rails where trains pass, while also having a low cost.

[0017] 3. The present invention adopts the spatiotemporal graph convolutional network model for human behavior recognition, which can effectively judge the spatiotemporal dynamic information of the staff at work and form an effective information feedback between people and trains, so as to accurately obtain the behavior information of the staff and monitor whether they have the possibility of derailment and failure to avoid.

[0018] 4. The present invention takes both train and staff aspects into consideration, uses vibration sensors to obtain the effective position and track of the train, and uses computer vision algorithms to track the position and behavior information of the operating personnel, so as to judge the threat and transmit danger signals, so that the operating personnel can avoid the moving train in time, ensure their own safety, and avoid casualties in railway engineering operations. The present invention takes two aspects into consideration, performs two-way monitoring, and improves monitoring accuracy.

[0019] 5. The present invention can systematically provide a good working environment for operators, accurately predict the train position and timely convey warning information, reduce the occurrence of accidents, and eliminate the instability brought by traditional protection schemes.

[0020] In summary, the present invention can be widely used in the field of railway maintenance section operation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings: Figure 1It is a schematic diagram of the overall flow of the method provided by an embodiment of the present invention; Figure 2 is a schematic diagram of a train track array sensor provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of a human behavior recognition process provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of a warning information feedback process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0023] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0024] Although the terms first, second, third, etc. can be used in the text to describe multiple elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms can only be used to distinguish an element, component, region, layer or section from another region, layer or section. Unless the context clearly indicates, terms such as "first", "second" and other numerical terms do not imply order or sequence when used in the text. Therefore, the first element, component, region, layer or section discussed below can be referred to as the second element, component, region, layer or section without departing from the teaching of the example embodiments.

[0025] Due to the adverse external conditions of continuous curves and rugged terrain encountered during construction and maintenance work in railway engineering sections, the safety protection field of vision is limited. In addition, the current railway operation protection plan is outdated, the joint control links are complicated, the information transmission is lengthy, the walkie-talkie channels are busy, the transmission chain is unstable, and some working staff illegally transfer to other sites and cross the line, which poses a great threat to the personal safety of railway workers. In order to provide a safe and good external environment for operating personnel in a railway maintenance section, fully protect the personal and property safety of the operating personnel, improve the accuracy of danger alarms, control related costs, and reduce the investment of manpower and material resources, the present invention proposes a two-way early warning method for maintenance section operations, including: arranging vibration sensors on each track, and the vibration sensors of all tracks in a certain area respectively constitute corresponding vibration sensor arrays; processing the signals collected by each vibration sensor, extracting the vibration signals of the train passing through the track collected by each vibration sensor, and determining the track of the approaching train; using a computer vision algorithm, based on the video monitoring data of the track of the approaching train, detecting the behavior status of the staff working on the track, and sending it to the train central control of the approaching train; the train central control of the approaching train determines whether there is a potential threat to the staff working on the track according to the detected behavior status of the staff working on the track, and if so, issuing an early warning, and sending response feedback to the train central control after the staff takes risk avoidance behavior. The present invention uses low-cost vibration sensors and computer vision algorithms, uses vibration sensor arrays and machine learning to accurately collect train approach information, uses target detection and human behavior detection to collect dynamic information of working staff, and forms a warning information feedback transmission loop with the train. The present invention can accurately determine the train approach information, so that the staff can avoid it in time. At the same time, for the working staff and other illegal crossing personnel, the train can well obtain their action behavior information, and send out early warning signals in time, achieving the effect of fusion detection of train information and personnel information, which can not only effectively protect the personal safety of the staff, avoid unnecessary railway accidents, but also save a certain amount of manpower and material resources.

[0026] Example 1 like Figure 1 As shown, this embodiment provides a two-way early warning method for maintenance section operations, including the following steps: 1) Vibration sensors are arranged on each track, and the vibration sensors of all tracks in a certain area constitute corresponding vibration sensor arrays.

[0027] Specifically, the vibration sensor is arranged on one side of the corresponding track.

[0028] More specifically, since it is difficult to identify which track a train is about to travel on at some track switches by computer vision methods alone, and vibration sensors have the advantages of low cost, high sensitivity, stable and reliable operation, they are widely used as detection alarm sensors. However, simply arranging sensors alone cannot effectively determine the travel status of a train before and after changing tracks. Therefore, the present invention arranges vibration sensors on one side of the track to sense the upcoming train, such as Figure 2 As shown, d is the width of the vibration sensor from the rail edge, r is the length of the vibration sensor from the track bifurcation, and each vibration sensor is arranged at the rail edge of the corresponding track according to the same d and r.

[0029] 2) Process the signals collected by each vibration sensor, extract the vibration signals of the train passing the track collected by each vibration sensor, and determine the track of the oncoming train, specifically: 2.1) Extract the vibration signals of the train passing through the track collected by each vibration sensor.

[0030] Specifically, since the vibration sensor automatically collects multiple sounds when collecting sound, such as the sound of trains passing over tracks, the sound of workers working, the sound of cars, and the sound of other foreign objects such as falling rocks, the present invention uses a machine learning algorithm for sound training to extract the vibration signal of the train passing over the rails, which is convenient for subsequent processing and is not interfered by other sounds.

[0031] 2.2) Perform Fourier transform on the vibration signals of the train passing through the track collected by each vibration sensor, and obtain the vibration signals of the train passing through the track after Fourier transform, as well as the information in the time domain and frequency domain, to obtain more features. The Fourier transform formula is: (1) in, is the transformation function; is the frequency; For time.

[0032] 2.3) The vibration signal characteristics of the train passing through the track after Fourier transformation obtained by the vibration sensor array are combined into a vibration vector matrix. Figure 2 Take the five vibration sensors in as an example: (2) in, For time The matrix result is output below; , , , , for Figure 2 The vibration signals transmitted by the 5 vibration sensors in the instrument.

[0033] 2.4) Using machine learning algorithms, the distance between the vibration sensor and the train is used as the distance limit and the distance between the vibration sensor and the train is used as the distance limit. A vibration decision is made for the vibration threshold, and based on the corresponding vibration vector matrix, it is determined whether a train is coming on a specific track.

[0034] Specifically, when a train is running on a certain track and the distance between it and the vibration sensor on the track is L, if the vibration signal collected by a certain vibration sensor exceeds the vibration threshold , then the vibration vector matrix of the vibration sensor array where the vibration sensor is located is collected, and the machine learning algorithm is used for training. The SVM classifier is used in this process, which is expressed as follows: The hyperplane is in the form of a vector: (3) Recorded as in the form of is the normal vector, is the displacement term, The vibration signal collected by each vibration sensor.

[0035] The decision function is: (4) This function is called a linearly separable support vector machine.

[0036] Whenever a train comes on a certain track, the vibration signals of all vibration sensors are collected once. By collecting a large number of samples and conducting training, it is possible to accurately detect which track the train is coming on.

[0037] This method can still accurately determine which track it is when dealing with a scene where two tracks are close to each other. The SVM classifier has a strong generalization ability for unknown data and still performs better than other traditional algorithms even when the amount of data is small.

[0038] 3) Using computer vision algorithms, based on video surveillance data of the track of the oncoming train, the behavior status of the workers working on the track is detected and sent to the train control center of the oncoming train to determine whether they are threatening to the moving train. Specifically: 3.1) Collect video surveillance data of the track where the train is approaching.

[0039] 3.2) Use target detection algorithms to identify workers working on the track based on video surveillance data of the track where the train is approaching.

[0040] 3.3) The spatiotemporal graph convolutional network model (ST-GCN) is used to perform human behavior recognition on the identified staff and obtain the behavioral status of the staff.

[0041] Specifically, Figure 3 As shown in Figure 2, the construction process of the spatiotemporal graph convolutional network model is as follows: a) Joint point labeling: Label the human skeleton of the staff member according to key nodes, such as head, shoulders, elbows, hands, hips, knees and feet, to obtain a labeled skeleton diagram.

[0042] b) Human posture behavior recognition and training: Use the labeled skeleton image to train the algorithm, analyze the pixel information in the video surveillance data, extract the key points and skeleton information of the human body, and model and recognize the human posture.

[0043] c) Construct a dataset corresponding to actions and skeleton graph time series: Correspond the skeleton actions to the time series one by one, construct a one-to-one dataset, and match and organize different skeleton actions with the corresponding time series.

[0044] d) Based on the constructed data set, the spatiotemporal graph convolutional network model is trained to obtain a trained spatiotemporal graph convolutional network model, wherein the output of the spatiotemporal graph convolutional network model is the recognized behavior state.

[0045] Specifically, the present invention introduces the temporal attention mechanism and the spatial attention mechanism into the training of the spatiotemporal graph convolutional network model, which can effectively improve the recognition accuracy. The temporal attention mechanism can capture the relationship and dynamic changes of the human body joint conditions in different time periods, so as to more accurately identify the action; the spatial attention mechanism can locate the motion area and focus on the discriminative non-motion area, which helps the spatiotemporal graph convolutional network model to accurately extract the key areas related to action recognition in complex backgrounds and reduce noise interference. The spatiotemporal attention mechanism combines the attention in the two dimensions of space and time, so that the spatiotemporal graph convolutional network model can simultaneously capture the spatial structure and temporal dynamics of the action. This comprehensive spatiotemporal feature representation helps the spatiotemporal graph convolutional network model to understand the action more comprehensively and improve the robustness and stability of recognition.

[0046] Specifically, in video action recognition, the temporal attention mechanism and the spatial attention mechanism are used to capture the temporal relationship and spatial features between video frames. The basic formula is: (5) in, For inquiries; is the key; is the value; is the dimension of the key; This is the mathematical notation for "transpose".

[0047] Specifically, ST-GCN is a general model for graph-based dynamic skeleton modeling, which can realize dual modeling of space and time. The model is built on a series of skeleton graphs, in which each node corresponds to a joint point of the human body, including two types of edges, namely spatial edges and temporal edges. Spatial edges are edges that are naturally connected in accordance with the joint points; temporal edges are time edges that connect the same nodes across consecutive time steps, thus realizing the integration of information along the graph dimension and the time dimension.

[0048] Specifically, the recognition of the behavior state includes detecting the posture and action of the staff, such as sitting, squatting, lying, and knocking on the rails and other possible actions. Through the algorithm of this step, the posture and action of the human body in the video frame of the video data can be recognized, and then the potential dangerous behavior of the human body in the video data can be recognized.

[0049] 4) If Figure 4 As shown, the train control of the approaching train determines whether there is a potential threat to the workers on the track based on the behavior of the workers working on the track. If there is, an early warning is issued. When the workers take risk avoidance actions, the train control sends a response feedback to inform the train that it can continue to move forward as planned. Specifically: 4.1) The central control of the oncoming train determines whether the working staff has potential threats based on the detected behavior status of the working staff on the track and the behavior status of the pre-stored threat information (such as sitting, running, lying, crossing the line, not wearing a helmet, etc.), that is, when the detected behavior status is one of the pre-stored behavior status of the threat information, it is determined that the working staff has potential threats.

[0050] 4.2) If there is a potential threat, execute Figure 4 Step ① in the above is that the train central control sends potential danger signals to the workers through the wearable devices worn by the workers who are working, so as to warn the workers to leave the location quickly to ensure their own safety.

[0051] 4.3) Execution Figure 4 In step ②, when the staff receives the potential danger signal and takes risk avoidance action, they can send response feedback to the oncoming train through the wearable device, giving the train a safety signal in time to inform the train that it can continue to move forward as planned. This is a response feedback transmission operation between the train and the staff.

[0052] Example 2 This embodiment provides a two-way early warning system for maintenance section operations, including: Vibration sensors are used to collect vibration signals from trains passing on corresponding tracks. The vibration sensors of all tracks in a certain area constitute corresponding vibration sensor arrays. A data acquisition module, used to process the signals collected by each of the vibration sensors, extract the vibration signals collected by each of the vibration sensors, and determine the track of the oncoming train; A behavior status detection module is used to detect the behavior status of workers working on the track based on the video surveillance data of the track of the oncoming train using a computer vision algorithm, and send the data to the train control center of the oncoming train; The train control center is used to determine whether there is a potential threat to the workers working on the track based on the behavioral status of the workers working on the track. If so, an early warning will be issued, and response feedback will be sent to the train control center when the workers take risk avoidance actions.

[0053] The system provided in this embodiment is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for specific processes and detailed contents, which will not be repeated here.

[0054] Example 3 This embodiment provides a processing device corresponding to the two-way warning method for engineering section operations provided in Embodiment 1. The processing device can be applicable to a client processing device, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of Embodiment 1.

[0055] The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The memory stores a computer program that can be run on the processing device, and when the processing device runs the computer program, the bidirectional early warning method for the maintenance section operation provided in this embodiment 1 is executed.

[0056] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage.

[0057] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0058] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0059] Those skilled in the art will understand that the structure of the above-mentioned computing device is only a partial structure related to the solution of the present invention, and does not constitute a limitation on the computing device to which the solution of the present invention is applied. The specific computing device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0060] Example 4 This embodiment provides a computer program product corresponding to the two-way early warning method for maintenance section operations provided in Embodiment 1. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing the two-way early warning method for maintenance section operations described in Embodiment 1 are loaded.

[0061] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0062] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0063] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0066] The above embodiments are only used to illustrate the present invention, wherein the structure, connection mode and manufacturing process of each component may be changed. Any equivalent transformations and improvements based on the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A two-way early warning method for engineering section operations, characterized in that: include: A vibration sensor is arranged on each track, and the vibration sensors of all tracks in a certain area respectively constitute corresponding vibration sensor arrays; Process the signals collected by each vibration sensor, extract the vibration signals of the train passing the track collected by each vibration sensor, and determine the track of the oncoming train; Computer vision algorithms are used to detect the behavior of workers working on the track based on video surveillance data of the oncoming train's track, and the data is sent to the train control center of the oncoming train. The train control center of the oncoming train determines whether there is a potential threat to the workers working on the track based on the behavior status of the workers working on the track. If so, an early warning is issued, and a response feedback is sent to the train control center when the workers take risk avoidance actions.

2. A two-way early warning method for engineering section operations as claimed in claim 1, characterized in that: The vibration sensors are arranged on each track, and the vibration sensors of all tracks in a certain area form a vibration sensor array, including: Arrange each vibration sensor at the railside of the corresponding track with the same d and r, where d is the width of the vibration sensor from the railside and r is the length of the vibration sensor from the track bifurcation.

3. A two-way early warning method for engineering section operations as claimed in claim 1, characterized in that: The processing of the signals collected by each vibration sensor, extracting the vibration signals of the train passing the track collected by each vibration sensor, and determining the track of the oncoming train includes: A machine learning algorithm is used to extract the vibration signals of the train passing through the track collected by each vibration sensor; Performing Fourier transform on the vibration signals of the train passing the track collected by each vibration sensor, respectively, to obtain the vibration signals of the train passing the track after Fourier transform, as well as information in the time domain and frequency domain; The vibration signal characteristics of the train passing the track after Fourier transformation obtained by the vibration sensor array are combined into a vibration vector matrix; A machine learning algorithm is used to make vibration decisions based on the distance between the vibration sensor and the train as the distance boundary and the vibration threshold, and based on the corresponding vibration vector matrix, it is determined whether it is a train coming on a specific track.

4. A two-way early warning method for engineering section operations as claimed in claim 1, characterized in that: The computer vision algorithm is used to detect the behavior status of the workers working on the track based on the video monitoring data of the track of the oncoming train, and sends it to the train control of the oncoming train, including: Collect video surveillance data of the track of the oncoming train; Using target detection algorithms, workers working on the track can be identified based on video surveillance data of the track where the train is approaching; The spatiotemporal graph convolutional network model is used to perform human behavior recognition on the identified staff and obtain the behavioral status of the staff.

5. A two-way early warning method for engineering section operations as claimed in claim 4, characterized in that: The construction process of the spatiotemporal graph convolutional network model is as follows: The human skeleton of the staff member is labeled according to key nodes to obtain a labeled skeleton diagram; The labeled skeleton image is used for algorithm training, the pixel information in the video surveillance data is analyzed, the key points and skeleton information of the human body are extracted, and the human body posture is modeled and recognized; Correspond skeleton actions to time series one by one to construct a one-to-one dataset; Based on the constructed data set, the spatiotemporal graph convolutional network model is trained to obtain a trained spatiotemporal graph convolutional network model, wherein the output of the spatiotemporal graph convolutional network model is the recognized behavior state.

6. A two-way early warning method for engineering section operations as claimed in claim 5, characterized in that: The spatiotemporal graph convolutional network model is trained based on the constructed data set to obtain a trained spatiotemporal graph convolutional network model, including: The temporal attention mechanism and the spatial attention mechanism are introduced into the training of the spatiotemporal graph convolutional network model. The temporal attention mechanism and the spatial attention mechanism are used to capture the temporal relationship and spatial features between video frames.

7. A two-way early warning method for engineering section operations as claimed in claim 1, characterized in that: The train control center of the oncoming train determines whether there is a potential threat to the workers on the track according to the behavior of the workers on the track. If there is, an early warning is issued. When the workers take risk avoidance actions, the train control center sends a response feedback, including: The train control center of the approaching train determines whether there is a potential threat to the staff working on the track based on the behavior status of the staff working on the track and the behavior status of the pre-stored threat information; If there is a potential threat, the train central control will send the potential danger signal to the staff through the wearable device worn by the staff who are working; When the staff receives a potential danger signal and takes evasive action, they send response feedback to the oncoming train through a wearable device.

8. A two-way early warning system for engineering section operations, characterized in that: include: Vibration sensors are used to collect vibration signals from trains passing on corresponding tracks. The vibration sensors of all tracks in a certain area constitute corresponding vibration sensor arrays. A data acquisition module, used to process the signals collected by each of the vibration sensors, extract the vibration signals collected by each of the vibration sensors, and determine the track of the oncoming train; A behavior status detection module is used to detect the behavior status of workers working on the track based on the video surveillance data of the track of the oncoming train using a computer vision algorithm, and send the data to the train control center of the oncoming train; The train control center is used to determine whether there is a potential threat to the workers working on the track based on the behavioral status of the workers working on the track. If so, an early warning will be issued, and response feedback will be sent to the train control center when the workers take risk avoidance actions.

9. A processing device, characterized in that: It includes computer program instructions, wherein the computer program instructions, when executed by a processing device, are used to implement the steps corresponding to the two-way early warning method for engineering section operations described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, wherein the computer program instructions, when executed by a processor, are used to implement the steps corresponding to the bidirectional early warning method for engineering section operations described in any one of claims 1-7.

Images