Railway safety early warning method and device based on communication base station tower and readable medium

Abstract

The invention discloses a railway safety early warning method and device based on a communication base station pole tower and a readable medium, and relates to the field of video processing, a radar, camera shooting equipment and alarm equipment which are arranged on the communication base station pole tower along a railway are adopted, and the method comprises the steps that when the radar monitors that a target enters a monitoring range, the camera shooting equipment is started; if yes, acquiring radar monitoring data and real-time video data acquired by camera equipment, inputting the real-time video data into the target detection model to obtain the position of each target and the confidence corresponding to the position, and determining the motion displacement of the target; inputting the confidence coefficient and the target motion displacement of each target corresponding to the radar monitoring data and the real-time video data into a trained spatio-temporal correlation degree prediction model to obtain a spatio-temporal correlation degree; and if the spatio-temporal correlation degree is greater than a threshold value, generating an event, a video recording instruction and an alarm instruction so as to carry out video recording and send out an alarm signal. The problem that it is difficult to accurately carry out early warning in advance before the train arrives is solved.

Description

Technical Field

[0001] This invention relates to the field of video processing, and specifically to a railway safety early warning method, device, and readable medium based on communication base station towers. Background Technology

[0002] In some areas around railways, the lack of enclosed fencing and the scarcity of culverts have led to safety incidents such as train stoppages, injuries, and even fatalities due to pedestrians and animals encroaching on the tracks. Currently, only a small number of sections have low-level surveillance, while the vast majority lack video surveillance and "intelligent monitoring" functions. The long and complex railway lines within the country make it difficult to deploy a sufficient number of railway maintenance personnel, thus limiting their effectiveness. Currently, there are stand-alone train arrival and departure monitoring poles that integrate radar and cameras, but these can only monitor people near the platform when trains are entering or leaving the station. The railway maintenance alarm time is too short to provide advance notice of an impending train arrival. Some railways have ordinary surveillance cameras nearby, but these have low alarm accuracy and are easily affected by environmental factors, making them difficult to function properly. Furthermore, early warning information cannot form a closed loop and be shared in a timely manner, and it is impossible to collect evidence or educate individuals who frequently linger near the railway. Summary of the Invention

[0003] The purpose of this application is to propose a railway safety early warning method, device, and readable medium based on communication base station towers to address the aforementioned technical problems.

[0004] In a first aspect, the present invention provides a railway safety early warning method based on communication base station towers, employing radar, camera equipment, and alarm equipment installed on communication base station towers along the railway line, comprising the following steps:

[0005] Obtain preset values ​​for radar parameters, construct a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters.

[0006] The system acquires echo signals from a radar operating at the optimal radar parameters at a preset distance from the monitoring point where the camera is located. In response to a target entering the monitoring range based on the echo signal, it acquires radar monitoring data and real-time video data from the camera. The real-time video data is input into the target detection model to obtain the position of each target and its corresponding confidence level. The target displacement is determined based on the position of each target in the real-time video data. The confidence levels and displacements of each target corresponding to the radar monitoring data and real-time video data are then input into a trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation.

[0007] In response to the determination that the spatiotemporal correlation is greater than a threshold, an event, a video recording command, and an alarm command are generated. The video recording command is sent to the camera device and controls it to record video. The alarm command is sent to the alarm device on the communication base station tower at the monitoring point where the camera device is located and controls it to issue an alarm signal.

[0008] Preferably, the radar parameters include maximum detection speed, continuous value, minimum detection speed, vertical deflection angle, sensitivity, horizontal deflection angle, energy threshold, distance limit setting, and detection direction. The radar monitoring data includes the distance between the radar and the monitoring point where the camera equipment is located, the target's moving speed, sensitivity, detection direction, and angle, including vertical deflection angle and horizontal deflection angle. The time point when the target detected by the radar enters the monitoring range is taken as the starting heartbeat time point, and the time point when the target detected by the radar leaves the monitoring range is taken as the ending heartbeat time point. Within the time range from the starting heartbeat time point to the ending heartbeat time point, the radar is controlled to emit transmission signals at preset time intervals, and radar monitoring data and real-time video data are collected. The video recording time starts from the ending heartbeat time point.

[0009] Preferably, the target detection model is obtained by fine-tuning the pre-trained target detection model using video data collected by camera equipment in historical events; the spatiotemporal correlation prediction model includes an LSTM model, and during the training process of the spatiotemporal correlation prediction model, a Bayesian optimization algorithm is used to iteratively optimize the parameters of the LSTM model and the radar parameters until the optimal parameters of the LSTM model and the optimal values ​​of the radar parameters are obtained.

[0010] Preferably, the target detection pre-trained model includes a pre-trained YOLO model, and the detected targets include trains, humans, or animals; the radar includes millimeter-wave radar; the alarm device includes a horn; and the alarm signal includes an alarm sound emitted by the horn.

[0011] As a preferred embodiment, the method further includes: assigning a target number to the same target in different video frames in real-time video data, constructing a continuous trajectory by combining the target's coordinates in the video frame with the corresponding time point, and determining whether the target is in a lingering state based on the continuous trajectory.

[0012] Preferably, the method also includes: in response to determining that the spatiotemporal correlation between the radar and the video is less than or equal to a threshold, then not generating events, video recording instructions, and alarm instructions.

[0013] Secondly, the present invention provides a railway safety early warning device based on communication base station towers, employing radar, camera equipment, and alarm equipment installed on communication base station towers along the railway line, including:

[0014] The spatiotemporal correlation prediction model building module is configured to obtain preset values ​​of radar parameters, build a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters.

[0015] The prediction module is configured to acquire echo signals collected by a radar operating at the optimal values ​​of radar parameters at a preset distance from the monitoring point where the camera is located. In response to detecting a target entering the monitoring range based on the echo signal, it acquires radar monitoring data and real-time video data collected by the camera. The real-time video data is input into the target detection model to obtain the position of each target and its corresponding confidence level. The target motion displacement is determined based on the position of each target in the real-time video data. The confidence level and target motion displacement of each target corresponding to the radar monitoring data and real-time video data are input into a trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation.

[0016] The early warning module is configured to generate an event, a video recording command, and an alarm command in response to a determination that the spatiotemporal correlation is greater than a threshold. The video recording command is sent to the camera device and controls it to record video. The alarm command is sent to the alarm device on the communication base station tower at the monitoring point where the camera device is located and controls it to issue an alarm signal.

[0017] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0019] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) The railway safety early warning method based on communication base station tower proposed in this invention constructs a spatiotemporal correlation prediction model to characterize the target matching degree of radar and video monitoring. When radar detects that a target has entered the monitoring area, the real-time video data collected by the camera equipment at the monitoring point can be input into the target detection model for target identification, obtain the position of each target and its corresponding confidence, and confirm the target movement displacement. The collected radar monitoring data, the confidence of each target and the target movement displacement are input into the spatiotemporal correlation prediction model, and the spatiotemporal correlation can be predicted. The spatiotemporal correlation determines whether an alarm is triggered and collects the recorded video data for the corresponding time period so as to promptly drive away people located near the railway and use the video of people staying near the railway as the basis for warning education.

[0022] (2) The railway safety early warning method based on communication base station towers proposed in this invention can simultaneously adjust radar parameters while training the spatiotemporal correlation prediction model, thus avoiding the problem that the radar cannot accurately monitor the entry and exit of trains due to radar parameter setting issues, resulting in low early warning accuracy.

[0023] (3) The railway safety early warning method based on communication base station towers proposed in this invention sets up radar, camera equipment and alarm equipment on communication base station towers along the railway line. It utilizes the powerful communication and power resources on the communication base station towers to achieve resource reuse and avoid repeated investment. The communication base station towers have wide coverage along the railway line. At the same time, the communication signals provided by the communication base station towers have high-density coverage. The power supply and supporting facilities make the overall system more stable. By using ordinary radar combined with monitoring video, it is possible to accurately predict whether a train is about to pass and issue an early warning. The equipment cost is low. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a railway safety early warning method based on communication base station towers, as an embodiment of this application.

[0026] Figure 2 This is a schematic diagram showing the radar, camera equipment, and alarm equipment installed on a communication base station tower in an embodiment of the railway safety early warning method based on a communication base station tower according to this application.

[0027] Figure 3The screenshot shows a person located near the railway, captured by a video recording of the railway safety early warning method based on a communication base station tower, as an embodiment of this application.

[0028] Figure 4 This is a schematic diagram of a railway safety early warning device based on a communication base station tower, as an embodiment of this application.

[0029] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0031] Figure 1 This application illustrates an embodiment of a railway safety early warning method based on communication base station towers. The method employs radar, camera equipment, and alarm equipment installed on communication base station towers along the railway line, and includes the following steps:

[0032] S1. Obtain the preset values ​​of radar parameters, construct a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters.

[0033] In a specific embodiment, the target detection model is obtained by fine-tuning the target detection pre-trained model using video data collected by camera equipment in historical events; the spatiotemporal correlation prediction model includes an LSTM model, and during the training process of the spatiotemporal correlation prediction model, a Bayesian optimization algorithm is used to iteratively optimize the parameters of the LSTM model and the radar parameters until the optimal parameters of the LSTM model and the optimal values ​​of the radar parameters are obtained.

[0034] In a specific embodiment, the target detection pre-training model includes a pre-trained YOLO model, and the detected targets include trains, humans, or animals; the radar includes millimeter-wave radar, the alarm device includes a horn, and the alarm signal includes an alarm sound emitted by the horn.

[0035] Specifically, in the embodiments of this application, the target detection pre-training model can be a pre-trained YOLO model. The pre-trained YOLO model has a preliminary target recognition capability. Further fine-tuning is performed on the pre-trained YOLO model using recorded video data from historical events. Finally, the fine-tuned target detection model can accurately identify trains, humans, or animals appearing near the railway.

[0036] Specifically, the radar, camera equipment, and alarm equipment used in the embodiments of this application can all be installed on communication base station towers along the railway line. In one example, refer to... Figure 2 Camera device 2 and alarm device 3 can be installed on the same communication base station tower near a railway line, such as near a residential area. Radar 1 can be installed on two communication base station towers at a certain distance before and after it to monitor whether trains are passing near the railway line. In one example, the alarm device can be a loudspeaker, and the radar can be a millimeter-wave radar. The camera, radar, loudspeaker, and other IoT devices are mounted on the communication base station tower. A communication protocol between the IoT devices and the FSU (Field Monitoring Unit) of the communication base station is developed, and data transmission is performed using the existing FSU equipment on the tower, improving the device sharing rate. In another example, a millimeter-wave radar is deployed at a location 1 kilometer away from the railway intersection in a rural area, in the direction of oncoming traffic, or at a custom distance, to monitor whether a train is about to enter the station. A high-volume camera and loudspeaker are set up at the railway intersection in the rural area, linked with the radar 1 kilometer away. Once the radar detects a train, the loudspeaker will issue a voice broadcast, providing safety warnings and eviction reminders, giving people time to leave the railway and ensuring the safety of people, vehicles, and property along the railway line. The millimeter-wave radar used in the embodiments of this application can adapt to complex environments. Under extreme conditions such as rain, snow, fog, haze, or night, the radar can still work stably to ensure that no early warning is missed. The camera equipment at the monitoring point is mounted on the support arm of the communication base station tower along the railway line. Therefore, the camera equipment reaches a certain height, making the monitoring range larger and the early warning effect better.

[0037] The advantages of installing sensing equipment such as radar and cameras on communication base station towers along railway lines are mainly reflected in the following aspects:

[0038] (1) Advantages of resource reuse: Reduce construction and operation and maintenance costs. The existing communication base station towers along the railway line can be used as carriers for the installation of sensing equipment without the need for additional planning of pole locations, which greatly shortens the construction period. The stable 220V AC power supply (dual-circuit backup) of the communication base station can directly power the sensing equipment, avoiding the safety hazards and construction costs of field power connection; at the same time, relying on the lightning protection grounding system of the communication base station (grounding resistance ≤10Ω), the lightning failure rate of the equipment can be reduced to below 0.1%, which is far superior to the protection level of similar products on the market.

[0039] (2) Advantages of full coverage: Solving the "blind spot problem" of railway protection, with no dead angles in spatial coverage: Communication base station towers generally have the advantage of a height of 15-30 meters. The camera equipment (such as high-definition infrared cameras with an effective monitoring distance of 150 meters) and radar (millimeter-wave radar with a detection radius of 200 meters) can achieve cross coverage. It can be further combined with AR panoramic stitching technology to eliminate traditional monitoring blind spots such as under bridges, tunnel entrances, and culverts. For complex areas around railway stations, the dense distribution of communication base station towers (average spacing of 300-500 meters) can form a high-density sensing network to accurately capture hidden dangers such as personnel gathering and vehicle encroachment. High reliability of signal transmission: Relying on the 5G / 4G dedicated link of the communication base station itself, the sensing data transmission rate can reach more than 100Mbps, and the end-to-end delay is ≤50ms, ensuring the real-time push of early warning information. Compared with traditional wireless transmission solutions, the signal coverage rate is increased to 99.9%, completely solving the problem of data transmission interruption in remote mountainous areas and bridge sections.

[0040] (3) System stability advantages: Adaptable to the complex operating environment of railways, with strong hardware environment adaptability: Based on the standardized installation interface of communication base station towers, the integrated sensing equipment adopts a high-level protection design (IP67 dustproof and waterproof, wide temperature range of -40℃ to 70℃), which can withstand extreme weather such as rainstorms, snowstorms, and strong winds. Combined with the constant temperature cabinet of the communication base station, the stability is improved by 40% compared with ordinary products on the market.

[0041] (4) Zero power interruption guarantee: The UPS uninterruptible power supply equipped in the communication base station can provide power for more than 4 hours after the mains power is interrupted. Combined with the backup power of the diesel generator, the sensing equipment can operate "without interruption". Compared with the solution that relies on solar power, it completely solves the problem of monitoring interruption caused by insufficient power supply on rainy days.

[0042] Furthermore, a spatiotemporal correlation prediction model based on an LSTM model is constructed and trained. The training process of this spatiotemporal correlation prediction model includes:

[0043] First, set the preset values ​​for the radar parameters as follows:

[0044] 1. Maximum detection speed, in km / h, set to 400;

[0045] 2. Continuous value, unit setting, set value to 1;

[0046] 3. Minimum detection speed, in km / h, set to 10;

[0047] 4. Vertical deflection angle, in degrees, set to 0;

[0048] 5. Sensitivity, unit: levels, setting value: 20;

[0049] 6. Horizontal deflection angle, in degrees, set to 0;

[0050] 7. Energy threshold, unit is level, set value is 40;

[0051] 8. Distance limit setting, unit is distance, set value is 0;

[0052] 9. Detection direction (0 indicates bidirectional, 1 indicates incoming, 2 indicates outgoing), set to 1.

[0053] Training data, including radar monitoring data, video data, and their corresponding spatiotemporal correlation values, is collected using camera equipment. The process is as follows: First, radar echo signals are collected when radar parameters are preset. When a target is detected entering the monitoring range based on the echo signal, radar monitoring data and video data simultaneously collected by the camera equipment are acquired. The detection of a target entering or leaving the monitoring area via radar echo signals is defined as a radar heartbeat event. Simultaneously, a train passing event is defined as the confidence level of the simultaneously collected video data exceeding a threshold. The ratio of the number of train passing events to the number of collected radar heartbeat events when radar parameters are preset is used as the true value of the spatiotemporal correlation. Training data is constructed by combining this data with the corresponding radar monitoring data and video data.

[0054] The video data from the training data is input into the target detection model to obtain the position of each target in the video data and its corresponding confidence score, and the target motion displacement is detected based on the position of each target in the video data. The radar monitoring data from the training data, along with the confidence scores and target motion displacements of each target corresponding to the video data, are input into the spatiotemporal correlation prediction model to predict the spatiotemporal correlation. A loss function is constructed based on the true value of the spatiotemporal correlation and the spatiotemporal correlation predicted by the model, and the spatiotemporal correlation prediction model is trained. During the training process, the accuracy of the spatiotemporal correlation prediction model is evaluated, and the radar parameters and the parameters of the spatiotemporal correlation prediction model are adjusted using an optimization algorithm. In the embodiments of this application, the optimization algorithm adopts the Bayesian optimization algorithm. The main principle of the Bayesian optimization algorithm is: to model the relationship between the parameters and the objective function (such as target accuracy) through a probabilistic model (prior), and to iteratively update the radar parameters and the parameters of the spatiotemporal correlation prediction model to the optimal value. Its mathematical framework revolves around a closed loop of "probabilistic modeling-acquisition-update," and the parameters are adjusted to the optimal value after optimization. The specific steps of the Bayesian optimization algorithm are not described in detail here. After the above optimization, not only are the optimal parameters of the spatiotemporal correlation prediction model obtained, but the optimal values ​​of the radar parameters are also determined, avoiding inaccurate spatiotemporal correlation predictions due to radar parameter settings. After obtaining the optimal parameters of the spatiotemporal correlation prediction model, a trained spatiotemporal correlation prediction model is obtained. Using the spatiotemporal correlation predicted by this model, the spatiotemporal correlation between targets detected by the radar and targets detected by the camera equipment can be measured. Therefore, this spatiotemporal correlation can be used to determine whether to issue a warning and record video. When the radar detects a target such as a train entering the monitoring area, if the video data collected by the camera equipment at a certain distance detects the train, it indicates a high spatiotemporal correlation, requiring a warning and recording. By learning from prior knowledge, the spatiotemporal correlation prediction model gains the ability to predict in advance whether a train will enter.

[0055] S2: Obtain the echo signal collected by the radar at a preset distance from the monitoring point where the camera is located and operating at the optimal value of the radar parameters. In response to the detection of a target entering the monitoring range based on the echo signal, obtain the radar monitoring data and the real-time video data collected by the camera. Input the real-time video data into the target detection model to obtain the position of each target and its corresponding confidence level. Determine the target motion displacement based on the position of each target in the real-time video data. Input the confidence level and target motion displacement of each target corresponding to the radar monitoring data and the real-time video data into the trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation.

[0056] In a specific embodiment, radar parameters include maximum detection speed, continuous value, minimum detection speed, vertical deflection angle, sensitivity, horizontal deflection angle, energy threshold, distance limit setting, and detection direction. Radar monitoring data includes the distance between the radar and the monitoring point where the camera is located, the target's moving speed, sensitivity, detection direction, and angles, including vertical and horizontal deflection angles. The time when the target detected by the radar enters the monitoring range is taken as the starting heartbeat time point, and the time when the target detected by the radar leaves the monitoring range is taken as the ending heartbeat time point. Radar monitoring data and real-time video data are collected within the time range from the starting heartbeat time point to the ending heartbeat time point. Video recording begins from the ending heartbeat time point. At this time, the train leaves the monitoring area corresponding to the radar and is about to enter the monitoring point where the camera is located.

[0057] Specifically, the target detection model and the trained spatiotemporal correlation prediction model are deployed, and in actual use, the radar is operated at the optimal value of the radar parameters, which can more accurately combine with the video data collected by the camera equipment to predict the spatiotemporal correlation. During the monitoring process, the train will first pass through the radar's monitoring area in its direction of travel and then through the monitoring point where the video is located. The radar first transmits a signal and receives echo signals. When the echo signal detects that a target has entered the monitoring area, the corresponding time point is recorded as the starting heartbeat time point. When the echo signal detects that the target has left the monitoring range, the corresponding time point is recorded as the ending heartbeat time point. This constitutes a radar heartbeat event, and radar monitoring data is acquired. The radar monitoring data includes the distance between the radar and the monitoring point where the camera is located, the target's moving speed, sensitivity, detection direction, and angle. At the same time, the real-time video data collected at the starting heartbeat time point is correlated and analyzed. Each video frame of the real-time video data is input into the target detection model to identify the position of the target in each video frame and its corresponding confidence level. A unique ID is assigned to the same target in different video frames. The target's displacement can be obtained by the positional change of the same target between different video frames. Here, the target can refer to a train and / or a human. By inputting radar monitoring data, target confidence level, and target motion displacement into a trained spatiotemporal correlation prediction model, the spatiotemporal correlation degree can be obtained, ranging from 0 to 1. This trained model can accurately extract and analyze features from the time-series radar monitoring data, target confidence level, and target motion displacement to characterize the target matching degree between radar and video surveillance, thereby improving alarm recognition accuracy.

[0058] S3, in response to the determination that the spatiotemporal correlation is greater than the threshold, generates an event, a video recording command and an alarm command. The video recording command is sent to the camera device and controls it to record video. The alarm command is sent to the alarm device on the communication base station tower of the monitoring point where the camera device is located and controls it to issue an alarm signal.

[0059] Specifically, the spatiotemporal correlation is compared with a threshold. If the spatiotemporal correlation is greater than the threshold, it is confirmed that a train is about to enter the monitoring point, thus triggering the generation of events and alarm commands. Simultaneously, a video recording command is generated to record video at the initial heartbeat time point. When generating an event, an event ID is constructed to associate the camera equipment, video recording data, and the event. Furthermore, the alarm equipment (i.e., the horn) at the corresponding monitoring point can be located via addressing, controlling it to emit a warning signal to deter personnel near the monitoring point. The recorded video can be sent to railway maintenance personnel for warning and education. In one example, the threshold can be set to 0.6; in other examples, other values ​​can be selected. In addition to the alarm sound emitted by the horn, alarm commands can also be sent to the intelligent early warning platform for railway maintenance for display and alarm. By constructing an intelligent early warning platform for railway maintenance, remote monitoring and maintenance can be performed. All equipment is connected to a unified management platform, supporting status monitoring, fault alarms, and remote configuration, significantly reducing operation and maintenance costs and improving system reliability. Figure 3 The railway safety early warning method based on communication base station towers proposed in the embodiments of this application records video screenshots of people appearing near railway lines. If this occurs, an early warning can be issued to these individuals, prioritizing their safety. The railway safety early warning method based on communication base station towers proposed in the embodiments of this application can identify and record information on individuals frequently loitering along railway lines and those who intrude, providing a basis for warning and education for management departments.

[0060] In a specific embodiment, it further includes: in response to determining that the spatiotemporal correlation between the radar and the video is less than or equal to a threshold, then not generating an event, video recording instruction, or alarm instruction.

[0061] Specifically, if the spatiotemporal correlation is less than or equal to the threshold, no event, video recording command, or alarm command will be generated. Steps S2-S3 can be repeated at different time periods to provide safety warnings to people near the railway and avoid the risk of accidents caused by people lingering near the tracks after a train passes.

[0062] In a specific embodiment, the method further includes: assigning a target number to the same target in different video frames in real-time video data, constructing a continuous trajectory by combining the target's coordinates in the video frame and the corresponding time point, and determining whether the target is in a stagnant state based on the continuous trajectory.

[0063] Specifically, embodiments of this application can combine target detection models to identify personnel or animals that frequently linger within the railway red line for evidence collection. This is mainly achieved through target detection → tracking → trajectory analysis → lingering determination. Lingering determination involves assigning a unique target ID to the same person across frames and constructing a continuous trajectory (Trajectory = {ID, (x1,y1,x2,y2)_t1, (x1,y1,x2,y2)_t2, ...}), where Trajectory represents the trajectory, (x1,y1,x2,y2)_t1 represents the coordinates of the target box within the video frame at time t1, (x1,y1,x2,y2)_t2 represents the coordinates of the target box within the video frame at time t2, and so on. If the time spent in the continuous trajectory corresponding to the target ID exceeds a time threshold, it is confirmed as lingering. Simultaneously, constructing a continuous trajectory can also solve the problem of target ID jumps caused by "occlusion" or "brief disappearance."

[0064] It should be noted that the data collected in the embodiments of this application are all legally collected data, the process of using the data is also legal and compliant, and the application is also a legal purpose.

[0065] Further reference Figure 4 As an implementation of the methods shown in the above figures, this application provides an embodiment of a railway safety early warning device based on a communication base station tower. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0066] This application provides a railway safety early warning device based on communication base station towers, employing radar, camera equipment, and alarm equipment installed on communication base station towers along the railway line, including:

[0067] The spatiotemporal correlation prediction model construction module 1 is configured to obtain preset values ​​of radar parameters, construct a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters.

[0068] Prediction module 2 is configured to acquire echo signals collected by a radar operating at the optimal value of radar parameters at a preset distance from the monitoring point where the camera is located. In response to detecting a target entering the monitoring range based on the echo signal, it acquires radar monitoring data and real-time video data collected by the camera. The real-time video data is input into the target detection model to obtain each target and its corresponding confidence level. The target motion displacement is determined based on the position of each target in the real-time video data. The confidence level and target motion displacement of each target corresponding to the radar monitoring data and real-time video data are input into the trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation.

[0069] The early warning module 3 is configured to generate an event, a video recording instruction, and an alarm instruction in response to a determination that the spatiotemporal correlation is greater than a threshold. The video recording instruction is sent to the camera device and controls it to record video. The alarm instruction is sent to the alarm device on the communication base station tower at the monitoring point where the camera device is located and controls it to issue an alarm signal.

[0070] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 5 As shown, the electronic device of this embodiment includes a processor 501 and a memory 502; wherein the memory 502 is used to store computer execution instructions; and the processor 501 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0071] Alternatively, the memory 502 can be either standalone or integrated with the processor 501.

[0072] When the memory 502 is set up independently, the electronic device also includes a bus 503 for connecting the memory 502 and the processor 501.

[0073] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 501, implement the above method.

[0074] This invention also provides a computer program product, including a computer program that, when executed by a processor 501, implements the above-described method.

[0075] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0076] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0077] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0078] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 501 to execute some steps of the methods of the various embodiments of this application.

[0079] It should be understood that the processor 501 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 501 can be any conventional processor 501. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 501 executing the steps, or as a combination of hardware and software modules within the processor 501 executing the steps.

[0080] The memory 502 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0081] Bus 503 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 503 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 503 in the accompanying drawings of this application is not limited to only one bus 503 or one type of bus 503.

[0082] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0083] An exemplary storage medium is coupled to processor 501, enabling processor 501 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of processor 501. Processor 501 and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, processor 501 and storage medium can exist as discrete components in an electronic device or host device.

[0084] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A railway safety early warning method based on communication base station towers, characterized in that, The method involves using radar, camera, and alarm equipment installed on communication base station towers along the railway line, including the following steps: Obtain preset values ​​for radar parameters, construct a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters. The system acquires echo signals collected by a radar operating at the optimal values ​​of the radar parameters at a preset distance from the monitoring point where the camera is located. In response to detecting a target entering the monitoring range based on the echo signals, it acquires radar monitoring data and real-time video data collected by the camera. The real-time video data is input into a target detection model to obtain the position of each target and its corresponding confidence level. The target displacement is determined based on the position of each target in the real-time video data. The radar monitoring data, the confidence levels of each target corresponding to the real-time video data, and the target displacement are input into a trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation. In response to determining that the spatiotemporal correlation degree is greater than a threshold, an event, a video recording instruction, and an alarm instruction are generated. The video recording instruction is sent to the camera device and controls it to record video. The alarm instruction is sent to the alarm device on the communication base station tower of the monitoring point where the camera device is located and controls it to issue an alarm signal.

2. The railway safety early warning method based on communication base station towers according to claim 1, characterized in that, The radar parameters include maximum detection speed, continuous value, minimum detection speed, vertical deflection angle, sensitivity, horizontal deflection angle, energy threshold, distance limit setting, and detection direction. The radar monitoring data includes the distance between the radar and the monitoring point where the camera device is located, the target's movement speed, sensitivity, detection direction, and angle, including vertical deflection angle and horizontal deflection angle. The time point when the target detected by the radar enters the monitoring range is taken as the starting heartbeat time point, and the time point when the target detected by the radar leaves the monitoring range is taken as the ending heartbeat time point. Within the time range from the starting heartbeat time point to the ending heartbeat time point, the radar is controlled to emit a transmission signal at preset time intervals, and the radar monitoring data and real-time video data are collected. The video recording time starts from the ending heartbeat time point.

3. The railway safety early warning method based on communication base station towers according to claim 1, characterized in that, The target detection model is obtained by fine-tuning a pre-trained target detection model using video data collected by camera equipment in historical events; the spatiotemporal correlation prediction model includes an LSTM model, and during the training process of the spatiotemporal correlation prediction model, a Bayesian optimization algorithm is used to iteratively optimize the parameters of the LSTM model and the radar parameters until the optimal parameters of the LSTM model and the optimal values ​​of the radar parameters are obtained.

4. The railway safety early warning method based on communication base station towers according to claim 1, characterized in that, The target detection pre-trained model includes a pre-trained YOLO model, and the detected targets include trains, humans, or animals; the radar includes millimeter-wave radar, the alarm device includes a horn, and the alarm signal includes an alarm sound emitted by the horn.

5. The railway safety early warning method based on communication base station towers according to claim 1, characterized in that, Also includes: A target number is assigned to the same target in different video frames in the real-time video data, and a continuous trajectory is constructed by combining the target's coordinates in the video frame and the corresponding time point. Based on the continuous trajectory, it is determined whether the target is in a lingering state.

6. The railway safety early warning method based on communication base station towers according to claim 1, characterized in that, Also includes: In response to determining that the spatiotemporal correlation between the radar and the video is less than or equal to a threshold, no event, video recording command, or alarm command is generated.

7. A railway safety early warning device based on a communication base station tower, characterized in that, Radar, camera, and alarm equipment installed on communication base station towers along the railway line are employed, including: The spatiotemporal correlation prediction model construction module is configured to obtain preset values ​​of radar parameters, construct a spatiotemporal correlation prediction model between radar and video, train the spatiotemporal correlation prediction model by combining the preset values ​​of radar parameters and the target detection model, adjust the radar parameters, obtain the trained spatiotemporal correlation prediction model, and determine the optimal values ​​of radar parameters. The prediction module is configured to acquire echo signals collected by a radar operating at the optimal values ​​of the radar parameters at a preset distance from the monitoring point where the camera device is located; in response to detecting a target entering the monitoring range based on the echo signals, it acquires radar monitoring data and real-time video data collected by the camera device; inputs the real-time video data into a target detection model to obtain the position of each target and its corresponding confidence level; determines the target motion displacement based on the position of each target in the real-time video data; and inputs the radar monitoring data, the confidence level of each target corresponding to the real-time video data, and the target motion displacement into the trained spatiotemporal correlation prediction model to obtain the spatiotemporal correlation. The early warning module is configured to generate an event, a video recording instruction, and an alarm instruction in response to determining that the spatiotemporal correlation is greater than a threshold. The video recording instruction is sent to the camera device and controls it to record video. The alarm instruction is sent to the alarm device on the communication base station tower at the monitoring point where the camera device is located and controls it to issue an alarm signal.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images