Railway safety protection alarm method and system, electronic equipment and storage medium

Abstract

The invention provides a railway safety protection alarm method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining the alarm information of sensor equipment which is used for monitoring the boundary and limit of a railway, and sending out the alarm information when an alarm target is detected; based on the alarm information, determining an image acquisition device associated with the sensor device; based on the image acquisition device, performing image acquisition on an alarm point where the sensor device is located to obtain an alarm image, and performing target detection on the alarm image to obtain a detection result; and performing fusion judgment on the alarm information and the detection result, and determining whether to give an alarm or not based on a judgment result. According to the invention, the visual perception is combined with the sensor, the visual perception capability is improved, the false alarm rate of sensor equipment is reduced by using the image analysis technology, and the alarm accuracy and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the field of railway safety technology, and in particular to a railway safety protection alarm method, system, electronic equipment and storage medium. Background Art

[0002] With the rapid development of railway transportation, ensuring the safety of railway operation has become a vital link. In order to effectively prevent abnormal intrusions, natural disasters and other potential threats, the railway department has widely deployed various security protection measures in fences (i.e. boundaries) and restricted areas. Among them, surveillance cameras, as a traditional and intuitive monitoring method, are widely installed in key areas to monitor and record changes in the surrounding environment in real time. However, the camera's perception of the surrounding environment is limited, and a large amount of human resources are still required for auxiliary monitoring, which increases operating costs and management difficulties.

[0003] In order to make up for the shortcomings of surveillance cameras, some special fence (land boundary) and limit sensor devices have emerged in recent years, such as laser radar, vibration fiber, etc. These sensor devices can monitor the environment along the railway in real time through different physical principles. However, these sensor devices often need to work in complex and changing environments. Natural factors such as animal activities and the slightest disturbance may trigger false alarms, resulting in a high false alarm rate when these sensor devices are used. Therefore, how to effectively reduce the false alarm rate while improving the safety protection capabilities of railway fences (land boundaries) and limits has become a technical problem that needs to be solved in the current field of railway safety protection. Summary of the invention

[0004] The present invention provides a railway safety protection alarm method, system, electronic equipment and storage medium, which are used to solve the defect of a high false alarm rate in the railway safety protection alarm in the related art.

[0005] The present invention provides a railway safety protection alarm method, comprising: Acquiring alarm information from a sensor device, wherein the sensor device is used to monitor the boundary and limit of the railway and to issue an alarm information when an alarm target is detected; Based on the alarm information, determining an image acquisition device associated with the sensor device; Based on the image acquisition device, an image is acquired at the alarm point where the sensor device is located to obtain an alarm image, and target detection is performed on the alarm image to obtain a detection result; The alarm information and the detection result are combined and judged, and whether to issue an alarm is determined based on the judgment result.

[0006] According to a railway safety protection alarm method provided by the present invention, the target detection is performed on the alarm image to obtain the detection result, including: Performing target detection on the alarm image to obtain the category and position of the initial target; Based on the defense zone of the image acquisition device, the initial target is screened to obtain candidate targets; Based on the background image and the alarm image, as well as the position of the alarm target and the position of the candidate target carried in the alarm information, the candidate targets are screened to obtain abnormal targets, and the category and position of the abnormal targets are used as detection results. The background image is acquired by the image acquisition device before the sensor device issues an alarm message.

[0007] According to a railway safety protection alarm method provided by the present invention, based on the background image and the alarm image, as well as the position of the alarm target and the position of the candidate target carried in the alarm information, the candidate target is screened to obtain an abnormal target, including: Comparing the background image with the alarm image to obtain a comparison result; Converting the position of the alarm target to the image coordinate system of the candidate target, and matching the converted position of the alarm target with the position of the candidate target to obtain a matching result; Based on the comparison result and / or the matching result, the candidate targets are screened to obtain abnormal targets.

[0008] According to a railway safety protection alarm method provided by the present invention, the alarm information and the detection result are integrated and judged, and whether to alarm is determined based on the judgment result, including: Acquire weather information of a monitoring area where the sensor device is located, and determine a current usage scenario based on the weather information; Based on the current usage scenario, determining a weight of the sensor device and a weight of the image acquisition device; Determining a confidence weighted sum based on the weight and confidence of the sensor device and the weight and confidence of the image acquisition device; When the weighted sum of the confidences is greater than or equal to a preset threshold, it is determined to issue an alarm.

[0009] According to a railway safety protection alarm method provided by the present invention, the step of determining to alarm further includes: An audible and visual alarm is issued, and an emergency alarm notification is sent to the vehicle-road collaborative alarm system to notify the train to pay attention to protection.

[0010] According to a railway safety protection alarm method provided by the present invention, the determining of the image acquisition device associated with the sensor device based on the alarm information includes: Parsing the alarm information to obtain the location of the alarm target and the device number of the sensor device; Check whether the location of the alarm target is within the defense zone of the sensor device. If so, based on the device number of the sensor device, search for the image acquisition device number associated with the device number from the device association relationship library; Based on the image acquisition device number, an image acquisition device associated with the sensor device is determined.

[0011] A railway safety protection alarm method provided by the present invention also includes: Based on the image acquisition device, video acquisition of a specified duration is performed on the alarm point where the sensor device is located to obtain an alarm video.

[0012] The present invention also provides a railway safety protection alarm system, including a sensor device, an image acquisition device, a railway safety alarm platform and a vehicle-road cooperative alarm system; The sensor device is used to monitor the boundaries and limits of the railway and send alarm information to the railway safety alarm platform when an alarm target is detected; The railway safety alarm platform is used to determine the image acquisition device associated with the sensor device based on the alarm information, control the image acquisition device to acquire an image at the alarm point where the sensor device is located to obtain an alarm image, and perform target detection on the alarm image to obtain a detection result; The railway safety alarm platform is also used to integrate and judge the alarm information and the detection results, and when it is determined to issue an alarm based on the judgment result, send an emergency alarm notification to the vehicle-road collaborative alarm system.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the railway safety protection alarm method as described above is implemented.

[0014] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the railway safety protection alarm method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the railway safety protection alarm method as described above is implemented.

[0016] The railway safety protection alarm method, system, electronic device and storage medium provided by the present invention can immediately capture abnormal events within the boundaries and limits by using sensor devices to monitor the boundaries and limits of the railway in real time, and when the alarm information sent by the sensor device is obtained, the image acquisition device associated with the sensor device is determined based on the alarm information, thereby utilizing the rapid response capability of the image acquisition device to perform instant image acquisition of the alarm point, providing real-time data support for subsequent target detection and judgment. By introducing image recognition and target detection technology, it is possible to realize automatic processing and analysis of the collected alarm images, reduce manual intervention, and improve processing efficiency. The present invention not only relies on the alarm information of the sensor device, but also combines the real-time image acquired by the image acquisition device for target detection, realizing the combination of visual perception and sensor. This multiple verification mechanism can significantly reduce the false alarm rate and greatly improve the accuracy and reliability of the alarm. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the railway safety protection alarm method provided by the present invention; Figure 2 It is a schematic diagram of the process of video and image acquisition provided by the present invention; Figure 3 It is a schematic diagram of the flow of the image analysis algorithm provided by the present invention; Figure 4 It is a flowchart of the alarm fusion judgment provided by the present invention; Figure 5 It is a structural schematic diagram of the railway safety protection alarm system provided by the present invention; Figure 6 It is a structural schematic diagram of the railway safety alarm platform provided by the present invention; Figure 7 It is a schematic diagram of the alarm processing flow of the railway safety protection alarm system provided by the present invention; Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] With the rapid development of railway transportation, its safety and stability have attracted more and more attention from all walks of life. Railway fences (land boundaries) and limits are important barriers to ensure the safety of railway operation, and their safety protection work is particularly important. Railway fences (land boundaries) refer to the boundary areas within a certain range demarcated around the railway line, which are intended to protect the railway operation from external interference or potential threats. This boundary area may include the railway land boundary, protective fences, and other physical barriers set up to ensure safety. The setting of railway fences (land boundaries) helps prevent foreign objects (such as pedestrians, vehicles or animals, etc.) from invading the railway line, thereby ensuring the smoothness and safety of railway traffic. The railway limit refers to the outline dimension line that must not be crossed for the railway line and the buildings and equipment on both sides of the railway line in order to ensure the safety of railway transportation. These limits include locomotive and vehicle limits, building approach limits, etc., which strictly limit the minimum safe distance between locomotives, vehicles, buildings and equipment and railway lines to prevent accidents caused by mutual interference.

[0021] In order to effectively prevent the intrusion of foreign objects, natural disasters and other potential threats, the railway department has widely deployed various safety protection measures in fences (boundaries) and restricted areas. Among them, surveillance cameras, as a traditional and intuitive means of monitoring, are widely installed in key areas to monitor and record changes in the surrounding environment in real time. However, due to the limitations of the camera's own perception ability and environmental adaptability, such as limited vision at night and the influence of bad weather, relying solely on surveillance cameras for all-weather, no-dead-angle monitoring seems to be inadequate, so a large amount of human resources are required for auxiliary monitoring, which undoubtedly increases operating costs and management difficulties.

[0022] In order to make up for the shortcomings of surveillance cameras, a series of special fences (boundaries) and limit sensors have emerged in recent years, such as laser radar, vibration fiber, etc. These sensor devices can realize the perception and real-time monitoring of the environment along the railway through different physical principles. For example, laser radar can use laser beams to scan the surrounding environment, build a three-dimensional space model, and promptly detect and locate abnormal targets; vibration fiber monitors ground vibrations to warn of potential intrusions. However, these sensor devices still have certain defects in practical applications. One of the main problems is the high false alarm rate. Since sensor devices often need to work in complex and changing environments, natural factors such as animal activities and disturbances may trigger false alarms, which not only increases the workload of monitoring personnel, but also may cause real safety hazards to be ignored.

[0023] In this regard, the present invention provides a railway safety protection alarm method, which improves visual perception ability by utilizing the existing railway monitoring system and various sensor devices to cooperate with each other, and uses image analysis technology to analyze the alarm point image, reduce the false alarm rate of the sensor device, and more accurately judge the alarm information. At the same time, the alarm information of various sensor devices is uniformly managed, and the images and videos of the alarm points are collected to facilitate alarm handling, thereby overcoming the above defects.

[0024] Figure 1 Schematic diagram of the railway safety protection alarm method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, obtaining alarm information of a sensor device, wherein the sensor device is used to monitor fences (boundaries) and limits of the railway and issue an alarm information when an alarm target is detected; Specifically, the method provided in the embodiment of the present invention can be implemented based on a railway safety alarm platform, which is connected to various sensor devices and image acquisition devices deployed along the railway through a network. The sensor devices are used to monitor the fences (boundaries) and limits of the railway, and send alarm information to the railway safety alarm platform when an alarm target is detected.

[0025] Here, the alarm target refers to any abnormal situation or object detected by the sensor device, which may threaten the safe operation of the railway, such as illegal intruders, animal crossings, signs of equipment failure, etc. The alarm information of the sensor device refers to the warning signal or data issued by the sensor device installed near the railway fence (land boundary) or limit when an abnormal situation (such as an intruder, obstacle, equipment failure, etc.) is detected. These alarm information may include the specific target that triggered the alarm, timestamp, location information, and device information that issued the alarm information (such as device number, device name, device type, device installation location, device defense zone, etc.).

[0026] It is understandable that the sensor equipment may include laser radar, millimeter wave radar, vibration fiber, geological monitoring equipment (such as crack meter, inclinometer, geological water content meter, etc.), etc., to achieve unified protection of railway fences (boundaries) and limits. Among them, the laser radar is installed at the entrance of the railway tunnel, mainly used to detect whether there are foreign objects on the rails. If foreign objects are found, an alarm message will be sent to the railway safety alarm platform. Millimeter wave radar is installed at conventional roadbed / track surface, road-bridge junction and other locations where people and animals may appear, to detect the intrusion of people and animals, and send an alarm message to the railway safety alarm platform when an intrusion is detected. Vibration fiber is installed on the fences along the railway. Climbing and collision and other behaviors that cause the fence to vibrate will be detected by the vibration fiber, and an alarm message will be sent to the railway safety alarm platform. The geological monitoring equipment will send an alarm message to the railway safety alarm platform when it detects that the corresponding monitoring data exceeds the threshold.

[0027] It should be noted that all sensor devices are connected to the railway safety alarm platform through the network, but the connection protocol formats of different devices are different. Therefore, the railway safety alarm platform is equipped with a system control unit, which includes a device access module, through which different access protocols can be adapted. When the sensor device detects an alarm target, the alarm information can be reported to the device access module, which parses the alarm information and repackages the parsed content into a unified alarm format for unified management.

[0028] Step 120, based on the alarm information, determining the image acquisition device associated with the sensor device; Specifically, the system control unit pre-stores the device information of each sensor device and the device information of each image acquisition device, and the association relationship between the sensor device and the image acquisition device is also pre-stored in the system control unit. Each device is assigned a unique number, and the corresponding device information can be found from the storage of the system control unit according to this unique number. It should be understood that the system control unit mainly uses a database to store information, and a local configuration file can also be used. The format of the configuration file includes but is not limited to JSON format, XML format, and INI format.

[0029] When the system control unit receives the alarm information sent by the sensor device, it can parse the alarm information to obtain the device number of the sensor device carried in the alarm information. According to the device number, the image acquisition device number associated with it can be found from the pre-stored device association relationship, and the image acquisition device associated with the sensor device can be determined according to the number. It should be understood that one sensor device can be associated with multiple image acquisition devices.

[0030] Here, the image acquisition device refers to a device that can capture and record visual information, such as a video camera, etc., and the embodiment of the present invention does not specifically limit this. In the railway safety protection system, the image acquisition device can be installed at a key position to capture real-time images near the alarm point for subsequent analysis and judgment.

[0031] Step 130, based on the image acquisition device, perform image acquisition on the alarm point where the sensor device is located to obtain an alarm image, and perform target detection on the alarm image to obtain a detection result; Specifically, after determining the image acquisition device associated with the sensor device, the image acquisition device can be controlled to acquire images (such as taking photos, recording videos, etc.) at the alarm point where the sensor device is located to obtain an alarm image. Here, the alarm point where the sensor device is located refers to the specific location where the alarm information is triggered, that is, the exact location where the sensor device detects an abnormal situation. The alarm image refers to the image obtained by photographing the alarm point where the sensor device is located by the image acquisition device after receiving the alarm information.

[0032] After the alarm image is acquired, the image analysis algorithm can be used to perform target detection on the alarm image to obtain the detection result. Here, target detection refers to the process of automatically identifying and locating targets of interest (such as people, vehicles, animals, etc.) in the image using computer vision technology. In the railway safety protection system, target detection involves analyzing and processing the alarm image to identify targets that may threaten railway safety. The detection result refers to the comprehensive judgment result of whether there is a target in the image and the type, location, size, etc. of the target obtained after the image analysis algorithm performs target detection analysis on the alarm image.

[0033] Step 140: perform a fusion judgment on the alarm information and the detection result, and determine whether to issue an alarm based on the judgment result.

[0034] Specifically, the alarm information sent by the sensor device and the image detection results of the image acquisition device can be integrated to form a comprehensive description of the alarm event. Considering that environmental factors (such as weather, lighting conditions, etc.) will have a certain impact on the data collection of the sensor device and the image acquisition device, for example, heavy fog and heavy rain will cause the laser radar and camera to not work properly, but have little impact on other devices such as millimeter wave radar; strong winds and hail will cause the vibrating optical fiber to produce a large false alarm rate. Therefore, when the alarm information of the sensor device and the detection results of the image acquisition device are fused and judged, weights can be assigned to devices from different sources.

[0035] Specifically, different usage scenarios can be configured in advance according to weather conditions, and corresponding weights can be configured for all devices in each usage scenario. When making a fusion judgment, the weather information of the current monitoring area can be obtained first, and the current corresponding usage scenario can be determined based on the weather information, so that the device weights in the usage scenario can be applied for weighted calculation. For example, a weighted summation method can be used to calculate the weighted sum of the confidences of the sensor device and the associated image acquisition device. If the weighted sum of the confidences exceeds a certain threshold, an alarm is confirmed. It should be understood that the confidence of each device can use a default confidence (such as 100), or it can be determined based on the alarm information or detection results of the device, and the embodiments of the present invention do not specifically limit this.

[0036] It is understandable that the above judgment result refers to the conclusion drawn after the fusion judgment of the alarm information and the detection result. The conclusion can be a binary result (such as a "yes / no" alarm), a multivariate result (such as the specific threat type identified), or a report containing detailed information (such as the type, location, urgency, etc. of the threat). If the judgment result shows that there is an abnormal situation, the security alarm platform will immediately issue an alarm signal, which can be sent to relevant users in various ways such as sound and light signals so that they can take timely response measures. At the same time, an emergency alarm notification is sent to the vehicle-road collaborative alarm system to notify nearby trains to pay attention to protection. If the judgment result shows that there is no abnormal situation, the platform will not issue an alarm signal, but it can record relevant information about the incident for subsequent analysis and improvement.

[0037] The method provided by the embodiment of the present invention can immediately capture abnormal events within the fence (land boundary) and the boundary by using sensor equipment to monitor the fence (land boundary) and the boundary in real time, and when the alarm information issued by the sensor equipment is obtained, the image acquisition device associated with the sensor equipment is determined based on the alarm information, thereby utilizing the rapid response capability of the image acquisition device to perform instant image acquisition of the alarm point, and provide real-time data support for subsequent target detection and judgment. By introducing image recognition and target detection technology, it is possible to realize automatic processing and analysis of the collected alarm images, reduce manual intervention, and improve processing efficiency. The present invention not only relies on the alarm information of the sensor equipment, but also combines the real-time image acquired by the image acquisition device for target detection, realizing the combination of video perception and sensor. This multiple verification mechanism can significantly reduce the false alarm rate and greatly improve the accuracy and reliability of the alarm.

[0038] Based on the above embodiment, step 120 specifically includes: Step 121, parsing the alarm information to obtain the location of the alarm target and the device number of the sensor device; Specifically, when the railway safety alarm platform receives the alarm information sent by the sensor device, it first parses the alarm information to obtain information such as the location of the alarm target and the device number of the sensor device carried in the alarm information. Here, the location of the alarm target refers to the specific geographical location that triggers the alarm of the sensor device. For example, if the sensor device detects that an illegal intruder is approaching the railway, the location of the alarm target is the location of the intruder. The device number of the sensor device is a unique identifier used to distinguish and identify different sensor devices. Each sensor device will be assigned a unique number for easy management and maintenance. When the platform receives an alarm message from a sensor device, by parsing this information, it can obtain the device number of the sensor device that triggered the alarm, and then understand which specific device issued the alarm.

[0039] Step 122, checking whether the position of the alarm target is within the defense zone of the sensor device, if so, searching the image acquisition device number associated with the device number from the device association relationship library based on the device number of the sensor device; Step 123: Determine the image acquisition device associated with the sensor device based on the image acquisition device number.

[0040] Specifically, after parsing the location of the alarm target, it is necessary to check whether the alarm target is located within the defense zone of the sensor device. Here, the defense zone of the sensor device refers to the geographical area that the sensor device is responsible for monitoring and protecting. In order to improve the efficiency and accuracy of monitoring, the railway line is usually divided into multiple defense zones, and each defense zone is equipped with corresponding sensor devices. These sensor devices are responsible for monitoring the safety situation in their respective defense zones and issuing alarm information when abnormal conditions are detected. Therefore, when the system receives the alarm information, it is necessary to determine whether the location of the alarm target is within the defense zone of the sensor device that triggers the alarm to ensure the accuracy and pertinence of the alarm. It should be understood that the defense zone information of the sensor device is pre-configured and stored in the system control unit. In addition to the defense zone information, the system control unit also stores other information of the device, including but not limited to the device number, device name, device type, device installation location, device defense zone information, etc. Among them, the defense zone of the device is different according to the type of device, including but not limited to polygonal defense zone, circular defense zone, etc.

[0041] After determining that the location of the alarm target is within the defense zone of the sensor device, the associated image acquisition device number can be found from the device association library based on the device number of the sensor device obtained by analysis, and then the corresponding image acquisition device can be controlled to take photos and record videos based on the image acquisition device number. Here, the device association library refers to a database that stores the associations between different devices. Since multiple devices such as sensor devices and image acquisition devices need to work together, it is necessary to establish a device association library to record the association information between these devices. The device association library can include fields such as device number, device type, and associated device number to describe the connection and dependency between different devices. When the system needs to determine the image acquisition device associated with a certain sensor device, it can obtain the corresponding information by querying the device association library.

[0042] Based on any of the above embodiments, Figure 2 It is a schematic diagram of the process of video and image acquisition provided by the present invention, such as Figure 2 As shown, the steps of video and image acquisition include: Step S210, controlling the PTZ camera to continuously collect images at a preset frequency, and upon receiving an alarm message from a sensor device, obtaining an image within a preset time period before receiving the alarm message from a cache as a background image; It should be noted that the image acquisition device provided in the embodiment of the present invention may include a camera without a pan / tilt head and a camera with a pan / tilt head. The camera's shooting strategy may be different depending on whether the camera has a pan / tilt head. After installation, the camera without a pan / tilt head can capture images in a fixed orientation, while the camera with a pan / tilt head can capture images in more orientations. Different shooting strategies are used based on this feature.

[0043] Specifically, the PTZ-free camera will take pictures all the time. For example, the PTZ-free camera can collect images at intervals of 1 second, and the system control unit will cache historical photos for a period of time. When the system control unit receives an alarm message from a sensor device, it will obtain a certain number of historical photos from the cache and store them as background images. For example, when receiving an alarm message, 5 photos taken within 15 seconds before the alarm can be saved as background images. It should be understood that when using a PTZ-free camera to shoot background images, it is necessary to calculate the delay from the sensor device sending an alarm message to the start of the camera operation. Currently, photos from 15 seconds ago are saved to ensure that there are no intruding targets in the background image.

[0044] Here, background images refer to images obtained from the cache within a preset time period before receiving the alarm information when the image acquisition device (such as a pan / tilt camera) receives the alarm information from the sensor device during continuous image acquisition. These images are used as the background in subsequent processing to be compared and analyzed with the images collected at the time of the alarm (i.e., the alarm image). The background image provides a reference for a scene when it is normal or when no abnormality has occurred. By comparing and analyzing the alarm image with the background image, changes in the scene can be identified, thereby more effectively screening out abnormal targets and reducing false alarms and missed alarms.

[0045] Step S220, after receiving the alarm information of the sensor device, collect a video of the alarm point for a specified time to obtain an alarm video; Step S230: after receiving the alarm information from the sensor device, collect an image of the alarm point to obtain an alarm image.

[0046] Specifically, after receiving the alarm information sent by the sensor device, the system control unit controls the image acquisition device (such as a camera without a pan-tilt camera or a camera with a pan-tilt camera) to start recording the alarm point, and stops recording when the recording reaches a specified time to obtain an alarm video. Here, the specified time can be pre-configured to 20 seconds, or it can be set according to actual needs. The embodiment of the present invention does not specifically limit this. It should be understood that the alarm video provides a real-time picture when the alarm event occurs and records the entire process of the event, which is crucial for subsequent analysis, processing and tracing. By viewing the alarm video, you can intuitively understand the specific circumstances of the incident, including key information such as time, location, and people or objects involved.

[0047] In addition, when receiving the alarm information of the sensor device, the system control unit can control the image acquisition device (such as a camera without a pan-tilt tilt or a camera with a pan-tilt tilt) to take photos of the alarm point. The image acquisition device will save the photos according to the pre-configured interval and number of photos to ensure that the intrusion target is captured to the maximum extent and the alarm image is obtained.

[0048] Optionally, different operations are performed according to the type of camera: a camera without a pan / tilt head takes 10 photos at intervals of 1 second; a camera with a pan / tilt head first controls the pan / tilt head to rotate to a specified position, and after confirming that it is rotated in place, 10 photos are taken at intervals of 1 second. The video recording duration and shooting parameters can be determined according to the stored configuration file. It should be understood that when using a camera with a pan / tilt head, it is necessary to pre-calculate the time it takes for the pan / tilt head to rotate into place, and this rotation time can be the time it takes for the pan / tilt head to rotate one circle.

[0049] It is understandable that the purpose of using a camera to take pictures is to capture the intruder, so multiple pictures will be taken. It is more convenient to use a camera without a pan / tilt. If you use a camera with a pan / tilt, you need to wait for the pan / tilt to rotate to the right position before starting to take pictures. The interval and number of pictures taken can be configured according to the specific situation of the current scene.

[0050] Based on any of the above embodiments, Figure 3 It is a flow chart of the image analysis algorithm provided by the present invention, such as Figure 3 As shown, in step 130, the target detection is performed on the alarm image to obtain the detection result, including: Step 310, performing target detection on the alarm image to obtain the category and position of the initial target; Specifically, an image analysis algorithm can be used to perform target detection on the alarm image to obtain the category and position of the initial target. Here, the initial target refers to all possible target objects that are initially identified from the alarm image during the target detection process. These target objects may include people, cars, animals, objects, etc., which are regarded as potential objects that need further analysis and processing. The category of the initial target refers to the result of classifying the initial target, that is, determining which category each target object belongs to (such as people, cars, animals, etc.). The position of the initial target refers to the specific position of the target object in the image, which is usually expressed by coordinates (such as pixel coordinates or relative position in the image).

[0051] Optionally, the above-mentioned image analysis algorithm can be a YOLOv8 target detection algorithm. YOLOv8 continues the advantages of the YOLO (YouOnly Look Once, which is a target detection algorithm) series and has improved speed and accuracy. YOLOv8 model training requires collecting images in all scenes, annotating and organizing data sets according to the requirements of YOLOv8, and training models on the data sets to adapt to specific application scenarios.

[0052] Step 320, based on the defense zone of the image acquisition device, screening the initial target to obtain candidate targets; Specifically, according to the defense zone of the image acquisition device, the initial targets can be screened and the false alarm targets can be screened out. Here, the defense zone of the image acquisition device refers to the geographical area that the device is responsible for monitoring and protecting. In the railway safety protection system, in order to improve the efficiency and accuracy of monitoring, the railway line is usually divided into multiple defense zones, and each defense zone is equipped with a corresponding image acquisition device. It should be noted that the defense zone of the image acquisition device covers the defense zone of the sensor device, and multiple image acquisition devices can be used to monitor the defense zone of a sensor device.

[0053] It is understandable that the defense zone of the image acquisition device is pre-configured, which can be a polygonal area used to screen targets within the area. Specifically, it can be checked whether the position of each initial target is within the defense zone of the image acquisition device. If the target is within the defense zone, the target is retained as a candidate target; if the target is not within the defense zone, it is excluded to reduce false alarms and improve the accuracy of target detection. Here, candidate targets refer to the initial targets retained after defense zone screening. These targets are considered to be objects that appear in the defense zone of the image acquisition device and need to be further analyzed and processed.

[0054] Step 330, based on the background image and the alarm image, as well as the position of the alarm target and the position of the candidate target carried in the alarm information, the candidate targets are screened to obtain abnormal targets, and the category and position of the abnormal targets are used as detection results. The background image is acquired based on the image acquisition device before the sensor device issues an alarm message.

[0055] Specifically, the image analysis algorithm can use the background image and the alarm image to compare to determine the target location of the intrusion, further filter out the misidentified targets in the image, and improve the accuracy of the judgment. It should be understood that since the background image is collected by a non-pan-tilt camera, when the cameras associated with the sensor device are all pan-tilt cameras, this screening and filtering method is not applicable to pan-tilt cameras, and is only applicable to non-pan-tilt cameras.

[0056] Furthermore, the image analysis algorithm can also match the position of the alarm target with the position of each candidate target in the alarm image through coordinate transformation to filter out the false alarm targets in the image, thereby obtaining accurate abnormal targets. Here, abnormal targets refer to targets that are determined to truly represent abnormal situations or potential threats after further analysis of candidate targets. The category and location of the abnormal target will be output as the final detection result. The outline of the abnormal target can be marked with a box in the alarm image, and the category of the abnormal target can be marked with text, and the marked alarm image can be stored.

[0057] It is understandable that the use of image analysis algorithms is the focus of the embodiments of the present invention. By using the YOLOv8 algorithm to load the trained model, the alarm image is detected and the intruding target is distinguished. At the same time, the polygonal defense zone is used to screen the target within the defense zone, and the background image is used to further screen the target to improve the recognition accuracy. The training of the model is very important, and the model needs to be continuously optimized during use to improve the recognition accuracy.

[0058] Based on any of the above embodiments, step 330 specifically includes: Step 331, comparing the background image with the alarm image to obtain a comparison result; Specifically, the background image and the alarm image may be preprocessed as necessary, such as denoising, contrast enhancement, etc., to ensure that the image quality is suitable for comparison. Then, key features are extracted from the background image and the alarm image, respectively. These features may be pixel-level (such as grayscale value, color histogram, etc.) or higher-level (such as edge, corner point, texture, etc.). Finally, the extracted features are compared, for example, the similarity or distance between the features is calculated, and the difference between the two images is judged based on these similarities or distances to obtain the image comparison results.

[0059] Step 332, converting the position of the alarm target to the image coordinate system of the candidate target, and matching the converted position of the alarm target with the position of the candidate target to obtain a matching result; Specifically, after screening by the position of the alarm target, first determine the coordinate conversion relationship between the position of the alarm target (usually in the coordinate system of the sensor device) and the candidate target image (taken by the image acquisition device). Then, using the coordinate conversion relationship, convert the position of the alarm target to the image coordinate system of the candidate target. Next, match the converted alarm target position with the position of the candidate target, which can be achieved by calculating the distance or overlap between the two and judging whether certain matching conditions are met (such as the distance is less than a certain threshold, the overlap area is greater than a certain ratio, etc.). Judge the matching result according to the matching conditions. If the conditions are met, it is considered that the candidate target corresponding to the alarm target has been found.

[0060] Step 333: Based on the comparison result and / or the matching result, the candidate targets are screened to obtain abnormal targets.

[0061] Specifically, the comparison result obtained in step 331 and / or the matching result obtained in step 332 are comprehensively judged. The comparison result can reflect the overall difference of the image, while the matching result can locate the specific target object more accurately. The screening conditions are set according to the application scenario and security requirements. These conditions may include the similarity threshold of the comparison result, the distance threshold of the matching result, etc. The candidate targets are screened according to the screening conditions. If a candidate target meets the screening conditions of the comparison result and the matching result at the same time, it is regarded as an abnormal target. Finally, the category and position of the screened abnormal target are output as the detection result.

[0062] Based on any of the above embodiments, Figure 4 It is a flowchart of the alarm fusion judgment provided by the present invention, such as Figure 4 As shown, step 140 specifically includes: Step 410, obtaining weather information of a monitoring area where the sensor device is located, and determining a current usage scenario based on the weather information; Step 420, determining a weight of the sensor device and a weight of the image acquisition device based on the current usage scenario; Step 430, determining a confidence weighted sum based on the weight and confidence of the sensor device and the weight and confidence of the image acquisition device; Step 440: When the weighted sum of the confidences is greater than or equal to a preset threshold, determine to issue an alarm.

[0063] Specifically, when the detection results of the image analysis algorithm and the alarm information of the sensor device are fused and judged, the basis for the fusion judgment can be the weighted sum of the confidences of the current sensor device and the associated image acquisition device. When the weighted sum of the confidences is greater than or equal to a preset threshold (such as 100), the alarm is determined. After determining to make an alarm, the alarm management client of the railway safety alarm platform can use pop-up boxes and sounds to remind users to handle the alarm, and send an emergency alarm to the vehicle-road collaborative alarm system to notify nearby trains to pay attention to protection. It should be understood that after determining to make an alarm, the alarm information and detection results can be fused and displayed on the alarm management client for user viewing and handling. In addition, voice can also be used to broadcast the key content of the current alarm.

[0064] The confidence weighted sum is used to integrate the confirmation of multiple alarm-related devices on the alarm, and its weight is determined according to the accuracy of the device in the current usage scenario. There are multiple usage scenarios preset in the system control unit. Each scenario corresponds to the device weight configuration under a specific weather condition. The default confidence value of each device is 100, and the weight is a value in the [0,1] interval. If the weight of a device is configured to 1, it means that the alarm of the device is very important and must be sent to the alarm management client for processing; if the weight of the device is configured to a value less than 1, it means that the alarm of the device needs the assistance of other devices to confirm before it can be sent to the alarm management client for processing.

[0065] It is understandable that determining the weight of equipment based on weather conditions is an empirical judgment. For example, heavy fog and heavy rain will cause lidar and cameras to not work properly, but will have little effect on other equipment such as millimeter-wave radar. Strong winds and hail will cause vibrating optical fibers to produce many false alarms. Based on these experiences, the weights of equipment in different weather conditions can be determined.

[0066] Specifically, when calculating the weighted sum of the confidences of the sensor device and its associated image acquisition device, the weather information of the current monitoring area (i.e., the monitoring area where the sensor device is located) can be obtained first, and the current usage scenario can be determined based on the weather information. According to the current usage scenario, the pre-configured device weights for the scenario can be obtained from the system control unit, including the weights of the sensor device and the weights of the image acquisition device. Subsequently, the weighted sum of the confidences of the associated devices can be calculated based on the obtained weights and the confidences of the pre-configured devices. When the weighted sum of the confidences is greater than or equal to a preset threshold, it can be determined to issue an alarm. It should be understood that the weather information of the current monitoring area can be obtained by deploying a meteorological station, or by obtaining a weather forecast online, and the embodiments of the present invention do not specifically limit this.

[0067] In the embodiment of the present invention, different usage scenarios are configured in advance according to weather conditions, and corresponding device weights are configured in each usage scenario. According to the weather information of the current monitoring area obtained, the corresponding usage scenario can be switched to automatically or manually. When it is found that there are many false alarms in the alarm of a certain monitoring area, the usage scenario corresponding to the monitoring area can be manually adjusted.

[0068] The method provided by the embodiment of the present invention can reduce false alarms to a certain extent by fusing and judging the sensor equipment and image acquisition equipment associated with the alarm, and can adapt to various weather conditions by adopting scene switching, making the platform more environmentally adaptable.

[0069] Based on any of the above embodiments, Figure 5 Schematic diagram of the railway safety protection alarm system provided by the present invention. Figure 5 As shown, the system includes a sensor device 510, an image acquisition device 520, a railway safety alarm platform 530, and a vehicle-road cooperative alarm system 540; The sensor device 510 is used to monitor the fence (boundary) and limit of the railway, and send alarm information to the railway safety alarm platform 530 when an alarm target is detected; The railway safety alarm platform 530 is used to determine the image acquisition device 520 associated with the sensor device 510 based on the alarm information, control the image acquisition device 520 to acquire an image at the alarm point where the sensor device 510 is located to obtain an alarm image, and perform target detection on the alarm image to obtain a detection result; The railway safety alarm platform 530 is also used to perform a fusion judgment on the alarm information and the detection result, and when it is determined to issue an alarm based on the judgment result, send an emergency alarm notification to the vehicle-road collaborative alarm system 540.

[0070] The system provided by the embodiment of the present invention can immediately capture abnormal events within the fence (land boundary) and the boundary by using sensor equipment to monitor the fence (land boundary) and the boundary in real time, and when the alarm information issued by the sensor equipment is obtained, the image acquisition device associated with the sensor equipment is determined based on the alarm information, thereby utilizing the rapid response capability of the image acquisition device to perform instant image acquisition of the alarm point, and provide real-time data support for subsequent target detection and judgment. By introducing image recognition and target detection technology, it is possible to realize automatic processing and analysis of the collected alarm images, reduce manual intervention, and improve processing efficiency. The present invention not only relies on the alarm information of the sensor equipment, but also combines the real-time image acquired by the image acquisition device for target detection, realizing the combination of video perception and sensor. This multiple verification mechanism can significantly reduce the false alarm rate and greatly improve the accuracy and reliability of the alarm.

[0071] Based on any of the above embodiments, Figure 6 It is a structural schematic diagram of the railway safety alarm platform provided by the present invention, such as Figure 6 As shown, the platform includes an image analysis unit 531, an object storage unit 532, a system control unit 533, a message distribution unit 534 and an alarm management client 535; The image analysis unit 531 is used to analyze the alarm image collected by the image acquisition device during the alarm, and identify abnormal targets in the image; The object storage unit 532 is used to store the alarm image and alarm video saved during the alarm; The system control unit 533 is used to receive alarm information from the sensor device, control the image acquisition device to record and take pictures, call the image analysis unit to perform target detection on the alarm image, and send the detection result to the alarm management client 535; The message distribution unit 534 is used to cache messages and provide a publish and subscribe method to distribute messages; The alarm management client 535 is used to manage alarms.

[0072] It is understandable that by setting up the message distribution unit, the platform uses the message queue to cache messages, publishes and subscribes to messages according to the message topic, and other unit modules can conveniently subscribe to alarm messages and collaboratively process alarms. The above-mentioned unit modules jointly realize the functions of the railway safety alarm platform. The division of specific unit modules is designed according to the software functions. Different designers will have different division methods, but the purpose is the same.

[0073] Furthermore, the system control unit may include an algorithm scheduling module, a fusion analysis module, a video access module and a device access module, wherein the algorithm scheduling module is used to connect with the image analysis unit, send the image to be analyzed, and receive the analysis result of the image analysis algorithm. The fusion analysis module is used to process the alarm information reported by the device access module, call the image analysis unit and the algorithm scheduling module to take pictures and perform image analysis, and fuse the results of multiple devices to determine the final alarm report. The video access module is used to access the videos recorded by all image acquisition devices and improve the video stream for the algorithm scheduling module to take pictures and record videos. The device access module is used to adapt to various sensor device access protocols and receive alarm information from sensor devices.

[0074] Based on any of the above embodiments, Figure 7 FIG. 1 is a schematic diagram of the alarm processing flow of the railway safety protection alarm system provided by the present invention. Figure 7 As shown, the alarm information sent by the sensor device is published to the message distribution unit through the device access module; the fusion analysis module subscribes to the alarm message topic from the message distribution unit to receive the alarm message, parses the alarm message, finds the image acquisition device number (such as the camera number) associated with the alarm sensor device, and notifies the algorithm scheduling module; the algorithm scheduling module controls the corresponding image acquisition device (such as the camera) to start recording and taking pictures, calls the image analysis unit to perform target detection on the alarm image, and receives the detection result, determines the alarm result based on the fusion judgment of the sensor device and the image acquisition device associated with the alarm, and publishes it to the message distribution unit; the fusion analysis module subscribes to the alarm result message topic from the message distribution unit to receive the alarm result, and publishes the alarm to the message distribution unit if it determines that the alarm is a positive alarm based on the alarm result; the alarm management client subscribes to the alarm topic published by the fusion analysis module to receive the confirmed alarm message, then reminds the user to handle the alarm, and sends an emergency alarm notification to the vehicle-road collaborative alarm system.

[0075] It is understandable that after the sensor device sends out an alarm message, the system control unit will take a photo of the alarm point, use the image analysis algorithm to analyze the intrusion target, make a secondary judgment on the alarm, and then make a fusion judgment on each alarm-related device to confirm the alarm. After the judgment of multiple devices and the calculation of the device confidence weight, the accuracy of the alarm is improved.

[0076] In the embodiment of the present invention, the image acquisition device is combined with the sensor device to improve the visual perception ability, and the image analysis algorithm is used to analyze the alarm point image to reduce the false alarm rate of the sensor device. By using a multi-scene switching solution, it can adapt to different weather conditions and use different operating scenes according to the weather conditions to improve the accuracy of the alarm. In addition, all sensor device alarms are managed uniformly, and the images and videos of the alarm points are recorded to facilitate alarm handling.

[0077] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the railway safety protection alarm method, which includes: obtaining alarm information of a sensor device, the sensor device is used to monitor the boundary and limit of the railway, and issue an alarm information when an alarm target is detected; based on the alarm information, determining the image acquisition device associated with the sensor device; based on the image acquisition device, performing image acquisition on the alarm point where the sensor device is located to obtain an alarm image, and performing target detection on the alarm image to obtain a detection result; performing a fusion judgment on the alarm information and the detection result, and determining whether to alarm based on the judgment result.

[0078] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a storage medium. When the computer program is executed by a processor, the computer can execute the railway safety protection alarm method provided by the above-mentioned methods, and the method includes: obtaining alarm information of a sensor device, wherein the sensor device is used to monitor the boundaries and limits of the railway and issue an alarm information when an alarm target is detected; based on the alarm information, determining an image acquisition device associated with the sensor device; based on the image acquisition device, performing image acquisition on the alarm point where the sensor device is located to obtain an alarm image, and performing target detection on the alarm image to obtain a detection result; performing a fusion judgment on the alarm information and the detection result, and determining whether to issue an alarm based on the judgment result.

[0080] On the other hand, the present invention also provides a storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the railway safety protection alarm method provided by the above-mentioned methods, the method comprising: obtaining alarm information of a sensor device, the sensor device being used to monitor the boundaries and limits of the railway, and issuing an alarm information when an alarm target is detected; based on the alarm information, determining an image acquisition device associated with the sensor device; based on the image acquisition device, performing image acquisition on the alarm point where the sensor device is located to obtain an alarm image, and performing target detection on the alarm image to obtain a detection result; performing a fusion judgment on the alarm information and the detection result, and determining whether to issue an alarm based on the judgment result.

[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A railway safety protection alarm method, characterized in that: include: Acquiring alarm information from a sensor device, wherein the sensor device is used to monitor the boundary and limit of the railway and to issue an alarm information when an alarm target is detected; Based on the alarm information, determining an image acquisition device associated with the sensor device; Based on the image acquisition device, an image is acquired at the alarm point where the sensor device is located to obtain an alarm image, and target detection is performed on the alarm image to obtain a detection result; The alarm information and the detection result are combined and judged, and whether to issue an alarm is determined based on the judgment result.

2. The railway safety protection alarm method according to claim 1, characterized in that: The performing target detection on the alarm image to obtain a detection result includes: Performing target detection on the alarm image to obtain the category and position of the initial target; Based on the defense zone of the image acquisition device, the initial target is screened to obtain candidate targets; Based on the background image and the alarm image, as well as the position of the alarm target and the position of the candidate target carried in the alarm information, the candidate targets are screened to obtain abnormal targets, and the category and position of the abnormal targets are used as detection results. The background image is acquired by the image acquisition device before the sensor device issues an alarm message.

3. The railway safety protection alarm method according to claim 2 is characterized in that: The method of screening the candidate targets based on the background image and the alarm image, and the position of the alarm target and the position of the candidate targets carried in the alarm information to obtain abnormal targets includes: Comparing the background image with the alarm image to obtain a comparison result; Converting the position of the alarm target to the image coordinate system of the candidate target, and matching the converted position of the alarm target with the position of the candidate target to obtain a matching result; Based on the comparison result and / or the matching result, the candidate targets are screened to obtain abnormal targets.

4. The railway safety protection alarm method according to claim 1, characterized in that: The step of fusing and judging the alarm information and the detection result, and determining whether to issue an alarm based on the judgment result, includes: Acquire weather information of a monitoring area where the sensor device is located, and determine a current usage scenario based on the weather information; Based on the current usage scenario, determining a weight of the sensor device and a weight of the image acquisition device; Determining a confidence weighted sum based on the weight and confidence of the sensor device and the weight and confidence of the image acquisition device; When the weighted sum of the confidences is greater than or equal to a preset threshold, it is determined to issue an alarm.

5. The railway safety protection alarm method according to claim 4, characterized in that: The step of determining to issue an alarm further includes: An audible and visual alarm is issued, and an emergency alarm notification is sent to the vehicle-road collaborative alarm system to notify the train to pay attention to protection.

6. The railway safety protection alarm method according to any one of claims 1 to 5, characterized in that: The step of determining the image acquisition device associated with the sensor device based on the alarm information includes: Parsing the alarm information to obtain the location of the alarm target and the device number of the sensor device; Check whether the location of the alarm target is within the defense zone of the sensor device. If so, based on the device number of the sensor device, search for the image acquisition device number associated with the device number from the device association relationship library; Based on the image acquisition device number, an image acquisition device associated with the sensor device is determined.

7. The railway safety protection alarm method according to any one of claims 1 to 5, characterized in that: Also includes: Based on the image acquisition device, video acquisition of a specified duration is performed on the alarm point where the sensor device is located to obtain an alarm video.

8. A railway safety protection alarm system, characterized in that: Including sensor equipment, image acquisition equipment, railway safety alarm platform and vehicle-road collaborative alarm system; The sensor device is used to monitor the boundaries and limits of the railway and send alarm information to the railway safety alarm platform when an alarm target is detected; The railway safety alarm platform is used to determine the image acquisition device associated with the sensor device based on the alarm information, control the image acquisition device to acquire an image at the alarm point where the sensor device is located to obtain an alarm image, and perform target detection on the alarm image to obtain a detection result; The railway safety alarm platform is also used to integrate and judge the alarm information and the detection results, and when it is determined to issue an alarm based on the judgment result, send an emergency alarm notification to the vehicle-road collaborative alarm system.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the railway safety protection alarm method as described in any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the railway safety protection alarm method as described in any one of claims 1 to 7 is implemented.

Images