Image processing method and system for subway along protection zone

Abstract

The application provides an image processing method and system for a subway along a protection area. The method can obtain monitoring videos of a subway along a range through a camera in a camera cluster. The video data contains subway vehicle operating conditions of a track driving area and dynamic information of the protection area along the track. A monitoring platform intelligently detects and identifies subway vehicles and / or unmanned aerial vehicles in the monitoring videos to discover potential safety hazards in time.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to an image processing method and system for use in a protected area along a subway line. Background Art

[0002] With the acceleration of urbanization, subways, as a vital component of public transportation, are attracting increasing attention for their safety and operational efficiency. Metro line protection zones, a crucial component of system security, are crucial for ensuring the proper operation of subway vehicles and the safety of passengers. Traditionally, monitoring of subway line protection zones relies primarily on manual patrols and fixed-point monitoring. These methods are not only inefficient but also struggle to achieve real-time monitoring of large areas.

[0003] With the rapid development of video surveillance technology, camera clusters are widely used in monitoring systems along subway lines. These cameras can capture real-time images and video data along the subway line, providing monitoring personnel with an intuitive monitoring view. However, relying solely on raw video data captured by cameras makes it difficult for monitoring personnel to quickly and accurately identify potential safety hazards, especially when dynamic elements such as the operating status of subway vehicles and the presence of flying objects such as drones within protected areas along the line are involved. Summary of the Invention

[0004] The present application provides an image processing method and system for protection zones along subway lines. Based on the detection status of subway vehicles and / or drones in surveillance videos, the method comprehensively evaluates and determines the safety level of the protection zones along the lines, thereby promptly identifying potential safety hazards.

[0005] In a first aspect, the present application provides an image processing method for a subway line protection zone, which is applied to a subway line monitoring system. The subway line monitoring system includes: a camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication; the method includes:

[0006] Acquire surveillance videos of a subway line range through cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line range includes a track running area and a protection area along the line;

[0007] The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0008] In the above scheme, through the cameras in the camera cluster, real-time monitoring videos of the subway along the line range can be obtained. These video data contain the running status of the subway vehicle in the track running area and the dynamic information of the along-line protection area. The monitoring platform intelligently detects and identifies the subway vehicle and / or the unmanned aerial vehicle in the monitoring video. Through the detection of the subway vehicle in the track running area, the running status of the subway vehicle can be judged; and through the detection of the unmanned aerial vehicle in the along-line protection area, potential safety hazards can be found in time to avoid the unmanned aerial vehicle in the along-line protection area close to the subway vehicle during the running of the subway vehicle. According to the detection status of the subway vehicle and the unmanned aerial vehicle in the monitoring video, the monitoring platform can comprehensively evaluate and determine the safety level of the along-line protection area. When the monitoring platform determines that the safety level of the along-line protection area is high risk or medium risk, it can send early warning information to the relevant departments in time and take corresponding measures. For example, for the case of unmanned aerial vehicle intrusion, the monitoring platform directly sends a safety adjustment instruction to the unmanned aerial vehicle, or the monitoring platform sends the safety adjustment instruction to the monitoring device (including the above-mentioned camera) set on the line monitoring position first, and then sends it to the unmanned aerial vehicle through the communication module on the monitoring device to instruct it to fly away from the track running area.

[0009] Optionally, the camera is a first camera, and the monitoring video is a first monitoring video; the first camera is arranged on a fixed object outside the subway along-line range;

[0010] The monitoring platform extracts a first feature image sequence from the first monitoring video, the first feature image sequence being a video frame image in which the track running area exists a subway vehicle in the first monitoring video;

[0011] The monitoring platform determines the safety level of the along-line protection area according to a preset target detection model and the first feature image sequence.

[0012] In the above scheme, the first camera is arranged on a fixed object outside the subway along-line range. This layout strategy can avoid adjusting the subway vehicle itself and the inspection unmanned aerial vehicle itself, but can set an additional device that does not affect the original equipment operation on the basis of the original subway vehicle and the inspection unmanned aerial vehicle, so that the system has better expandability and compatibility, especially suitable for upgrading scenarios of old subway lines.

[0013] Optionally, the monitoring platform determines the safety level of the along-line protection area according to a preset target detection model and the first feature image sequence, comprising:

[0014] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0015] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the second safety level, and the first safety level is lower than the second safety level.

[0016] In the above solution, the monitoring platform uses a preset object detection model to detect drones in the first feature image sequence (i.e., video frames containing subway vehicles). Based on the drone detection results, the monitoring platform classifies the safety level of the protected area along the line into two levels: if a drone is detected, the safety level is set to Level 1 (lower); if no drone is detected, the safety level is set to Level 2 (higher). This clear safety level classification standard provides subway operations management departments with a clear and quantifiable basis for safety assessment. Once a drone intrusion is detected (i.e., Level 1), the monitoring platform can immediately trigger an early warning mechanism, notifying relevant departments to take countermeasures. The monitoring platform can send the safety level assessment results and related information (such as the drone's location and flight trajectory) to the subway operations management department in real time. Based on the information provided by the monitoring platform, the subway operations management department can formulate scientific safety management measures and emergency response plans. Real-time monitoring and a rapid response mechanism promote collaboration between the subway operations management department and relevant departments, improving the ability to respond to emergencies.

[0017] Optionally, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0018] The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one of time advance and time delay based on the first feature image sequence;

[0019] Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0020] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0021] If the monitoring platform does not detect the drone in any of the feature images in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protected area along the line is the second safety level, and the first safety level is lower than the second safety level;

[0022] If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, then the safety level of the protected area along the line is determined to be a third safety level, which is lower than the second safety level and higher than the first safety level.

[0023] If the monitoring platform does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, the safety level of the protection zone along the line is determined to be the fourth safety level, which is higher than the second safety level.

[0024] In the above scheme, after the monitoring platform extracts a first feature image sequence (i.e., video frames containing subway train images) from the first surveillance video, it further extends this sequence in time to generate a second feature image sequence. The second feature image sequence is extended based on the first feature image sequence by at least one of time advance and time delay. By extending the time advance and / or time delay, the second feature image sequence covers a wider time range, enabling the monitoring platform to obtain more historical information and future trends on drone activity. This expanded feature image sequence provides more data samples for drone detection, helping to reduce false detections and missed detections and improve detection accuracy. The monitoring platform uses a pre-set object detection model to detect drones in the first and second feature image sequences and determines the safety level of the protected area along the line based on the detection results. By dually detecting the first and second feature image sequences, the monitoring platform can more comprehensively capture drone activity within the protected area along the line, including both transient and persistent drones. Based on the detection results, the monitoring platform categorizes the safety level of the protected area along the line into four levels (Safety Level 1 to Safety Level 4), each corresponding to different drone activity patterns and security risks. If a drone is detected in the first feature image sequence, the security level is set to Level 1, indicating a significant security risk. If no drone is detected in the first feature image sequence, but a drone is detected in the difference between the second feature image sequence and the first feature image sequence, the security level is set to Level 3, indicating a potential security risk, but the risk level is lower than Level 1. If no drone is detected in either the first or second feature image sequence, the security level is set to Level 2 or Level 4 (depending on the specific detection results of the second feature image sequence), indicating a low or very low security risk.

[0025] Optionally, the camera is a second camera, which is provided on the drone, and the monitoring video is a second monitoring video captured when the drone is located in the protected area along the line; correspondingly, the monitoring platform determines the safety level of the protected area along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protected area along the line in the monitoring video, including:

[0026] If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, determining that the safety level of the protection zone along the track is the first safety level;

[0027] If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0028] In the above scheme, the surveillance video is acquired by the second camera carried by the drone and transmitted to the monitoring platform in real time. Then, the monitoring platform determines the safety level of the protection zone along the track based on the detection status of the subway vehicle in the track running area in the second surveillance video. If a subway vehicle is detected in at least one characteristic image, the safety level is determined to be the first safety level, indicating that there may be safety risks related to the operation of subway vehicles in the protection zone along the track. If no subway vehicle is detected in any of the characteristic images, the safety level is determined to be the second safety level, indicating that the current safety risk of the protection zone along the track is low. The above-mentioned method of acquiring surveillance video by the second camera carried by the drone can be based on the original camera of the drone, without the need to add other equipment, and the implementation cost is low.

[0029] Optionally, the camera is a third camera, which is installed on the subway vehicle, and the monitoring video is a third monitoring video captured when the subway vehicle is located in the track travel area; correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track travel area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0030] If the monitoring platform detects the presence of a drone in the protected area along the line in at least one characteristic image in the third monitoring video, determining that the safety level of the protected area along the line is the first safety level;

[0031] If the monitoring platform does not detect the presence of a drone in the protection zone along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0032] In the above solution, a third camera is installed on a subway car. As the subway car moves, its field of view dynamically covers the protected areas along the line, enabling continuous and uninterrupted monitoring. Compared to fixed cameras, subway car-viewpoint monitoring provides more comprehensive coverage of the protected areas along the line, reducing blind spots and improving monitoring continuity and effectiveness. The third surveillance video captured by the third camera is transmitted in real time to the monitoring platform, which analyzes the video content and identifies potential threats, such as drones, within the protected areas along the line. The monitoring platform determines the security level based on the drone detection status of the protected areas along the line in the third surveillance video. If a drone is detected in at least one feature image, the security level is determined to be Level 1, indicating a high security risk in the protected areas along the line. If no drone is detected in any feature image, the security level is determined to be Level 2, indicating a low security risk in the protected areas along the line. Through drone detection, the monitoring platform can accurately identify potential threats within the protected areas along the line, providing a reliable basis for security risk assessment. The subway car-viewpoint monitoring method significantly enhances the comprehensiveness and real-time nature of monitoring through dynamic field of view coverage, real-time data transmission, and instant analysis.

[0033] Optionally, after the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, the method further includes:

[0034] If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

[0035] In this solution, after determining the safety level of protected areas along the route through the monitoring platform's monitoring of subway vehicles and drones, a safety adjustment instruction mechanism was introduced. This mechanism is designed to quickly instruct drones to take appropriate measures when safety risks are detected, effectively preventing accidents.

[0036] In a second aspect, the present application provides a subway line monitoring system, comprising: a camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication;

[0037] Acquire surveillance videos of a subway line range through cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line range includes a track running area and a protection area along the line;

[0038] The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0039] Optionally, the camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line;

[0040] The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video;

[0041] The monitoring platform determines the safety level of the protection zone along the line based on a preset target detection model and the first feature image sequence.

[0042] Optionally, the monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence, including:

[0043] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0044] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the second safety level, and the first safety level is lower than the second safety level.

[0045] Optionally, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0046] The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one of time advance and time delay based on the first feature image sequence;

[0047] Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0048] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0049] If the monitoring platform does not detect the UAV in any of the first feature image sequence using the preset target detection model, it is determined that the security level of the along-line protection area is a second security level, the first security level is lower than the second security level;

[0050] If the monitoring platform detects the UAV in at least one feature image in the difference set of the second feature image sequence and the first feature image sequence using the preset target detection model, it is determined that the security level of the along-line protection area is a third security level, the third security level is lower than the second security level, and the third security level is higher than the first security level;

[0051] If the monitoring platform does not detect the UAV in any of the second feature image sequence using the preset target detection model, it is determined that the security level of the along-line protection area is a fourth security level, the fourth security level is higher than the second security level.

[0052] Optionally, the camera is a second camera, the second camera is arranged on the UAV, and the monitoring video is a second monitoring video captured when the UAV is located in the along-line protection area; correspondingly, the monitoring platform determines the security level of the along-line protection area according to the detection state of the subway vehicle in the track running area in the monitoring video and / or the detection state of the UAV in the along-line protection area in the monitoring video, including:

[0053] If the monitoring platform detects the presence of subway vehicles in the track running area in at least one feature image in the second monitoring video, it is determined that the security level of the along-line protection area is a first security level;

[0054] If the monitoring platform does not detect the presence of subway vehicles in the track running area in any of the second monitoring video, it is determined that the security level of the along-line protection area is a second security level.

[0055] Optionally, the camera is a third camera, the third camera is arranged on the subway vehicle, and the monitoring video is a third monitoring video captured when the subway vehicle is located in the track running area; correspondingly, the monitoring platform determines the security level of the along-line protection area according to the detection state of the subway vehicle in the track running area in the monitoring video and / or the detection state of the UAV in the along-line protection area in the monitoring video, including:

[0056] If the monitoring platform detects the presence of UAVs in the along-line protection area in at least one feature image in the third monitoring video, it is determined that the security level of the along-line protection area is a first security level;

[0057] If the monitoring platform does not detect the UAV in any feature image in the third monitoring video, it is determined that the security level of the track protection zone is a second security level.

[0058] Optionally, after the monitoring platform determines the security level of the track protection zone according to the metro vehicle detection state of the track running area in the monitoring video and / or the UAV detection state of the track protection zone in the monitoring video, it further includes:

[0059] If the monitoring platform determines that the security level is the first security level or lower than the first security level, the monitoring platform sends a security adjustment instruction to the UAV, wherein the security adjustment instruction is used to instruct the UAV to fly away from the track running area.

[0060] In a third aspect, the present application provides an electronic device, comprising:

[0061] a processor; and,

[0062] a memory for storing executable instructions of the processor;

[0063] wherein the processor is configured to execute any one of the possible methods described in the first aspect by executing the executable instructions.

[0064] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement any one of the possible methods described in the first aspect.

[0065] The image processing method and system for the protection zone along the subway provided in this application can obtain real-time monitoring videos of the subway line range through the cameras in the camera cluster. These video data contain the operating status of subway vehicles in the track driving area and dynamic information of the protection zone along the line. The monitoring platform performs intelligent detection and identification of subway vehicles and / or drones in the monitoring video. By detecting subway vehicles in the track driving area, the operating status of subway vehicles can be judged; and by detecting drones in the protection zone along the line, potential safety hazards can be discovered in time to avoid drones patrolling in the protection zone along the line close to subway vehicles during the driving of subway vehicles. Based on the detection status of subway vehicles and drones in the monitoring video, the monitoring platform can comprehensively evaluate and determine the safety level of the protection zone along the line. When the monitoring platform determines that the safety level of the protection zone along the line is high risk or medium risk, it can send early warning information to the relevant departments in a timely manner and take corresponding response measures. For example, in the event of a drone intrusion, the monitoring platform directly sends a safety adjustment instruction to the drone, or the monitoring platform first sends the safety adjustment instruction to a monitoring device (including the above-mentioned camera) set at the line monitoring position, and then sends it to the drone through the communication module on the monitoring device to instruct it to fly away from the track driving area. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1 1 is a flowchart illustrating an image processing method for a protected area along a subway line according to an exemplary embodiment of the present application;

[0068] Figure 2 is a flowchart illustrating an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application;

[0069] Figure 3 is a flowchart of an image processing method for a protected area along a subway line according to yet another exemplary embodiment of the present application;

[0070] Figure 4 1 is a schematic structural diagram of a subway line monitoring system according to an exemplary embodiment of the present application;

[0071] Figure 5 It is a structural diagram of an electronic device according to an exemplary embodiment of the present application.

[0072] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0073] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0074] To address the aforementioned issues, the embodiments provided in this application achieve comprehensive coverage and real-time monitoring of the entire subway line by deploying a cluster of cameras at various locations along the subway line. These cameras include fixed cameras located outside the subway line, mobile cameras mounted on drones, and onboard cameras mounted on subway vehicles. They can capture surveillance video from different perspectives and locations, providing a rich data source for subsequent image processing and analysis.

[0075] The monitoring platform intelligently identifies and analyzes the surveillance videos it receives. First, it uses target detection algorithms (such as convolutional neural networks in deep learning) to identify subway vehicles and drones in the video. Based on the drone's detection status within the protected areas along the line, the monitoring platform assesses the safety level of the protected areas. If a drone is detected entering a protected area along the line, it is determined to be high-risk and an emergency response mechanism is immediately triggered. For example, a safety adjustment instruction is sent to the drone, instructing it to fly away from the track to avoid collisions with subway vehicles and other safety accidents.

[0076] Figure 1 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to an exemplary embodiment of the present application. Figure 1 As shown, the method provided in this embodiment includes:

[0077] S101. Obtain surveillance video along the subway line through cameras in the camera cluster.

[0078] In this step, a first surveillance video of the subway line is captured by the first camera in the camera cluster and sent to the monitoring platform. The subway line includes the track running area and the line protection area. The first camera is set on a fixed object outside the subway line.

[0079] Specifically, the first camera can be installed on a fixed object outside the subway line, such as a high-rise building, a lighthouse, or a specially installed monitoring tower. These fixed objects should have high stability and a good field of view to ensure that images of the subway line (including the track running area and the protection area along the line) can be clearly captured. The first camera starts working, capturing the surveillance video of the subway line in real time, and sending it to the monitoring platform as the first monitoring video. The first monitoring video should have high resolution and real-time performance so that the monitoring platform can accurately analyze and process it.

[0080] S102: The monitoring platform extracts a first feature image sequence from a first monitoring video.

[0081] In this step, the monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track running area in the first monitoring video.

[0082] Specifically, after receiving the first surveillance video, the monitoring platform processes the video using computer vision processing equipment (such as a GPU-accelerated server). The monitoring platform uses advanced video processing technologies, such as frame differencing, background subtraction, motion detection algorithms, advanced learning algorithms, image segmentation algorithms, and feature matching algorithms, to automatically extract video frame images of subway vehicles in the track travel area from the first surveillance video. These video frame images constitute a first feature image sequence. To ensure accuracy, the monitoring platform may also use additional screening mechanisms, such as image quality assessment and noise removal, to further optimize the first feature image sequence.

[0083] S103: The monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence.

[0084] A pre-trained target detection model is deployed on the monitoring platform. This model is built on a deep learning framework (such as TensorFlow and PyTorch) and trained on a large number of drone images to accurately detect drone targets. The monitoring platform inputs the first feature image sequence into the pre-set target detection model, which analyzes each feature image to detect the presence of drone targets.

[0085] If the model detects a drone in at least one feature image in the first feature image sequence, the monitoring platform determines that the safety level of the protected area along the line is the first safety level (a lower safety level), indicating that there is a potential safety risk.

[0086] If the model does not detect a drone in any of the feature images in the first feature image sequence, the monitoring platform determines that the safety level of the protected area along the line is the second safety level (a higher safety level), indicating that the current safety situation is good.

[0087] Further, before determining that the drone is located in the along-line protection zone, the monitoring platform can further extract an outer contour of the drone from the target feature image, generate a contour feature rectangle according to the outer contour of the drone, and determine a contour feature center point of the contour feature rectangle, the contour feature rectangle being a minimum rectangle for completely containing the outer contour of the drone, and the target feature image being a video frame image including the drone in the monitoring video. The monitoring platform determines that the contour feature center point is located in a calibration region range in the target feature image, and then determines that the drone is located in the along-line protection zone.

[0088] Specifically, the monitoring platform screens video frame images containing the drone from the first feature image sequence, which are referred to as target feature images. The screening process can be based on a preset drone detection model or simple image analysis techniques such as color, shape, and other feature matching. This step mainly relies on the high-performance computing and storage capabilities of the monitoring platform to ensure that a large number of video frame images can be processed quickly and accurately.

[0089] For each target feature image, the monitoring platform uses an edge detection algorithm (such as Canny edge detection) to extract the outer contour of the drone. The edge detection algorithm identifies the edge contour of an object by analyzing the pixel gradient changes in the image. Before edge detection, the image may need to be preprocessed, such as grayscale, denoising, etc., to improve the accuracy of edge detection. In addition, in order to more accurately extract the contour of the drone, morphological operations (such as dilation, erosion) can be used to optimize the edge detection results.

[0090] After the outer contour of the drone is extracted, the monitoring platform uses a minimum bounding rectangle algorithm to generate a contour feature rectangle. This algorithm calculates the minimum enclosing rectangle of all points on the contour to obtain the minimum rectangle for completely containing the outer contour of the drone. The implementation of the minimum bounding rectangle algorithm can be based on related functions in image processing libraries (such as OpenCV). By calling these functions, the position and size of the contour feature rectangle can be quickly and accurately calculated.

[0091] The monitoring platform uses the method of contour moments to determine the center point of the contour feature rectangle. Contour moments is a mathematical tool for describing the shape of an image. By calculating the zeroth moment and the first moment of the contour, the center point coordinates of the contour can be obtained.

[0092] The monitoring platform predefines a calibration region in the target feature image, which is usually determined according to the actual location and size of the along-line protection zone of the subway. Then, the monitoring platform compares the coordinates of the contour feature center point with the boundary of the calibration region to determine whether it is located within the calibration region. If the coordinates of the contour feature center point are located within the calibration region, the monitoring platform determines that the drone is located in the along-line protection zone; otherwise, it is considered that the drone is not in the along-line protection zone.

[0093] In the above scheme, the edge detection algorithm accurately identifies the drone's outline by calculating changes in pixel gradients within the image. The edge detection algorithm's high computational speed allows it to process large amounts of image data in a short period of time, meeting the requirements of real-time monitoring. The step of generating a contour feature rectangle based on the drone's outer contour uses a minimum bounding rectangle algorithm to generate a minimum rectangle that tightly encloses the drone's outline. This minimum bounding rectangle tightly encloses the drone's outline, reducing redundant areas and providing a more precise target area for subsequent steps. Generating a unified minimum bounding rectangle allows for standardized processing of drones of varying sizes, facilitating subsequent feature extraction and position determination. The geometric center of the rectangle is then calculated to obtain the precise image position coordinates of the drone. This center coordinate remains stable even when the drone's posture changes or the image rotates, improving the robustness of the system. The step of determining whether the drone is within the protected area along the line by comparing the location of the contour feature center point with the boundary of the calibration area allows for accurate determination of the drone's location. Furthermore, the scope and location of the calibration area can be adjusted to meet the protection requirements of different subway lines. It is worth noting that for the above-mentioned solution in which the first camera is set on a fixed object, especially in the scenario where the shooting angle of the first camera is also fixed, the computational complexity of image processing can be greatly reduced by calibrating the area range in the target feature image, thereby improving the recognition speed.

[0094] In this embodiment, surveillance videos of the subway line are acquired in real time through the cameras in the camera cluster. Then, the monitoring platform intelligently detects and identifies the subway vehicles and / or drones in the surveillance videos. Based on the identification results and the detection status of the subway vehicles and drones in the surveillance videos, the safety level of the protection zone along the line is comprehensively evaluated and determined, thereby timely discovering potential safety hazards.

[0095] exist Figure 1 Based on the embodiment shown, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0096] The monitoring platform determines a second characteristic image sequence from the first monitoring video according to the first characteristic image sequence, wherein the second characteristic image sequence is a characteristic image sequence that is extended in at least one of time directions of time advance and time delay based on the first characteristic image sequence;

[0097] Correspondingly, the monitoring platform determines the safety level of the protection zone along the route based on the preset target detection model and the first feature image sequence, including:

[0098] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0099] If the monitoring platform does not detect the drone in any of the feature images in the first feature image sequence using the preset target detection model, the safety level of the protected area along the line is determined to be the second safety level, and the first safety level is lower than the second safety level;

[0100] If the monitoring platform detects a drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the third safety level, which is lower than the second safety level and higher than the first safety level.

[0101] If the monitoring platform does not detect a drone in any feature image in the second feature image sequence using the preset target detection model, the safety level of the protected area along the line is determined to be the fourth safety level, which is higher than the second safety level.

[0102] In the above scheme, after the monitoring platform extracts a first feature image sequence (i.e., video frames containing subway train images) from the first surveillance video, it further extends this sequence in time to generate a second feature image sequence. The second feature image sequence is extended based on the first feature image sequence by at least one of time advance and time delay. By extending the time advance and / or time delay, the second feature image sequence covers a wider time range, enabling the monitoring platform to obtain more historical information and future trends on drone activity. This expanded feature image sequence provides more data samples for drone detection, helping to reduce false detections and missed detections and improve detection accuracy. The monitoring platform uses a pre-set object detection model to detect drones in the first and second feature image sequences and determines the safety level of the protected area along the line based on the detection results. By dually detecting the first and second feature image sequences, the monitoring platform can more comprehensively capture drone activity within the protected area along the line, including both transient and persistent drones. Based on the detection results, the monitoring platform categorizes the safety level of the protected area along the line into four levels (Safety Level 1 to Safety Level 4), each corresponding to different drone activity patterns and security risks. If a drone is detected in the first feature image sequence, the security level is set to Level 1, indicating a significant security risk. If no drone is detected in the first feature image sequence, but a drone is detected in the difference between the second feature image sequence and the first feature image sequence, the security level is set to Level 3, indicating a potential security risk, but the risk level is lower than Level 1. If no drone is detected in either the first or second feature image sequence, the security level is set to Level 2 or Level 4 (depending on the specific detection results of the second feature image sequence), indicating a low or very low security risk.

[0103] Figure 2 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application. Figure 2 As shown, the image processing method for the protection zone along the subway provided in this embodiment includes:

[0104] S201. Obtain surveillance video along the subway line through cameras in the camera cluster.

[0105] In this step, the second camera in the camera cluster captures a second surveillance video of the subway line, including the track area and the protected area along the line, and transmits it to the monitoring platform. The second camera is mounted on a drone, and the second surveillance video is captured while the drone is within the protected area. It is worth noting that in this embodiment, the drone is a patrol drone, which is connected to the monitoring platform and, while uploading the surveillance video, also reports its location information in real time.

[0106] Specifically, the second camera can be a high-definition, wide-angle camera with night vision capabilities to ensure clear images of the protected areas along the subway line under various lighting conditions. This camera is securely mounted on the drone to ensure its field of view fully covers the track area and the protected areas along the line, while avoiding becoming an obstacle during flight.

[0107] The drone can fly along a pre-set route, ensuring comprehensive coverage of protected areas along the subway line. During flight, the drone maintains a stable altitude and speed to capture high-quality secondary surveillance video. This secondary surveillance video is transmitted to the monitoring platform in real time via wireless communication technology. To ensure data integrity and traceability, the surveillance video is also stored on the drone's local storage media for subsequent analysis.

[0108] After receiving the second surveillance video, the monitoring platform first preprocesses it, including noise reduction and contrast enhancement, to improve the accuracy and efficiency of subsequent image analysis. Feature images are extracted from the preprocessed second surveillance video at regular intervals (e.g., one frame per second). These feature images are used for subsequent subway vehicle and drone detection. For each feature image frame, the monitoring platform uses advanced image recognition algorithms (such as convolutional neural networks from deep learning) to inspect the track travel area. If a subway vehicle is detected, this status is recorded and analysis proceeds to the next frame. If a subway vehicle is detected in multiple consecutive frames, the presence of the subway vehicle can be further confirmed. Simultaneously, the monitoring platform also detects the drone itself in protected areas along the track. This ensures that the drone does not violate any safety regulations during flight, such as entering a prohibited flight zone or colliding with other flying objects. If any abnormal drone behavior (such as deviation from the route or abnormal altitude) is detected, this status is also recorded and appropriate safety measures are taken.

[0109] S202: If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, the safety level of the protection zone along the track is determined to be the first safety level.

[0110] Specifically, if the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image of the second surveillance video, it means that a subway vehicle may be currently operating in the protected area. In this case, the safety level of the protected area is determined to be level 1, indicating that there is a risk. In this case, the drone should be quickly controlled to evacuate the area to prevent any incidents that may endanger the safe operation of subway vehicles.

[0111] S203: If the monitoring platform does not detect any subway vehicle in the track travel area in any feature image in the second monitoring video, it determines that the safety level of the protection area along the track is the second safety level.

[0112] If the monitoring platform detects no subway vehicles in the track zone in any of the feature images in the second surveillance video, this may indicate that the protected area is not currently operating subway vehicles or that subway vehicles have not yet entered the area. In this case, the safety level of the protected area is determined to be Level 2, indicating that the safety situation in the protected area is relatively stable, but vigilance is still required to prevent any emergencies.

[0113] In the above solution, surveillance video is captured by a second camera mounted on the drone and transmitted to a monitoring platform in real time. The monitoring platform then determines the safety level of the protected area along the track based on the detection status of subway vehicles in the track-travelling area in the second surveillance video. If a subway vehicle is detected in at least one of the feature images, the safety level is determined to be the first safety level, indicating that the protected area along the track may have a safety risk related to subway vehicle operation. If no subway vehicle is detected in any of the feature images, the safety level is determined to be the second safety level, indicating that the current safety risk in the protected area along the track is low. This method of capturing surveillance video using the second camera mounted on the drone can be performed based on the drone's existing camera, eliminating the need for additional equipment and resulting in low implementation costs. It is worth noting that the above solution is particularly suitable for scenarios where the subway vehicle monitoring system and the subway line inspection system operate independently and their data cannot be interoperable. In such scenarios, subway vehicle operation takes precedence over subway line inspection. However, since the backend data between the two systems cannot be interoperable or shared, it is impossible to simultaneously utilize the location data of both systems for direct relative position determination.

[0114] Figure 3 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application. Figure 3 As shown, the image processing method for the protection zone along the subway provided in this embodiment includes:

[0115] S301. Obtain surveillance video along the subway line through cameras in the camera cluster.

[0116] In this step, a third surveillance video of the subway line is obtained through the third camera in the camera cluster, and the third surveillance video is sent to the monitoring platform. The subway line includes the track running area and the protection area along the line. Among them, the second camera is set on the subway vehicle, and the third surveillance video is the third surveillance video taken when the subway vehicle is in the track running area. It is worth noting that in this embodiment, the subway vehicle is connected to the monitoring platform for communication, and while uploading the surveillance video, it also reports the location information in real time. The above-mentioned drone is a drone of a third-party platform and cannot directly share data with the monitoring platform. However, optionally, the monitoring platform can be provided with a function of broadcasting an expulsion signal to the drone on a common frequency band.

[0117] Specifically, the third camera is mounted on the front or top of the subway car. It uses a high-definition, anti-shake camera with night vision capabilities, or it can directly use the subway car's own recorder camera. Its installation position ensures clear images of the protected areas along the line in front of and on both sides of the subway car. This camera can capture dynamic images of the protected areas along the line in real time as the subway car moves.

[0118] The captured third-party surveillance video is transmitted in real time to the monitoring platform via onboard wireless communication equipment. To ensure the real-time and accuracy of the data, efficient data compression and encryption technologies are employed during transmission to prevent data loss or tampering. In addition to real-time transmission, the third-party surveillance video is also stored on the subway vehicle's local storage media for subsequent analysis. The monitoring platform also regularly backs up and stores the received surveillance video to ensure data integrity and traceability.

[0119] After receiving the third surveillance video, the monitoring platform first performs preprocessing operations on it, including denoising, contrast enhancement, image stabilization, etc., to improve the accuracy and efficiency of subsequent image analysis. From the preprocessed third surveillance video, feature images are extracted at certain time intervals (such as extracting one frame every 5 seconds). These feature images will be used for subsequent subway vehicle and drone detection. For each frame of feature image, the monitoring platform uses an image recognition algorithm (such as a target detection algorithm based on deep learning) to detect the protected area along the line to identify whether there is a drone. If the presence of a drone is detected, the status is recorded and the analysis of the next frame of image continues. If a drone is detected in multiple consecutive frames of images, the presence of the drone can be further confirmed.

[0120] S302: If the monitoring platform detects the presence of a drone in the protected area along the line in at least one characteristic image in the third monitoring video, the monitoring platform determines that the safety level of the protected area along the line is the first safety level.

[0121] If the monitoring platform detects a drone in the protected area in at least one of the feature images in the third surveillance video, this indicates a potential safety risk. For example, the drone may enter a prohibited flight zone or collide with a subway vehicle. In this case, the protected area is assigned a safety level of Level 1, indicating the need for immediate emergency measures to ensure the safety of the protected area and subway vehicles.

[0122] S303: If the monitoring platform does not detect any drone in the protected area along the line in any characteristic image in the third monitoring video, the security level of the protected area along the line is determined to be the second security level.

[0123] If the monitoring platform detects no drones in any of the characteristic images in the third surveillance video, the protected area is considered relatively safe. At this point, the safety level of the protected area is determined to be Level 2, indicating that the situation is relatively stable, but vigilance is still required to prevent any emergencies.

[0124] Depending on the safety level, the monitoring platform will take appropriate monitoring measures. For example, at Safety Level 1, the monitoring platform might immediately notify the subway control center to suspend subway train operations and notify relevant departments to drive away or intercept the drone. At the same time, the monitoring platform will increase the frequency and intensity of monitoring of protected areas along the line to ensure the safety of these areas and subway vehicles. At Safety Level 2, the monitoring platform might appropriately reduce the frequency of monitoring, but will still maintain continuous monitoring of protected areas along the line.

[0125] The monitoring platform also promptly reports the safety level to the subway control center, the management of protected areas along the line, and relevant safety agencies. These agencies can then implement appropriate safety measures or emergency response plans based on this feedback to ensure the safe operation of protected areas and subway vehicles. Furthermore, the monitoring platform continuously optimizes and adjusts its monitoring strategies and methods based on this feedback to improve the overall safety level of protected areas along the line.

[0126] In addition, it is worth mentioning that before determining the safety level of the protection zone along the line as the first safety level, the monitoring platform can also extract the outer contour of the drone and the outer contour of the protection zone along the line from the target feature image. The target feature image is a video frame image including the drone in the monitoring video. The monitoring platform generates a contour feature rectangle based on the outer contour of the drone and determines the contour feature center point of the contour feature rectangle. The contour feature rectangle is the minimum rectangle used to completely accommodate the outer contour of the drone. Then, the monitoring platform uses formula 1 and calculates the contour feature rectangle B and the contour feature center point B according to the contour feature rectangle B and the contour feature center point B. mid and the outer contour P of the protected area along the line edgeThe corresponding protection zone P along the line determines the position characteristic value T, where formula 1 is:

[0127]

[0128] Among them, Area(B∩P) is the area of ​​the intersection of the contour feature rectangle B and the protection zone P on the image, Area(B) is the area of ​​the contour feature rectangle B, d(B mid , P edge ) is the center point B of the contour feature mid To the outer contour P of the protection zone along the line edge The minimum distance is α, which is the first weight value and β, which is the second weight value. If the monitoring platform determines that the location feature value T is greater than the preset location feature threshold, it determines that the drone is located in the protected area along the line.

[0129] Specifically, after the monitoring platform receives the third monitoring video, it first filters out the video frame images containing the drone as the target feature image. These images are the basis for the subsequent extraction of the outer contour of the drone and the outer contour of the protected area along the line. The outer contour of the drone is extracted from the target feature image using image segmentation or edge detection algorithms (such as the Canny edge detection algorithm). This step is the key to identifying the specific position of the drone in the image. Based on the pre-set boundary information of the protected area along the line (which may come from map data, GIS system, or positioning based on tracks and pre-set boundary feature points of the protected area along the line), the outer contour of the protected area along the line is drawn in the target feature image. This step is the basis for determining the relative position of the drone and the protected area along the line.

[0130] Then, based on the outer contour of the drone, the contour feature rectangle is obtained by calculating its minimum circumscribed rectangle (that is, the minimum rectangle that can completely accommodate the outer contour of the drone). This step simplifies the subsequent calculation process while retaining the key position information of the drone in the image. The center point of the contour feature rectangle is then calculated using the contour moment method. This step provides a key reference point for the subsequent calculation of the position feature value. Then, according to Formula 1, the monitoring platform calculates the position feature value of the drone in the image. This formula comprehensively considers factors such as the area of ​​the intersection between the drone and the protected area along the line, the area of ​​the contour feature rectangle, and the minimum distance between the center point of the contour feature and the outer contour of the protected area along the line.

[0131] Among them, for The part represents the ratio of the intersection area of ​​the outline feature rectangle and the protected area along the line in the image to the area of ​​the outline feature rectangle. It reflects the degree of overlap between the drone and the protected area along the line. The higher the ratio, the closer the drone is to or has entered the protected area along the line. The reciprocal of the minimum distance between the center point of the contour feature and the outer contour of the protected area along the line is shown in the figure. It reflects the relative distance between the drone and the protected area along the line. The closer the distance, the larger the reciprocal, indicating that the drone is closer to the protected area along the line.

[0132] Next, two weights are set to adjust the contribution of the intersection area ratio and the inverse distance to the location feature value calculation. Adjusting these two parameters allows for flexible adjustment of drone position determination in different application scenarios. The monitoring platform compares the calculated location feature value with a preset location feature threshold. If the location feature value exceeds the preset threshold, the drone is determined to be within the protected area along the route; otherwise, the drone is considered to have not entered the protected area along the route.

[0133] It is worth noting that in this solution, since the images taken by the subway vehicle are dynamic images, that is, the images taken will change as the subway vehicle moves, and the images taken are two-dimensional images, it is impossible to directly determine the relative position relationship between the drone and the protected area along the line in the image, and it is also impossible to use Figure 1 In the illustrated embodiment, the relative positions of the unmanned areas and the protection areas along the line are determined by marking the protection areas along the line in the image.

[0134] Therefore, through the above scheme, video frames containing the drone are extracted from the surveillance video as target feature images. Image segmentation or edge detection algorithms are then used to precisely extract the drone's outer contour. The outer contour of the protected area is then drawn based on the pre-defined boundary information for the protected area along the line, achieving accurate identification of the drone and the protected area boundary. This step provides highly accurate basic data for subsequent position determination. Next, the minimum enclosing rectangle (the contour feature rectangle) of the drone's outer contour is calculated, simplifying the subsequent position calculation process. This step not only preserves the drone's key position information in the image but also significantly reduces computational complexity and improves processing efficiency. The center point of the contour feature rectangle is then calculated using the contour moment method, providing a precise reference point for the subsequent calculation of the position feature value. This step is crucial for accurately determining the relative position of the drone and the protected area along the line. The position feature value is then calculated using Formula 1, which comprehensively considers multiple factors, including the area of ​​the intersection between the drone and the protected area along the line, the area of ​​the contour feature rectangle, and the minimum distance between the center point of the contour feature and the outer contour of the protected area along the line, ensuring accurate and comprehensive position determination. Furthermore, the weights in Formula 1 can be flexibly adjusted based on the actual application scenario. By adjusting these two parameters, precise control of drone position determination can be achieved in different situations, improving the system's adaptability and flexibility. High-precision position determination: By comparing the calculated position feature value with the preset position feature threshold, it can accurately determine whether a drone has entered the protected area along the subway line. This step provides an important basis for subsequent safety level determination and emergency response, effectively improving the safety monitoring level of the protected area along the subway line.

[0135] Based on the above embodiments, after the monitoring platform determines the safety level of the protection zone along the track based on the subway vehicle detection status in the track running area in the monitoring video and / or the drone detection status in the protection zone along the track in the monitoring video, the following further steps are included:

[0136] If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the UAV, wherein the safety adjustment instruction is used to instruct the UAV to fly in a direction away from the track driving area.

[0137] In this solution, after determining the safety level of protected areas along the route through the monitoring platform's monitoring of subway vehicles and drones, a safety adjustment instruction mechanism was introduced. This mechanism is designed to quickly instruct drones to take appropriate measures when safety risks are detected, effectively preventing accidents.

[0138] Figure 4 FIG. 1 is a schematic diagram of a subway line monitoring system according to an exemplary embodiment of the present application. Figure 4As shown, the subway line monitoring system 400 provided in this embodiment includes: a camera cluster 410 and a monitoring platform 420, and each camera in the camera cluster 410 is respectively communicated with the monitoring platform 420;

[0139] The cameras in the camera cluster 410 are used to obtain surveillance videos of the subway line, and the surveillance videos are sent to the monitoring platform 420 . The subway line includes the track running area and the protection area along the line.

[0140] The monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0141] Optionally, the camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line;

[0142] The monitoring platform 420 extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video;

[0143] The monitoring platform 420 determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence.

[0144] Optionally, the monitoring platform 420 determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence, including:

[0145] If the monitoring platform 420 detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0146] If the monitoring platform 420 does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the second safety level, and the first safety level is lower than the second safety level.

[0147] Optionally, after the monitoring platform 420 extracts the first feature image sequence from the first monitoring video, the process further includes:

[0148] The monitoring platform 420 determines a second feature image sequence from the first monitoring video according to the first feature image sequence, where the second feature image sequence is a feature image sequence that is extended in at least one of time advance and time delay based on the first feature image sequence;

[0149] Correspondingly, the monitoring platform 420 determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0150] If the monitoring platform 420 detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level;

[0151] If the monitoring platform 420 does not detect the drone in any feature image in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protected area along the line is the second safety level, and the first safety level is lower than the second safety level;

[0152] If the monitoring platform 420 detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, then the safety level of the protected area along the line is determined to be a third safety level, which is lower than the second safety level and higher than the first safety level.

[0153] If the monitoring platform 420 does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the fourth safety level, which is higher than the second safety level.

[0154] Optionally, the camera is a second camera, which is provided on the drone, and the monitoring video is a second monitoring video captured when the drone is located in the line protection zone. Accordingly, the monitoring platform 420 determines the safety level of the line protection zone based on the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the line protection zone in the monitoring video, including:

[0155] If the monitoring platform 420 detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, then the safety level of the protection zone along the track is determined to be the first safety level;

[0156] If the monitoring platform 420 does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0157] Optionally, the camera is a third camera, which is installed on the subway vehicle, and the monitoring video is a third monitoring video captured when the subway vehicle is located in the track travel area. Correspondingly, the monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track travel area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0158] If the monitoring platform 420 detects the presence of a drone in the protected area along the line in at least one characteristic image in the third monitoring video, determining that the security level of the protected area along the line is the first security level;

[0159] If the monitoring platform 420 does not detect the presence of a drone in the protection zone along the line in any of the feature images in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0160] Optionally, after the monitoring platform 420 determines the safety level of the protection zone along the track according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the track in the monitoring video, the method further includes:

[0161] If the monitoring platform 420 determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform 420 sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

[0162] Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 5 As shown, this embodiment provides an electronic device 500 including: a processor 501 and a memory 502; wherein:

[0163] The memory 502 is used to store computer programs, and the memory may also be a flash memory.

[0164] The processor 501 is configured to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0165] Optionally, the memory 502 may be independent or integrated with the processor 501 .

[0166] When the memory 502 is a device independent of the processor 501, the electronic device 500 may further include:

[0167] The bus 503 is used to connect the memory 502 and the processor 501 .

[0168] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the various aforementioned embodiments.

[0169] This embodiment further provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor can execute the computer program to cause the electronic device to implement the methods provided in the various embodiments described above.

[0170] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0171] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method for a protected area along a subway line, characterized in that: The method is applied to a subway line monitoring system, the subway line monitoring system comprising: a camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication; the method comprises: Acquire surveillance videos of a subway line range through cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line range includes a track running area and a protection area along the line; The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video; The camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line; The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video; The monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence; including: If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level; Before determining that the safety level of the protection zone along the line is the first safety level, the method further includes: The monitoring platform extracts the outer contour of the drone from the target feature image, generates an outline feature rectangle based on the outer contour of the drone, and determines the outline feature center point of the outline feature rectangle, where the outline feature rectangle is the minimum rectangle for completely accommodating the outer contour of the drone. The target feature image is a video frame image including the drone in the monitoring video. When the monitoring platform determines that the center point of the contour feature is within the calibrated area of ​​the target feature image, it determines that the UAV is located in the protected area along the line; If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the second safety level, and the first safety level is lower than the second safety level.

2. The image processing method for a protected area along a subway line according to claim 1, characterized in that: After the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes: The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one of time advance and time delay based on the first feature image sequence; Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including: If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level; If the monitoring platform does not detect the drone in any of the feature images in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protected area along the line is the second safety level, and the first safety level is lower than the second safety level; If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, then the safety level of the protected area along the line is determined to be a third safety level, which is lower than the second safety level and higher than the first safety level. If the monitoring platform does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, the safety level of the protection zone along the line is determined to be the fourth safety level, which is higher than the second safety level.

3. The image processing method for a protected area along a subway line according to claim 1, characterized in that: The camera is a second camera, which is installed on the drone. The monitoring video is a second monitoring video captured by the drone when it is located in the protected area along the line. Correspondingly, the monitoring platform determines the safety level of the protected area along the line based on the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protected area along the line in the monitoring video, including: If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, determining that the safety level of the protection zone along the track is the first safety level; If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

4. The image processing method for a subway protection zone according to claim 1, characterized in that: The camera is a third camera, which is installed on the subway vehicle. The monitoring video is a third monitoring video captured when the subway vehicle is located in the track travel area. Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track travel area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including: If the monitoring platform detects the presence of a drone in the protected area along the line in at least one characteristic image in the third monitoring video, determining that the safety level of the protected area along the line is the first safety level; If the monitoring platform does not detect the presence of a drone in the protection zone along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

5. The image processing method for a subway protection zone according to any one of claims 1 to 3, characterized in that: After the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, the method further includes: If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

6. A subway line monitoring system, characterized in that: include: A camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication; Acquire surveillance videos of a subway line range through cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line range includes a track running area and a protection area along the line; The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video; The camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line; The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video; The monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence; including: If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, the safety level of the protected area along the line is determined to be the first safety level; Before determining that the safety level of the protection zone along the line is the first safety level, the method further includes: The monitoring platform extracts the outer contour of the drone from the target feature image, generates an outline feature rectangle based on the outer contour of the drone, and determines the outline feature center point of the outline feature rectangle, where the outline feature rectangle is the minimum rectangle for completely accommodating the outer contour of the drone. The target feature image is a video frame image including the drone in the monitoring video. When the monitoring platform determines that the center point of the contour feature is within the calibrated area of ​​the target feature image, it determines that the UAV is located in the protected area along the line; If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the second safety level, and the first safety level is lower than the second safety level.

7. An electronic device, characterized in that: include: processor; as well as, a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

Images