Image processing method and system for protection area along subway

Abstract

The invention provides an image processing method and system for a protection area along a subway. According to the method, through the cameras in the camera cluster, the monitoring videos along the subway can be acquired in real time. The video data comprises the running condition of the metro vehicle in the track running area and the dynamic information of the protection area along the line. And the monitoring platform performs intelligent detection and identification on the metro vehicle and / or the unmanned aerial vehicle in the monitoring video so as to find potential safety hazards in time.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to an image processing method and system for a protected area along a subway line. Background Art

[0002] With the acceleration of urbanization, subways, as an important part of public transportation, have received more and more attention for their safety and operational efficiency. As an important part of subway system safety, the management of subway protection areas is crucial to ensure the normal operation of subway vehicles and the safety of passengers. Traditionally, the monitoring of subway protection areas mainly relies on manual patrols and fixed-point monitoring, which are not only inefficient, but also difficult to achieve real-time monitoring of large areas.

[0003] With the rapid development of video surveillance technology, camera clusters are widely used in monitoring systems along subway lines. These cameras can capture images and video data along the subway line in real time, providing monitoring personnel with intuitive monitoring images. However, relying solely on the raw video data captured by the camera, it is difficult for monitoring personnel to quickly and accurately identify potential safety hazards, especially when it comes to dynamically changing elements such as the operating status of subway vehicles and flying objects such as drones that may appear in the protected areas along the line. Summary of the invention

[0004] The present application provides an image processing method and system for protection areas along subway lines. According to the detection status of subway vehicles and / or drones in monitoring videos, the safety level of the protection areas along the lines is comprehensively evaluated and determined, so as to timely discover potential safety hazards.

[0005] In a first aspect, the present application provides an image processing method for a subway line protection zone, which is applied to a subway line monitoring system, wherein the subway line monitoring system comprises: a camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication; the method comprises:

[0006] Acquire surveillance videos along the subway line through the cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line includes the track running area and the protection area along the line;

[0007] The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0008] In the above scheme, the monitoring video of the subway line can be obtained in real time through the cameras in the camera cluster. These video data contain the running status of the subway vehicles in the track driving area and the dynamic information of the protection area along the line. The monitoring platform performs intelligent detection and identification of the subway vehicles and / or drones in the monitoring video. By detecting the subway vehicles in the track driving area, the running status of the subway vehicles can be judged; and by detecting the drones in the protection area along the line, potential safety hazards can be discovered in time to avoid the drones from patrolling in the protection area along the line close to the subway vehicles during the driving of the subway vehicles. According to the detection status of the subway vehicles and drones in the monitoring video, the monitoring platform can comprehensively evaluate and determine the safety level of the protection area along the line. When the monitoring platform determines that the safety level of the protection area along the line is high risk or medium risk, it can send early warning information to the relevant departments in time and take corresponding countermeasures. For example, in the case of drone intrusion, the monitoring platform directly sends a safety adjustment instruction to the drone, or the monitoring platform first sends the safety adjustment instruction to the monitoring device (including the above-mentioned camera) set at the line monitoring position, and then sends it to the drone through the communication module on the monitoring device to instruct it to fly away from the track driving area.

[0009] Optionally, the camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line;

[0010] The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video;

[0011] The monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence.

[0012] In the above scheme, the first camera is set on a fixed object outside the subway line. This layout strategy can avoid adjusting the subway vehicle itself and the inspection drone itself. Instead, an additional set of devices that do not affect the operation of the original equipment can be set on the basis of the original subway vehicle and inspection drone, so that the system has better scalability and compatibility, which is especially suitable for upgrading and renovating old subway lines.

[0013] Optionally, the monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence, including:

[0014] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0015] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level.

[0016] In the above scheme, the monitoring platform uses the preset target detection model to detect drones in the first feature image sequence (i.e., the video frame image containing the subway vehicle). According to the drone detection results, the monitoring platform divides the security level of the protection area along the line into two levels: if a drone is detected, the security level is the first security level (lower); if no drone is detected, the security level is the second security level (higher). Through clear security level classification standards, a clear and quantifiable security assessment basis is provided for the subway operation management department. Once a drone intrusion is detected (i.e., the security level is the first security level), the monitoring platform can immediately trigger the early warning mechanism and notify the relevant departments to take countermeasures. The monitoring platform can send the security level assessment results and related information (such as drone location, flight trajectory, etc.) to the subway operation management department in real time. The subway operation management department can scientifically formulate safety management measures and emergency plans based on the information provided by the monitoring platform. Through real-time monitoring and rapid response mechanisms, the collaborative work between the subway operation management department and relevant departments is promoted, and the ability to respond to emergencies is improved.

[0017] Optionally, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0018] The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one time direction of time advance and time delay based on the first feature image sequence;

[0019] Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0020] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0021] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level;

[0022] If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is a third safety level, which is lower than the second safety level and higher than the first safety level;

[0023] If the monitoring platform does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the fourth safety level, which is higher than the second safety level.

[0024] In the above scheme, after the monitoring platform extracts the first feature image sequence (i.e., the video frame image containing the subway vehicle) from the first monitoring video, it further expands the sequence in time to generate a second feature image sequence. The second feature image sequence is an extension of the first feature image sequence in at least one of the time directions of time advance and time delay. Through the extension of time advance and / or time delay, the second feature image sequence covers a wider time range, so that the monitoring platform can obtain more historical information and future trends about drone activities. The extended feature image sequence provides more data samples for drone detection, which helps to reduce the possibility of false detection and missed detection and improve the accuracy of detection. The monitoring platform uses a preset target detection model to detect drones on the first feature image sequence and the second feature image sequence, and determines the safety level of the protection zone along the line according to the detection results. Through the dual detection of the first feature image sequence and the second feature image sequence, the monitoring platform can more comprehensively capture the activities of drones in the protection zone along the line, including drones that appear briefly and persist. According to the detection results, the monitoring platform divides the safety level of the protection zone along the line into four levels (first safety level to fourth safety level), and each level corresponds to different drone activities and safety risk levels. If a drone is detected in the first feature image sequence, the security level is the first security level, indicating that there is an obvious security risk. If a drone is not detected in the first feature image sequence, but a drone is detected in the difference between the second feature image sequence and the first feature image sequence, the security level is the third security level, indicating that there is a potential security risk, but the risk level is lower than the first security level. If a drone is not detected in both the first feature image sequence and the second feature image sequence, the security level is the second security level or the fourth security level (depending on the specific detection result of the second feature image sequence), indicating that the security risk is low or very low.

[0025] Optionally, the camera is a second camera, which is arranged on the drone, and the monitoring video is a second monitoring video taken when the drone is located in the protection zone along the line; correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0026] If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, then determining that the safety level of the protection area along the line is the first safety level;

[0027] If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

[0028] In the above scheme, the surveillance video is acquired through the second camera carried by the drone and transmitted to the monitoring platform in real time. Then, the monitoring platform determines the safety level of the protection area along the line according to the detection status of the subway vehicle in the track running area in the second surveillance video. If a subway vehicle is detected in at least one characteristic image, the safety level is determined to be the first safety level, indicating that there may be safety risks related to the operation of subway vehicles in the protection area along the line. If no subway vehicle is detected in any of the characteristic images, the safety level is determined to be the second safety level, indicating that the current safety risk of the protection area along the line is low. The above-mentioned method of acquiring the surveillance video through the second camera carried by the drone can be based on the original camera of the drone, without the need to add other equipment, and the implementation cost is low.

[0029] Optionally, the camera is a third camera, and the third camera is arranged on the subway vehicle. The monitoring video is a third monitoring video taken when the subway vehicle is located in the track driving area. Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track driving area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0030] If the monitoring platform detects the presence of a drone in the protection zone along the line in at least one characteristic image in the third monitoring video, determining that the security level of the protection zone along the line is the first security level;

[0031] If the monitoring platform does not detect the presence of a drone in the protection zone along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0032] In the above scheme, the third camera is set on the subway vehicle. As the subway vehicle travels, its field of view can dynamically cover the protection area along the line, realizing continuous and uninterrupted monitoring. Compared with fixed cameras, the subway vehicle perspective monitoring method can cover the protection area along the line more comprehensively, reduce monitoring blind spots, and improve the continuity and effectiveness of monitoring. The third monitoring video shot by the third camera can be transmitted to the monitoring platform in real time. The platform analyzes the video content and identifies potential threats such as drones in the protection area along the line. The monitoring platform determines the security level according to the drone detection status of the protection area along the line in the third monitoring video. If a drone is detected in at least one feature image, the security level is determined to be the first security level, indicating that there is a high security risk in the protection area along the line. If no drone is detected in any feature image, the security level is determined to be the second security level, indicating that the current security risk of the protection area along the line is low. Through drone detection, the monitoring platform can accurately identify potential threats in the protection area along the line and provide a reliable basis for security risk assessment. The subway vehicle perspective monitoring method significantly enhances the comprehensiveness and real-time nature of monitoring through dynamic field of view coverage, real-time data transmission and instant analysis technology.

[0033] Optionally, after the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, it also includes:

[0034] If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

[0035] In the above scheme, after determining the safety level of the protection zone along the line through the monitoring platform's detection status of subway vehicles and drones, a safety adjustment instruction mechanism was further introduced. This mechanism is designed to quickly guide drones to take corresponding measures when safety risks are detected, thereby effectively avoiding the occurrence of safety accidents.

[0036] In a second aspect, the present application provides a subway line monitoring system, comprising: a camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication;

[0037] Acquire surveillance videos along the subway line through the cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line includes the track running area and the protection area along the line;

[0038] The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0039] Optionally, the camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line;

[0040] The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video;

[0041] The monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence.

[0042] Optionally, the monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence, including:

[0043] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0044] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level.

[0045] Optionally, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0046] The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one time direction of time advance and time delay based on the first feature image sequence;

[0047] Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0048] If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0049] If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level;

[0050] If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is a third safety level, which is lower than the second safety level and higher than the first safety level;

[0051] If the monitoring platform does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the fourth safety level, which is higher than the second safety level.

[0052] Optionally, the camera is a second camera, which is arranged on the drone, and the monitoring video is a second monitoring video taken when the drone is located in the protection zone along the line; correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0053] If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, then determining that the safety level of the protection area along the line is the first safety level;

[0054] If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

[0055] Optionally, the camera is a third camera, and the third camera is arranged on the subway vehicle. The monitoring video is a third monitoring video taken when the subway vehicle is located in the track driving area. Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track driving area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0056] If the monitoring platform detects the presence of a drone in the protection zone along the line in at least one characteristic image in the third monitoring video, determining that the security level of the protection zone along the line is the first security level;

[0057] If the monitoring platform does not detect the presence of a drone in the protection zone along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

[0058] Optionally, after the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, it also includes:

[0059] If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

[0060] In a third aspect, the present application provides an electronic device, including:

[0061] processor; and,

[0062] A memory, configured to store executable instructions of the processor;

[0063] The processor is configured to perform any possible method described in the first aspect by executing the executable instructions.

[0064] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any possible method described in the first aspect.

[0065] The image processing method and system for the protection zone along the subway provided in the present application can obtain the monitoring video of the subway line in real time through the camera in the camera cluster. These video data contain the operation status of the subway vehicle in the track driving area and the dynamic information of the protection zone along the line. The monitoring platform performs intelligent detection and identification of the subway vehicles and / or drones in the monitoring video. By detecting the subway vehicles in the track driving area, the operation status of the subway vehicles can be judged; and by detecting the drones in the protection zone along the line, potential safety hazards can be discovered in time to avoid the drones patrolling in the protection zone along the line close to the subway vehicles during the driving of the subway vehicles. According to the detection status of the subway vehicles and drones in the monitoring video, the monitoring platform can comprehensively evaluate and determine the safety level of the protection zone along the line. When the monitoring platform determines that the safety level of the protection zone along the line is high risk or medium risk, it can send early warning information to the relevant departments in time and take corresponding countermeasures. For example, in the event of a drone intrusion, the monitoring platform directly sends a safety adjustment instruction to the drone, or the monitoring platform first sends the safety adjustment instruction to a monitoring device (including the above-mentioned camera) set at the line monitoring position, and then sends it to the drone through the communication module on the monitoring device, to instruct it to fly in a direction away from the track driving area. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1 is a flowchart of an image processing method for a protected area along a subway line according to an exemplary embodiment of the present application;

[0068] Figure 2 is a flowchart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application;

[0069] Figure 3 is a flowchart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application;

[0070] Figure 4 is a schematic structural diagram of a subway line monitoring system according to an exemplary embodiment of the present application;

[0071] Figure 5 It is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application.

[0072] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0073] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0074] In order to solve the above problems, the embodiment provided in this application realizes comprehensive coverage and real-time monitoring of the subway line by deploying camera clusters at different locations along the subway line. These cameras include fixed cameras set outside the subway line, mobile cameras set on drones, and on-board cameras set on subway vehicles. They can obtain monitoring videos from different perspectives and positions, providing a rich data source for subsequent image processing and analysis.

[0075] The monitoring platform intelligently identifies and analyzes the received surveillance videos. First, the subway vehicles and drones in the video are identified through target detection algorithms (such as convolutional neural networks in deep learning). According to the detection status of the drone in the protection zone along the line, the monitoring platform evaluates the safety level of the protection zone along the line. If a drone is detected entering the protection zone along the line, it is judged as a high-risk level and the emergency response mechanism is immediately triggered, such as sending a safety adjustment instruction to the drone, instructing it to fly away from the track driving area to avoid safety accidents such as collisions with subway vehicles.

[0076] Figure 1 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to an exemplary embodiment of the present application. Figure 1 As shown, the method provided in this embodiment includes:

[0077] S101. Obtain surveillance video along the subway line through cameras in the camera cluster.

[0078] In this step, a first surveillance video of the subway line is obtained by a first camera in the camera cluster, and the first surveillance video is sent to the monitoring platform, where the subway line includes the track running area and the line protection area. The first camera is set on a fixed object outside the subway line.

[0079] Specifically, the first camera can be installed on a fixed object outside the subway line, such as a high-rise building, a lighthouse, or a specially installed monitoring tower. These fixed objects should have high stability and a good field of view to ensure that images of the subway line (including the track running area and the protection area along the line) can be clearly captured. The first camera starts working, capturing the monitoring video of the subway line in real time, and sending it to the monitoring platform as the first monitoring video. The first monitoring video should have high resolution and real-time performance so that the monitoring platform can accurately analyze and process it.

[0080] S102: The monitoring platform extracts a first feature image sequence from a first monitoring video.

[0081] In this step, the monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in a track travel area in the first monitoring video.

[0082] Specifically, after receiving the first monitoring video, the monitoring platform uses a computer vision processing device (such as a GPU acceleration server) to process the video. The monitoring platform uses advanced video processing technologies, such as frame difference method, background subtraction, motion detection algorithm, superior reading learning algorithm, image segmentation algorithm and feature matching algorithm, to automatically extract video frame images of subway vehicles in the track travel area from the first monitoring video. These video frame images constitute a first feature image sequence. To ensure accuracy, the monitoring platform may also use additional screening mechanisms, such as image quality assessment, noise removal, etc., to further optimize the first feature image sequence.

[0083] S103: The monitoring platform determines the safety level of the protection area along the line according to the preset target detection model and the first feature image sequence.

[0084] On the monitoring platform, a target detection model is pre-trained and deployed. The model is built based on a deep learning framework (such as TensorFlow, PyTorch, etc.) and is annotated and trained with a large number of drone images to achieve accurate detection of drone targets. The monitoring platform inputs the first feature image sequence into the preset target detection model, and the model analyzes each feature image to detect whether there is a drone target.

[0085] If the model detects a drone in at least one feature image in the first feature image sequence, the monitoring platform determines that the safety level of the protection area along the line is the first safety level (a lower safety level), indicating that there is a potential safety risk.

[0086] If the model does not detect a drone in any of the feature images in the first feature image sequence, the monitoring platform determines that the safety level of the protection area along the line is the second safety level (a higher safety level), indicating that the current safety situation is good.

[0087] Furthermore, before determining that the safety level of the protection zone along the line is the first safety level, the monitoring platform may also extract the outer contour of the drone in the target feature image, generate an outline feature rectangle based on the outer contour of the drone, and determine the outline feature center point of the outline feature rectangle, where the outline feature rectangle is the minimum rectangle for completely accommodating the outer contour of the drone, and the target feature image is a video frame image including the drone in the monitoring video. If the monitoring platform determines that the outline feature center point is within the calibrated area in the target feature image, it is determined that the drone is located in the protection zone along the line.

[0088] Specifically, the monitoring platform selects video frame images containing drones from the first feature image sequence. These images are called target feature images. The screening process may be based on a preset drone detection model or simple image analysis technology, such as color, shape and other feature matching. This step mainly relies on the high-performance computing and storage capabilities of the monitoring platform to ensure that a large number of video frame images can be processed quickly and accurately.

[0089] For each target feature image, the monitoring platform uses an edge detection algorithm (such as Canny edge detection) to extract the outer contour of the drone. The edge detection algorithm identifies the edge contour of the object by analyzing the pixel gradient changes in the image. Before edge detection, the image may need to be preprocessed, such as grayscale, denoising, etc., to improve the accuracy of edge detection. In addition, in order to extract the drone contour more accurately, morphological operations (such as dilation and erosion) can be used to optimize the edge detection results.

[0090] After extracting the outer contour of the drone, the monitoring platform uses the minimum enclosing rectangle algorithm to generate the contour feature rectangle. The algorithm calculates the minimum enclosing rectangle of all points on the contour to obtain the minimum rectangle that completely contains the outer contour of the drone. The implementation of the minimum enclosing rectangle algorithm can be based on relevant functions in image processing libraries (such as OpenCV). By calling these functions, the position and size of the contour feature rectangle can be quickly and accurately calculated.

[0091] The monitoring platform uses the contour moment method to determine the center point of the contour feature rectangle. Contour moment is a mathematical tool to describe the shape of an image. By calculating the zero-order moment and first-order moment of the contour, the coordinates of the center point of the contour can be obtained.

[0092] The monitoring platform predefines a calibration area in the target feature image, which is usually determined based on the actual location and size of the subway protection zone. Then, the monitoring platform compares the coordinates of the center point of the contour feature with the boundary of the calibration area to determine whether it is within the calibration area. If the coordinates of the center point of the contour feature are within the calibration area, the monitoring platform determines that the drone is in the protection zone; otherwise, it is considered that the drone is not in the protection zone.

[0093] In the above scheme, the edge detection algorithm can accurately identify the contour boundary of the drone by calculating the change of pixel gradient in the image. Among them, the edge detection algorithm has a fast calculation speed and can process a large amount of image data in a short time to meet the needs of real-time monitoring. According to the step of generating the contour feature rectangle of the drone outer contour, a minimum rectangle that tightly surrounds the drone contour can be generated by the minimum circumscribed rectangle algorithm. The minimum circumscribed rectangle can tightly surround the drone contour, reduce the redundant area, and provide a more accurate target area for subsequent steps. By generating a unified minimum circumscribed rectangle, drones of different sizes can be standardized, which is convenient for subsequent feature extraction and position judgment. Then, by calculating the geometric center of the rectangle, the precise image position coordinates of the drone can be obtained. In the case of drone posture changes or image rotation, the center point coordinates can still remain stable, which improves the robustness of the system. By comparing the position of the contour feature center point with the boundary of the calibration area, the step of judging whether the drone is located in the protection area along the line is realized, and the accurate judgment of the drone position is realized. In addition, the scope and position of the calibration area can be adjusted according to the actual situation to meet the protection needs of different subway lines. It is worth noting that, for the above-mentioned solution in which the first camera is set on a fixed object, especially in the scenario in which the shooting angle of the first camera is also fixed, the computational complexity of image processing can be greatly reduced by calibrating the area range in the target feature image, thereby improving the recognition speed.

[0094] In this embodiment, surveillance videos of the subway line are acquired in real time through the cameras in the camera cluster. Then, the monitoring platform performs intelligent detection and identification of the subway vehicles and / or drones in the surveillance video, so as to comprehensively evaluate and determine the safety level of the protection zone along the line according to the identification results and the detection status of the subway vehicles and drones in the surveillance video, thereby timely discovering potential safety hazards.

[0095] exist Figure 1 On the basis of the embodiment shown, after the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes:

[0096] The monitoring platform determines a second characteristic image sequence from the first monitoring video according to the first characteristic image sequence, wherein the second characteristic image sequence is a characteristic image sequence that is extended in at least one time direction of time advance and time delay based on the first characteristic image sequence;

[0097] Correspondingly, the monitoring platform determines the safety level of the protection area along the line according to the preset target detection model and the first feature image sequence, including:

[0098] If the monitoring platform detects the drone in at least one feature image in the first feature image sequence using the preset target detection model, the safety level of the protection area along the line is determined to be the first safety level;

[0099] If the monitoring platform does not detect the drone in any of the feature images in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level;

[0100] If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using the preset target detection model, the safety level of the protection area along the line is determined to be the third safety level, which is lower than the second safety level and higher than the first safety level;

[0101] If the monitoring platform does not detect a drone in any of the feature images in the second feature image sequence using the preset target detection model, the safety level of the protection area along the line is determined to be the fourth safety level, which is higher than the second safety level.

[0102] In the above scheme, after the monitoring platform extracts the first feature image sequence (i.e., the video frame image containing the subway vehicle) from the first monitoring video, it further expands the sequence in time to generate a second feature image sequence. The second feature image sequence is an extension of the first feature image sequence in at least one of the time directions of time advance and time delay. Through the extension of time advance and / or time delay, the second feature image sequence covers a wider time range, so that the monitoring platform can obtain more historical information and future trends about drone activities. The extended feature image sequence provides more data samples for drone detection, which helps to reduce the possibility of false detection and missed detection and improve the accuracy of detection. The monitoring platform uses a preset target detection model to detect drones on the first feature image sequence and the second feature image sequence, and determines the safety level of the protection zone along the line according to the detection results. Through the dual detection of the first feature image sequence and the second feature image sequence, the monitoring platform can more comprehensively capture the activities of drones in the protection zone along the line, including drones that appear briefly and persist. According to the detection results, the monitoring platform divides the safety level of the protection zone along the line into four levels (first safety level to fourth safety level), and each level corresponds to different drone activities and safety risk levels. If a drone is detected in the first feature image sequence, the security level is the first security level, indicating that there is an obvious security risk. If a drone is not detected in the first feature image sequence, but a drone is detected in the difference between the second feature image sequence and the first feature image sequence, the security level is the third security level, indicating that there is a potential security risk, but the risk level is lower than the first security level. If a drone is not detected in both the first feature image sequence and the second feature image sequence, the security level is the second security level or the fourth security level (depending on the specific detection result of the second feature image sequence), indicating that the security risk is low or very low.

[0103] Figure 2 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application. Figure 2 As shown, the image processing method for the protection area along the subway provided in this embodiment includes:

[0104] S201. Obtain surveillance video along the subway line through cameras in the camera cluster.

[0105] In this step, the second monitoring video of the subway line is obtained through the second camera in the camera cluster, and the second monitoring video is sent to the monitoring platform. The subway line includes the track driving area and the protection area along the line. Among them, the second camera is set on the drone, and the second monitoring video is the monitoring video taken when the drone is in the protection area along the line. It is worth noting that in this embodiment, the drone is a patrol drone, which is connected to the monitoring platform in communication, and while uploading the monitoring video, it also reports the location information in real time.

[0106] Specifically, the second camera can be a high-definition, wide-angle camera with night vision function to ensure that the image of the protected area along the subway line can be clearly captured under different lighting conditions. The camera is firmly mounted on the drone to ensure that its field of view can fully cover the track running area and the protected area along the line, while avoiding itself becoming an obstacle in flight.

[0107] The drone can fly according to the preset route to ensure full coverage of the protected area along the subway line. During the flight, the drone maintains a stable flight altitude and speed to obtain high-quality second monitoring video. The captured second monitoring video is transmitted to the monitoring platform in real time via wireless communication technology. At the same time, in order to ensure the integrity and traceability of the data, the monitoring video will also be stored in the drone's local storage medium for subsequent analysis.

[0108] After receiving the second surveillance video, the monitoring platform first preprocesses it, including denoising, contrast enhancement and other operations, to improve the accuracy and efficiency of subsequent image analysis. From the preprocessed second surveillance video, feature images are extracted at a certain time interval (such as extracting one frame per second). These feature images are used for subsequent subway vehicle and drone detection. For each frame of feature image, the monitoring platform uses advanced image recognition algorithms (such as convolutional neural networks in deep learning) to detect the track driving area. If the presence of a subway vehicle is detected, the state is recorded and the next frame of image analysis continues. If a subway vehicle is detected in multiple consecutive frames of images, the presence of the subway vehicle can be further confirmed. At the same time, the monitoring platform will also detect the drone itself in the protected area along the line. This is to ensure that the drone does not violate any safety regulations during the flight, such as entering a prohibited flight area or colliding with other flying objects. If an abnormal state of the drone is detected (such as deviation from the route, abnormal flight altitude, etc.), the state will also be recorded and corresponding safety measures will be taken.

[0109] S202: If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, it is determined that the safety level of the protection area along the line is the first safety level.

[0110] Specifically, if the monitoring platform detects the presence of a subway vehicle in the track driving area in at least one characteristic image of the second monitoring video, it means that the protection area along the line may currently be in a state of subway vehicle operation. At this time, the safety level of the protection area along the line is determined to be the first safety level, that is, there is a risk. At this time, the drone should be controlled to quickly evacuate the area to prevent any incidents that may endanger the safety of subway vehicle operation.

[0111] S203: If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any of the feature images in the second monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

[0112] If the monitoring platform does not detect the presence of subway vehicles in the track driving area in any of the feature images of the second monitoring video, it may mean that the current protection area along the line is not in the operation period of subway vehicles or subway vehicles have not entered the area temporarily. At this time, the security level of the protection area along the line is determined to be the second security level, indicating that the security situation of the protection area along the line is relatively stable, but vigilance is still required to prevent any emergencies.

[0113] In the above scheme, the monitoring video is acquired by the second camera carried by the drone and transmitted to the monitoring platform in real time. Then, the monitoring platform determines the safety level of the protection zone along the line according to the detection status of the subway vehicle in the track driving area in the second monitoring video. If the subway vehicle is detected in at least one characteristic image, the safety level is determined to be the first safety level, indicating that there may be safety risks related to the operation of the subway vehicle in the protection zone along the line. If the subway vehicle is not detected in any characteristic image, the safety level is determined to be the second safety level, indicating that the current safety risk of the protection zone along the line is low. The above method of acquiring the monitoring video by the second camera carried by the drone can be based on the original camera of the drone, without adding other equipment, and the implementation cost is low. It is worth noting that the above scheme is particularly suitable for the application scenario where the subway vehicle monitoring system and the subway line inspection system are two independent entities, and the data of the two cannot be interoperable. In this scenario, the operation priority of the subway vehicle is higher than the priority of the subway line inspection. However, since the background data of the two cannot be interoperable and shared, it is impossible to use the position data of both parties at the same time for direct relative position judgment.

[0114] Figure 3 FIG. 1 is a flow chart of an image processing method for a protected area along a subway line according to another exemplary embodiment of the present application. Figure 3 As shown, the image processing method for the protection area along the subway provided in this embodiment includes:

[0115] S301. Obtain surveillance video of the subway line through cameras in the camera cluster.

[0116] In this step, the third surveillance video of the subway line is obtained through the third camera in the camera cluster, and the third surveillance video is sent to the monitoring platform. The subway line includes the track running area and the protection area along the line. Among them, the second camera is set on the subway vehicle, and the third surveillance video is the third surveillance video taken when the subway vehicle is in the track running area. It is worth noting that in this embodiment, the subway vehicle is connected to the monitoring platform in communication, and while uploading the surveillance video, it also reports the location information in real time, while the above-mentioned drone is a drone of a third-party platform and cannot directly share data with the monitoring platform. However, optionally, the monitoring platform can be provided with a function of broadcasting an expulsion signal to the drone on a common frequency band.

[0117] Specifically, the third camera is installed at the front or top of the subway vehicle, and uses a high-definition, anti-shake, and night-vision camera, or can directly use the subway vehicle's own recorder camera. Its installation position ensures that it can clearly capture the images of the protected areas along the line in front of and on both sides of the subway vehicle. The camera can capture dynamic images of the protected areas along the line in real time as the subway vehicle travels.

[0118] The captured third surveillance video is transmitted to the monitoring platform in real time through the on-board wireless communication device. In order to ensure the real-time and accuracy of the data, efficient data compression and encryption technology is used in the transmission process to prevent data loss or tampering. In addition to real-time transmission, the third surveillance video will also be stored in the local storage medium of the subway vehicle for subsequent analysis. At the same time, the monitoring platform will also regularly back up and store the received surveillance video to ensure the integrity and traceability of the data.

[0119] After receiving the third surveillance video, the monitoring platform first performs preprocessing operations on it, including denoising, contrast enhancement, image stabilization, etc., to improve the accuracy and efficiency of subsequent image analysis. From the preprocessed third surveillance video, feature images are extracted at certain time intervals (such as extracting one frame every 5 seconds). These feature images will be used for subsequent subway vehicle and drone detection. For each frame of feature image, the monitoring platform uses an image recognition algorithm (such as a target detection algorithm based on deep learning) to detect the protected areas along the line to identify whether there are drones. If the presence of a drone is detected, the status is recorded and the next frame of the image is analyzed. If a drone is detected in multiple consecutive frames of images, the presence of the drone can be further confirmed.

[0120] S302: If the monitoring platform detects the presence of a drone in the protection zone along the line in at least one characteristic image in the third monitoring video, it determines that the safety level of the protection zone along the line is the first safety level.

[0121] If the monitoring platform detects the presence of a drone in the line protection area in at least one characteristic image of the third monitoring video, it means that there may be a safety risk in the current line protection area. For example, the drone may enter a prohibited flight area or there is a possibility of collision with a subway vehicle. At this time, the safety level of the line protection area is determined to be the first safety level, indicating that emergency measures need to be taken immediately to ensure the safety of the line protection area and subway vehicles.

[0122] S303: If the monitoring platform does not detect any drone in the protection area along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

[0123] If the monitoring platform does not detect the presence of drones in any of the feature images of the third monitoring video, it means that the current protection area along the line is in a relatively safe state. At this time, the security level of the protection area along the line is determined to be the second security level, indicating that the security situation of the protection area along the line is relatively stable, but vigilance is still required to prevent any emergencies.

[0124] According to different security levels, the monitoring platform will take corresponding monitoring measures. For example, at the first security level, the monitoring platform may immediately notify the subway control center to suspend the operation of subway vehicles and notify relevant departments to drive away or intercept drones. At the same time, the monitoring platform will increase the monitoring frequency and intensity of the protection areas along the line to ensure the safety of the protection areas along the line and subway vehicles. At the second security level, the monitoring platform may appropriately reduce the monitoring frequency, but it still needs to maintain continuous attention to the protection areas along the line.

[0125] The monitoring platform will also promptly feedback the determined safety level to the subway control center, the management department of the protection area along the line, and relevant safety agencies. These departments can take corresponding safety measures or emergency plans based on the feedback information to ensure the safe operation of the protection area along the line and subway vehicles. At the same time, the monitoring platform will continuously optimize and adjust its own monitoring strategies and methods based on the feedback information to improve the overall safety level of the protection area along the line.

[0126] In addition, it is worth noting that before determining that the safety level of the protection zone along the line is the first safety level, the monitoring platform can also extract the outer contour of the drone and the outer contour of the protection zone along the line in the target feature image. The target feature image is a video frame image including the drone in the monitoring video. The monitoring platform generates a contour feature rectangle based on the outer contour of the drone and determines the contour feature center point of the contour feature rectangle. The contour feature rectangle is the minimum rectangle used to completely accommodate the outer contour of the drone. Then, the monitoring platform uses formula 1 and calculates the contour feature rectangle B based on the contour feature rectangle B and the contour feature center point B. mid And the outer contour of the protected area along the line P edgeThe corresponding protection zone range P along the line determines the position characteristic value T, where formula 1 is:

[0127]

[0128] Among them, Area(B∩P) is the area of ​​the intersection of the contour feature rectangle B and the protection area P along the line on the image, Area(B) is the area of ​​the contour feature rectangle B, and d(B mid , P edge ) is the center point B of the contour feature mid To the outer contour P of the protection area along the line edge The minimum distance, α is the first weight value, and β is the second weight value. If the monitoring platform determines that the location feature value T is greater than the preset location feature threshold, it is determined that the drone is located in the protection zone along the line.

[0129] Specifically, after the monitoring platform receives the third monitoring video, it first screens out the video frame images containing the drone as the target feature image. These images are the basis for the subsequent extraction of the outer contour of the drone and the outer contour of the protected area along the line. The outer contour of the drone is extracted from the target feature image using image segmentation or edge detection algorithms (such as the Canny edge detection algorithm). This step is the key to identifying the specific location of the drone in the image. Based on the pre-set boundary information of the protected area along the line (which may come from map data, GIS system, or positioning based on tracks and preset boundary feature points of the protected area along the line), the outer contour of the protected area along the line is drawn in the target feature image. This step is the basis for determining the relative position of the drone and the protected area along the line.

[0130] Then, based on the outer contour of the drone, the contour feature rectangle is obtained by calculating its minimum circumscribed rectangle (that is, the minimum rectangle that can completely accommodate the outer contour of the drone). This step simplifies the subsequent calculation process while retaining the key position information of the drone in the image. The center point of the contour feature rectangle is calculated using the contour moment method. This step provides a key reference point for the subsequent calculation of the position feature value. Then, according to Formula 1, the monitoring platform calculates the position feature value of the drone in the image. This formula comprehensively considers factors such as the area of ​​the intersection between the drone and the protected area along the line, the area of ​​the contour feature rectangle, and the minimum distance between the center point of the contour feature and the outer contour of the protected area along the line.

[0131] Among them, for The part represents the ratio of the intersection area of ​​the contour feature rectangle and the protection area along the line in the image to the area of ​​the contour feature rectangle. It reflects the degree of overlap between the drone and the protection area along the line. The higher the ratio, the closer the drone is to or has entered the protection area along the line. The part represents the reciprocal of the minimum distance from the center point of the contour feature to the outer contour of the protection zone along the line. It reflects the relative distance between the drone and the protection zone along the line. The closer the distance, the larger the reciprocal, indicating that the drone is closer to the protection zone along the line.

[0132] Then, two weight values ​​are set to adjust the contribution of the area ratio of the intersection area and the inverse of the distance in the calculation of the position characteristic value. By adjusting these two parameters, flexible adjustment of the drone position judgment in different application scenarios can be achieved. The monitoring platform compares the calculated position characteristic value with the preset position characteristic threshold. If the position characteristic value is greater than the preset threshold, it is determined that the drone is located in the protection area along the line; otherwise, it is considered that the drone has not entered the protection area along the line.

[0133] It is worth noting that in this solution, since the images taken by the subway vehicle are dynamic images, that is, the images taken will change as the subway vehicle moves, and the images taken are two-dimensional images, it is impossible to directly determine the relative position relationship between the drone and the protected area along the line in the image, and it is also impossible to use Figure 1 In the illustrated embodiment, the relative position of the unmanned area and the protection area along the line is determined by marking the protection area along the line in the image.

[0134] Therefore, through the above scheme, the video frame image containing the drone is extracted from the monitoring video as the target feature image, and the outer contour of the drone is accurately extracted by using the image segmentation or edge detection algorithm, and the outer contour of the protected area is drawn according to the pre-set boundary information of the protected area along the line, so as to realize the accurate identification of the drone and the boundary of the protected area. This step provides high-precision basic data for subsequent position judgment. Then, by calculating the minimum circumscribed rectangle (contour feature rectangle) of the outer contour of the drone, the subsequent position calculation process is simplified. This step not only retains the key position information of the drone in the image, but also significantly reduces the calculation complexity and improves the processing efficiency. The center point of the contour feature rectangle is calculated by the contour moment method, which provides an accurate reference point for the subsequent calculation of the position feature value. This step is crucial for accurately judging the relative position of the drone and the protected area along the line. Then, the position feature value is calculated by formula 1, which comprehensively considers multiple factors such as the area of ​​the intersection area between the drone and the protected area along the line, the area of ​​the contour feature rectangle, and the minimum distance between the center point of the contour feature and the outer contour of the protected area along the line, so as to ensure the accuracy and comprehensiveness of the position judgment. In addition, the weight value in formula 1 can be flexibly adjusted according to the actual application scenario. By adjusting these two parameters, precise control of the drone's position judgment in different situations can be achieved, improving the adaptability and flexibility of the system. High-precision position judgment: By comparing the calculated position feature value with the preset position feature threshold, it can accurately determine whether the drone has entered the protection zone along the subway. This step provides an important basis for subsequent safety level determination and emergency response, and effectively improves the safety monitoring level of the protection zone along the subway.

[0135] On the basis of the above embodiments, after the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, it also includes:

[0136] If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the UAV, wherein the safety adjustment instruction is used to instruct the UAV to fly in a direction away from the track driving area.

[0137] In the above scheme, after determining the safety level of the protection zone along the line through the monitoring platform's detection status of subway vehicles and drones, a safety adjustment instruction mechanism was further introduced. This mechanism is designed to quickly guide drones to take corresponding measures when safety risks are detected, thereby effectively avoiding the occurrence of safety accidents.

[0138] Figure 4 FIG. 1 is a schematic diagram of a subway line monitoring system according to an exemplary embodiment of the present application. Figure 4As shown, the subway line monitoring system 400 provided in this embodiment includes: a camera cluster 410 and a monitoring platform 420, and each camera in the camera cluster 410 is respectively connected to the monitoring platform 420 for communication;

[0139] Acquire surveillance video along the subway line through the cameras in the camera cluster 410, and send the surveillance video to the monitoring platform 420, wherein the subway line includes the track running area and the protection area along the line;

[0140] The monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

[0141] Optionally, the camera is a first camera, the surveillance video is a first surveillance video, and the first camera is set on a fixed object outside the range of the subway line;

[0142] The monitoring platform 420 extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video;

[0143] The monitoring platform 420 determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence.

[0144] Optionally, the monitoring platform 420 determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence, including:

[0145] If the monitoring platform 420 detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0146] If the monitoring platform 420 does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level.

[0147] Optionally, after the monitoring platform 420 extracts the first feature image sequence from the first monitoring video, the process further includes:

[0148] The monitoring platform 420 determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one time direction of time advance and time delay based on the first feature image sequence;

[0149] Correspondingly, the monitoring platform 420 determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including:

[0150] If the monitoring platform 420 detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level;

[0151] If the monitoring platform 420 does not detect the drone in any feature image in the first feature image sequence using the preset target detection model, it is determined that the security level of the protection area along the line is the second security level, and the first security level is lower than the second security level;

[0152] If the monitoring platform 420 detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is a third safety level, which is lower than the second safety level and higher than the first safety level;

[0153] If the monitoring platform 420 does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the fourth safety level, which is higher than the second safety level.

[0154] Optionally, the camera is a second camera, which is arranged on the drone, and the monitoring video is a second monitoring video taken when the drone is located in the protection zone along the line; correspondingly, the monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0155] If the monitoring platform 420 detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, it is determined that the safety level of the protection area along the line is the first safety level;

[0156] If the monitoring platform 420 does not detect the presence of a subway vehicle in the track travel area in any of the feature images in the second monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

[0157] Optionally, the camera is a third camera, and the third camera is arranged on the subway vehicle. The monitoring video is a third monitoring video taken when the subway vehicle is located in the track driving area. Correspondingly, the monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track driving area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including:

[0158] If the monitoring platform 420 detects the presence of a drone in the protection zone along the line in at least one characteristic image in the third monitoring video, it determines that the security level of the protection zone along the line is the first security level;

[0159] If the monitoring platform 420 does not detect the presence of a drone in the protection zone along the line in any of the feature images in the third monitoring video, it is determined that the security level of the protection zone along the line is the second security level.

[0160] Optionally, after the monitoring platform 420 determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, it also includes:

[0161] If the monitoring platform 420 determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform 420 sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

[0162] Figure 5 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 5 As shown, an electronic device 500 provided in this embodiment includes: a processor 501 and a memory 502; wherein:

[0163] The memory 502 is used to store computer programs, and the memory may also be a flash memory.

[0164] The processor 501 is used to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0165] Optionally, the memory 502 may be independent or integrated with the processor 501 .

[0166] When the memory 502 is a device independent of the processor 501, the electronic device 500 may further include:

[0167] The bus 503 is used to connect the memory 502 and the processor 501 .

[0168] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the above-mentioned various implementation modes.

[0169] This embodiment also provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device implements the methods provided in the above various embodiments.

[0170] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0171] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method for a protected area along a subway line, characterized in that: Applied to a subway line monitoring system, the subway line monitoring system comprises: a camera cluster and a monitoring platform, each camera in the camera cluster is respectively connected to the monitoring platform for communication; the method comprises: Acquire surveillance videos along the subway line through the cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line includes the track running area and the protection area along the line; The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

2. The image processing method for the protection area along the subway according to claim 1 is characterized in that: The camera is a first camera, the monitoring video is a first monitoring video, and the first camera is set on a fixed object outside the range of the subway line; The monitoring platform extracts a first feature image sequence from the first monitoring video, where the first feature image sequence is a video frame image of a subway vehicle in the track travel area in the first monitoring video; The monitoring platform determines the safety level of the protection zone along the line according to a preset target detection model and the first feature image sequence.

3. The image processing method for the protection area along the subway according to claim 2 is characterized in that: The monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including: If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level; If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level.

4. The image processing method for the protection area along the subway according to claim 2 is characterized in that: After the monitoring platform extracts the first feature image sequence from the first monitoring video, the method further includes: The monitoring platform determines a second feature image sequence from the first monitoring video according to the first feature image sequence, wherein the second feature image sequence is a feature image sequence that is extended in at least one time direction of time advance and time delay based on the first feature image sequence; Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the preset target detection model and the first feature image sequence, including: If the monitoring platform detects a drone in at least one feature image in the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is the first safety level; If the monitoring platform does not detect the drone in any feature image in the first feature image sequence using the preset target detection model, it is determined that the safety level of the protection area along the line is the second safety level, and the first safety level is lower than the second safety level; If the monitoring platform detects the drone in at least one feature image in the difference set between the second feature image sequence and the first feature image sequence using a preset target detection model, it is determined that the safety level of the protection zone along the line is a third safety level, which is lower than the second safety level and higher than the first safety level; If the monitoring platform does not detect the drone in any feature image in the second feature image sequence using a preset target detection model, it is determined that the safety level of the protection area along the line is the fourth safety level, which is higher than the second safety level.

5. The image processing method for the protection area along the subway according to claim 1, characterized in that: The camera is a second camera, which is arranged on the drone, and the monitoring video is a second monitoring video taken when the drone is located in the protection zone along the line; correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including: If the monitoring platform detects the presence of a subway vehicle in the track travel area in at least one characteristic image in the second monitoring video, then determining that the safety level of the protection area along the line is the first safety level; If the monitoring platform does not detect the presence of a subway vehicle in the track travel area in any characteristic image in the second monitoring video, it is determined that the safety level of the protection area along the line is the second safety level.

6. The image processing method for the protection area along the subway according to claim 1, characterized in that: The camera is a third camera, and the third camera is arranged on the subway vehicle. The monitoring video is a third monitoring video taken when the subway vehicle is located in the track driving area. Correspondingly, the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track driving area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, including: If the monitoring platform detects the presence of a drone in the protection zone along the line in at least one characteristic image in the third monitoring video, determining that the security level of the protection zone along the line is the first security level; If the monitoring platform does not detect the presence of a drone in the protection zone along the line in any characteristic image in the third monitoring video, it is determined that the safety level of the protection zone along the line is the second safety level.

7. The image processing method for a subway protection zone according to any one of claims 3 to 6, characterized in that: After the monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video, it also includes: If the monitoring platform determines that the safety level is the first safety level or lower than the first safety level, the monitoring platform sends a safety adjustment instruction to the drone, wherein the safety adjustment instruction is used to instruct the drone to fly in a direction away from the track driving area.

8. A subway line monitoring system, characterized in that: include: A camera cluster and a monitoring platform, wherein each camera in the camera cluster is respectively connected to the monitoring platform for communication; Acquire surveillance videos along the subway line through the cameras in the camera cluster, and send the surveillance videos to the monitoring platform, wherein the subway line includes the track running area and the protection area along the line; The monitoring platform determines the safety level of the protection zone along the line according to the subway vehicle detection status of the track running area in the monitoring video and / or the drone detection status of the protection zone along the line in the monitoring video.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images