Intelligent rail transit door monitoring and abnormality identification method and system

Abstract

The application provides a kind of intelligent rail transit door monitoring and abnormality identification method and system, the method includes: real-time acquisition door area image and pre-processing, input into shared feature extraction backbone network and double-branch parallel deep learning model, semantic segmentation branch outputs pixel-level mask to quantitatively judge that door is completely closed, there is gap or not closed state, target detection branch outputs the boundary box and class of foreign matter in gap area;When judging that there is gap or not closed in door, analyze whether there is clothes, backpack belt, limb part and other pre-defined safety risk target in gap;If there is risk target, trigger safety response strategy, including alarming to driver and linkage with door control system to execute emergency stop, anti-pinch release or keep opening action.The application realizes fine perception of door state and intelligent early warning and automatic disposal of safety risk, solves the technical problems that existing technology cannot accurately identify the closed state of door and gap foreign matter.

Description

Technical Field

[0001] This invention relates to the field of rail transit vehicle safety technology, specifically to a method and system for intelligent rail transit vehicle door monitoring and anomaly identification. Background Technology

[0002] Existing rail transit door monitoring systems mainly rely on mechanical or electronic sensors such as limit switches and travel switches in the door drive mechanism to obtain simple "open" or "closed" status signals. This method can only provide basic switch feedback and cannot accurately determine whether the door is closed tightly, nor can it identify whether there are flexible foreign objects such as clothing or backpack straps in the door gaps, or safety hazards such as passengers' limbs being trapped. Another common auxiliary method is to rely on the driver or platform staff to conduct visual inspections. However, this method has blind spots, is prone to response delays due to staff fatigue or negligence, and has low reliability in actual operation.

[0003] In recent years, although some technical solutions have attempted to introduce video surveillance—that is, installing cameras near the train doors and transmitting the images to the driver's cab or control center for personnel to view—this is still passive monitoring. It lacks intelligent analysis of the video content, and anomaly detection relies entirely on subjective judgment and sustained attention from personnel. In practical applications, this is inefficient and carries a high risk of missed detections. Furthermore, some existing detection systems based on simple image processing exhibit poor robustness in real-world operating environments with complex lighting changes, rapid passenger movement, and scene occlusion, resulting in frequent false alarms and missed detections. Therefore, existing technologies are insufficient to meet the demands for highly reliable and real-time safety monitoring. There is an urgent need for a method that can accurately identify train door status and provide intelligent early warnings of safety hazards to improve the safety and efficiency of rail transit operations. Summary of the Invention

[0004] To address the problems mentioned in the background section regarding existing rail transit door monitoring systems, this invention provides an intelligent rail transit door monitoring and anomaly identification method and system.

[0005] The above-mentioned objective of this application is achieved through the following technical solution:

[0006] A method for monitoring and identifying anomalies in intelligent rail transit vehicle doors includes the following steps:

[0007] Real-time acquisition of image data for each door area;

[0008] The image data is preprocessed and input into the built-in deep learning object detection and semantic segmentation model to perform door closure status recognition and foreign object detection in the gap area in parallel.

[0009] Based on the door closure status recognition result, it is determined whether the door is currently fully closed, has a gap, or is not closed.

[0010] If it is determined that there is a gap or that the gap is not closed, then based on the foreign object detection results in the gap area, it is analyzed whether there is a predefined safety risk target in the gap;

[0011] When the analysis results indicate the existence of a safety risk target, a preset safety response strategy is triggered. The safety response strategy includes sending a real-time alarm command to the driver's console and coordinating with the door control system.

[0012] By adopting the above technical solution, accurate identification of vehicle door status and intelligent early warning of safety hazards are achieved. This method acquires image data of the vehicle door area in real time and uses deep learning object detection and semantic segmentation models for efficient processing. It can perform vehicle door closure status identification and foreign object detection in gap areas in parallel, thereby accurately determining the current status of the vehicle door. When it is determined that there is a gap or the vehicle door is not closed, it further analyzes whether there are predefined safety risk targets in the gap, such as clothing, backpack straps, or passenger limbs, effectively avoiding potential safety hazards. When a safety risk target is detected, the system immediately triggers the preset safety response strategy, including sending real-time alarm commands to the driver's console and linkage control with the vehicle door control system, ensuring timely handling of abnormal situations and passenger safety.

[0013] In a preferred embodiment, this application can be further configured such that the preprocessing of the image data includes:

[0014] Illumination effects are eliminated from the acquired continuous images to reduce the impact of changes in ambient lighting on image analysis.

[0015] Based on the processed image, key regions related to door movement and security monitoring are extracted to obtain target images for model analysis.

[0016] By employing the above technical solutions to preprocess image data, the stability and accuracy of subsequent deep learning model analysis can be improved. Eliminating illumination effects on continuous images effectively overcomes the problem of drastic illumination fluctuations caused by train operation, such as entering and exiting tunnels, day-night cycles, or weather changes, ensuring that the image quality input to the model is not affected by the environment. Extracting key areas related to door movement and safety monitoring removes redundant background information from the image, focusing on core detection areas such as door gaps and door edges, reducing the computational burden on the model, improving processing speed and recognition accuracy, and providing a reliable and high-quality image input foundation for subsequent door status judgment and foreign object detection.

[0017] In a preferred embodiment, this application may be further configured such that the deep learning object detection and semantic segmentation model includes a shared feature extraction backbone network, a semantic segmentation branch network for state recognition, and an object detection branch network for foreign object detection.

[0018] The feature extraction backbone network is used to extract multi-scale feature maps from the input image;

[0019] The semantic segmentation branch network processes the multi-scale feature map and outputs pixel-level segmentation masks for the door leaf edge and the sealing strip outline;

[0020] The target detection branch network processes the multi-scale feature map and outputs the bounding box, category, and confidence score of each detected target within the door gap area.

[0021] By adopting the above technical solution, a deep learning model architecture based on a shared feature extraction backbone network combined with semantic segmentation and object detection branches is used to achieve efficient parallel and collaborative processing of door status recognition and gap foreign object detection. The shared backbone network extracts multi-scale features of the image in one go, avoiding redundant calculations and improving the system's real-time processing efficiency. The semantic segmentation branch network can accurately characterize the pixel-level positions of the door leaf edges and sealing strip contours, providing direct evidence for refined judgment of the door's closure status. Simultaneously, the object detection branch network focuses on identifying and locating various foreign objects and their categories within the gap area. This dual-task parallel execution enables the system to accurately determine whether the door is closed tightly and whether there is a gap, while simultaneously detecting whether any risky targets are trapped in the gap. While ensuring high detection accuracy, it meets stringent requirements for response speed and computational resources, thus providing a unified, efficient, and reliable visual analysis core for door safety monitoring.

[0022] In a preferred embodiment, this application can be further configured such that determining whether the car door is currently fully closed, has a gap, or is not closed specifically includes:

[0023] Based on the segmentation mask output by the semantic segmentation branch network, the horizontal pixel distance between the center lines of the left and right door edge masks is calculated as an estimated value of the gap width.

[0024] The estimated gap width is compared with a first preset threshold.

[0025] If the estimated gap width is less than or equal to the first preset threshold, it is determined to be in a completely closed state;

[0026] If the estimated gap width is greater than the first preset threshold but less than the second preset threshold, it is determined that a gap exists.

[0027] If the estimated gap width is greater than or equal to the second preset threshold, it is determined to be in an unclosed state.

[0028] By adopting the above technical solution, the quantitative and hierarchical judgment of the door closure status is realized, upgrading the traditional "open / closed" signal to a three-level state perception of "fully closed, gap present, and not closed" that better meets the needs of safety monitoring. Based on the pixel-level precise mask provided by the semantic segmentation network, the width of the actual physical gap can be objectively and continuously estimated by calculating the horizontal pixel distance between the center lines of the left and right door edges. By setting a first preset threshold and a second preset threshold, the system can intelligently distinguish between different situations such as the door being tightly closed, having a small gap, and being obviously not closed, thus providing a refined state input for subsequent anomaly analysis and risk response. This not only improves the detection accuracy of the door being fully closed, but also provides a reliable basis for timely detection of situations where the door is not fully closed or has an abnormal gap, effectively avoiding safety hazards caused by misjudgment of the door status.

[0029] In a preferred embodiment, this application can be further configured such that: the analysis of whether a predefined security risk target exists within the gap includes:

[0030] Receive the detection results output by the target detection branch network;

[0031] Targets whose bounding box center point falls within the preset gap danger zone geometric model are selected;

[0032] The categories of the selected detection targets are matched with a predefined risk category library, which includes clothing, backpack straps, and body parts;

[0033] Based on a pre-set confidence threshold, if there is a detection target that matches successfully and has a confidence level higher than the threshold, it is determined to be a target with a security risk.

[0034] By adopting the above technical solution, the system achieves accurate identification and efficient early warning of potential safety risks within vehicle door gaps. By receiving the detection results output by the target detection branch network, the system can obtain the bounding boxes, categories, and confidence information of all detected targets within the gap area. Further filtering identifies targets whose bounding box center points fall within a preset gap danger zone geometric model, effectively focusing on the core risk area of ​​the vehicle door gap and avoiding interference from irrelevant areas. Matching the filtered target categories with a predefined risk category library accurately identifies flexible foreign objects or passenger limbs that may pose safety hazards, such as clothing, backpack straps, and body parts. Combined with a pre-set confidence threshold, false detections with low confidence are eliminated; only when a matching target with a confidence level higher than the threshold is it determined to be a safety risk target, thus ensuring the accuracy and reliability of anomaly identification. This improves the intelligence level of vehicle door safety monitoring and provides greater protection for passenger safety.

[0035] In a preferred embodiment, this application can be further configured such that, after determining that a target poses a security risk, a target risk confirmation step is also included:

[0036] In a continuously acquired sequence of image data, multi-target tracking is performed on targets identified as safety risks to obtain their motion trajectories;

[0037] The number of frames in which the target remains within the preset gap danger zone geometric model within a preset time window is counted.

[0038] If the number of consecutive frames exceeds a preset threshold, the security risk target is confirmed, and the step of triggering the security response strategy is executed.

[0039] By adopting the above technical solution, after determining that there is a safety risk target, the introduction of multi-target tracking technology to continuously track the target can effectively avoid false alarms caused by misdetection of a single frame image or brief interference. The target position is located and a motion trajectory is generated in the continuously acquired image sequence, and the number of frames in which it stays in the preset gap danger zone is counted. When the number of frames exceeds a preset threshold, it indicates that the target is not passing by accidentally or staying briefly, but has an actual risk of being clamped. At this time, the safety risk target is confirmed and the response strategy is triggered. This secondary confirmation mechanism improves the reliability of anomaly identification and reduces misoperation caused by non-dangerous situations such as environmental vibration or brief obstruction by passengers. While ensuring the sensitivity of safety monitoring, it optimizes the actual application efficiency of the system.

[0040] In a preferred embodiment, this application can be further configured such that the linkage control with the door control system includes:

[0041] Generate standard control instructions corresponding to anomaly types and risk levels;

[0042] The standard control commands are sent to the corresponding door electronic control unit via the vehicle bus network;

[0043] The door electronic control unit executes at least one preset action based on the received command: emergency stop closing, anti-pinch release and retraction, or keeping the door open.

[0044] By adopting the above technical solution, after detecting and confirming a safety risk target, standard control commands matching the anomaly type and risk level are quickly generated. These commands can accurately and effectively respond to various door safety anomalies. The system quickly sends the standard control commands to the corresponding door electronic control unit via the vehicle bus network. As the "intelligent brain" of the door, the door electronic control unit can immediately parse and execute the corresponding preset actions after receiving the commands, such as emergency stop closing to prevent injury to passengers, anti-pinch release retraction to release trapped objects, or keeping the door open for further inspection and processing. This series of automated and intelligent linkage control measures improves the response speed and processing efficiency of door safety monitoring, providing a solid guarantee for passenger travel safety.

[0045] In a preferred embodiment, this application can be further configured such that the method also includes a model optimization step, the model optimization step comprising:

[0046] After each security response strategy is triggered, image sequences and corresponding intermediate model results and final judgment labels within a set time period before and after the event are automatically captured to form incremental training samples.

[0047] The parameters of the deep learning object detection and semantic segmentation model are periodically fine-tuned using the incremental training samples.

[0048] By adopting the above technical solution and introducing model optimization steps, the intelligent monitoring system achieves continuous self-improvement and performance enhancement. After each safety response strategy is triggered, the system automatically captures image sequences within a set time period before and after the event. These image sequences contain typical safety event scenarios occurring in actual operation, and simultaneously record the corresponding intermediate model processing results and final judgment labels, collectively forming an incremental training sample library. This sample library not only includes images of train doors under normal conditions but also focuses on keyframes in abnormal events, providing the model with rich and targeted learning materials. These incremental samples are used to periodically fine-tune the parameters of the deep learning object detection and semantic segmentation models, enabling the models to continuously adapt to complex changes in the actual operating environment, such as seasonal changes in lighting conditions, evolution of passenger clothing styles, and the emergence of new backpacks or clothing. Through continuous learning of new features and patterns in real-world scenarios, the model improves in terms of accuracy in train door status recognition, robustness in foreign object detection, and accuracy in judging abnormal situations. This ensures the system maintains a high level of performance over the long term, effectively reducing false alarm and false negative rates, and providing more reliable technical support for the safe operation of rail transit.

[0049] In a preferred embodiment, this application can be further configured such that the method also includes a data management and traceability step, the data management and traceability step comprising:

[0050] All alarm events, corresponding image evidence, model analysis logs, and linkage control records are stored in a structured format on the vehicle's storage device.

[0051] During off-peak hours, stored data is uploaded in batches to the ground data center via vehicle-to-ground wireless network to generate safety reports and operational analysis.

[0052] By adopting the above technical solutions, a comprehensive data management and traceability system is constructed, realizing digital closed-loop management of the entire process of door safety monitoring. When any alarm event occurs, the system automatically associates key information such as event timestamps, trigger locations, image evidence, model analysis process logs, and subsequent linkage control records, storing them in storage devices to ensure data integrity and traceability. During idle periods of train operation, onboard data is uploaded in batches to the ground data center via the train-to-ground wireless communication network, avoiding the occupation of operational bandwidth by real-time transmission. The ground data center generates multi-dimensional safety reports based on the uploaded data, including event type distribution, time pattern analysis, and high-risk area identification, providing decision support for operators. It also supports historical data backtracking and querying, allowing for the retrieval of complete evidence chains for specific events for review and analysis, assisting in optimizing monitoring strategies and model parameters. This provides a data foundation for continuous optimization of system performance, and by mining potential patterns in historical events, it drives the evolution of door safety monitoring towards a more intelligent and precise direction.

[0053] The above-mentioned objective 2 of this application is achieved through the following technical solution:

[0054] An intelligent rail transit vehicle door monitoring and anomaly recognition system includes:

[0055] The image acquisition module is deployed at each door to collect image data of the area around the door in real time.

[0056] The intelligent analysis module is communicatively connected to the image acquisition module and is used to process the image data. The intelligent analysis module includes a target detection and processing unit and a door status monitoring unit. The target detection and processing unit is used to run the deep learning target detection and semantic segmentation model to perform door closure status recognition and foreign object detection in the gap area. The door status monitoring unit is connected to the target detection and processing unit and is used to determine the door status and analyze whether the safety risk target exists based on the recognition and detection results.

[0057] The alarm and notification module is connected to the door status monitoring unit and is used to trigger the safety response strategy when the analysis result indicates that there is a safety risk target, send an alarm to the driver's console and notify the background system.

[0058] The data storage and analysis module is connected to the intelligent analysis module and the alarm and notification module, and is used to store monitoring data and abnormal records.

[0059] The background management and monitoring module is connected to the data storage and analysis module to remotely monitor the system status and perform data backtracking and system optimization.

[0060] By adopting the above technical solution, a complete intelligent rail transit door monitoring and anomaly recognition system was constructed. Each module works collaboratively to achieve efficient and accurate door safety monitoring. The image acquisition module is located at key positions on each door, responsible for capturing image data of the area surrounding the door in real time, providing raw material for subsequent analysis. The intelligent analysis module, as the core processing unit, communicates with the image acquisition module. Its internal target detection and processing unit runs a deep learning target detection and semantic segmentation model, performing door closure status recognition and foreign object detection tasks in gap areas. Through multi-scale feature extraction and a dual-branch network architecture, efficient parallel processing of status recognition and foreign object detection is achieved. Based on the output of the target detection and processing unit, the door status monitoring unit further determines the current door status (fully closed, with gaps, or not closed) and analyzes it. The presence or absence of safety risks provides crucial information for subsequent decision-making. The alarm and notification module immediately triggers a safety response strategy upon detecting a safety risk, sending alarm signals to the driver's console and notifying the backend system to ensure relevant personnel are promptly informed and can take appropriate measures, effectively preventing accidents. The data storage and analysis module stores all data and anomaly records during monitoring, including image evidence, model analysis logs, and linkage control records, providing data support for system optimization and accident tracing. The backend management and monitoring module enables remote monitoring of system status, data backtracking, and system optimization. By mining potential patterns in historical data, it continuously improves system performance, ensuring intelligent and precise door safety monitoring. This enhances the automation level of door safety monitoring and provides a guarantee for the safe operation of rail transit.

[0061] In summary, this application includes at least one of the following beneficial technical effects:

[0062] 1. Through deep learning-based target detection and semantic segmentation models, quantitative and hierarchical judgment of the door closure status and accurate identification of foreign objects in the gap area are achieved, upgrading traditional binary monitoring to refined three-level state perception, effectively improving the detection accuracy and reliability of door safety monitoring.

[0063] 2. The system adopts a multi-target tracking and secondary confirmation mechanism. After identifying a safety risk target, it avoids false alarms by continuously tracking and counting frames. Combined with the model optimization steps, it uses incremental samples to fine-tune the parameters periodically, enabling the system to adapt to changes in the operating environment and maintain a high level of performance.

[0064] 3. By building a complete data management and traceability system, the system enables the complete recording and structured storage of alarm events, image evidence, and analysis logs, supporting the generation of safety reports and operational analysis by the ground data center, and providing data support for system optimization and accident review;

[0065] 4. The collaborative work of each module forms a digital closed-loop management system, which not only improves the automation and intelligence level of door safety monitoring, but also provides comprehensive and traceable technical support for the safe operation of rail transit through real-time linkage control and remote management functions. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating an embodiment of the intelligent rail transit door monitoring and anomaly identification method of this application;

[0067] Figure 2 This is a flowchart illustrating the implementation of the deep learning target detection and semantic segmentation model in the embodiment of the intelligent rail transit door monitoring and anomaly recognition method of this application;

[0068] Figure 3 This is a flowchart illustrating the implementation of step S30 in the embodiment of the intelligent rail transit door monitoring and anomaly identification method of this application;

[0069] Figure 4 This is a flowchart illustrating the implementation of step S40 in the embodiment of the intelligent rail transit door monitoring and anomaly identification method of this application;

[0070] Figure 5 This is another implementation flowchart of step S40 in the embodiment of the intelligent rail transit door monitoring and anomaly identification method of this application;

[0071] Figure 6 This is a flowchart illustrating the implementation of step S50 in the embodiment of the intelligent rail transit door monitoring and anomaly identification method of this application.

[0072] Figure 7 This is a schematic diagram of an embodiment of the intelligent rail transit door monitoring and anomaly recognition system of this application. Detailed Implementation

[0073] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] In one embodiment, such as Figure 1 As shown, this application discloses a method for monitoring and identifying anomalies in intelligent rail transit doors, which specifically includes the following steps:

[0075] S10: Real-time acquisition of image data for each door area.

[0076] In this embodiment, real-time acquisition refers to acquiring image data of the area around the vehicle door using image acquisition devices deployed at each vehicle door at specific time intervals or in a continuous manner; each vehicle door area specifically refers to the key safety monitoring area around each vehicle door, especially the door gaps, door leaf edges, and door sills.

[0077] Specifically, the image acquisition device can be a high-definition camera, which has high resolution and good low-light performance, and can clearly capture detailed images of the car door area under different lighting conditions; the acquisition time interval is set according to actual needs, and a shorter time interval is used for scenarios with high security requirements to ensure that no key information is missed; by acquiring image data in real time, rich raw materials are provided for subsequent anomaly identification and analysis, enabling the system to perceive the real-time status of the car door in a timely manner.

[0078] S20: The image data is preprocessed and input into the built-in deep learning target detection and semantic segmentation model to perform door closure state recognition and foreign object detection in the gap area in parallel.

[0079] In this embodiment, image preprocessing includes operations such as illumination normalization, image enhancement, and key region cropping; the deep learning object detection and semantic segmentation model refers to a computer vision model based on neural networks, which has the ability to learn features autonomously; object detection is used to identify and locate specific objects in the image, such as foreign objects or limbs; semantic segmentation is used to classify the image at the pixel level and identify the contour regions of different components (such as door panels and sealing strips); parallel execution means that the model performs two tasks simultaneously while processing image data: door closure status recognition and foreign object detection in the gap area.

[0080] Specifically, the acquired raw images undergo illumination normalization processing. By adjusting the brightness distribution of the image, the differences in image quality caused by uneven illumination are eliminated, ensuring clear image effects under different lighting conditions. Next, image enhancement is performed to improve image contrast and make details in the image more prominent, facilitating subsequent target detection and semantic segmentation. Then, key regions are cropped from the image. Based on the actual needs of vehicle door monitoring, key areas such as door gaps, door leaf edges, and thresholds are accurately located to reduce interference from irrelevant information and improve the model's processing efficiency. The preprocessed image data is input into the built-in deep learning target detection and semantic segmentation model. This model is based on an advanced neural network architecture and has powerful feature extraction and classification capabilities through training on a large amount of labeled data. When processing image data, the model can simultaneously perform two tasks: vehicle door closure status recognition and foreign object detection in gap areas. Through multi-scale feature extraction and a dual-branch network architecture, it efficiently and in parallel completes the quantitative and graded judgment of vehicle door status and the accurate identification of foreign objects in gap areas.

[0081] S30: Based on the door closure status recognition result, determine whether the door is currently fully closed, has a gap, or is not closed.

[0082] In this embodiment, the door closure status recognition result refers to the pixel-level mask of the door component output by the semantic segmentation model; the fully closed state means that the door leaf and the sealing strip are tightly fitted with no obvious gaps; the gap state means that there are still small gaps after the door is closed, which may trap foreign objects; the unclosed state means that the door is obviously not closed properly, the gap width is large, and it has become a safety hazard.

[0083] Specifically, the semantic segmentation model can accurately identify the current state of the car door by analyzing the pixel-level mask of the door components. When the door leaf and the sealing strip are tightly fitted with no obvious gaps, it is determined to be in a fully closed state. When there are still small gaps after the door is closed, which may trap foreign objects, it is determined to be in a gap state. When the door is obviously not closed properly, and the gap is wide enough to pose a safety hazard, it is determined to be in an unclosed state. This quantitative and graded judgment method can accurately reflect the actual state of the car door and provide a basis for subsequent safety risk assessment.

[0084] S40: If it is determined that there is a gap or an unclosed state, then based on the foreign object detection results in the gap area, analyze whether there is a predefined safety risk target in the gap.

[0085] In this embodiment, the foreign object detection result in the gap area refers to the detection box and category information output by the target detection model; the predefined safety risk targets include, but are not limited to, objects that may be caught in the car door, such as clothing, backpack straps, and limb parts.

[0086] Specifically, when the target detection model detects a foreign object in the gap area, it outputs the corresponding detection box and category information, which describe in detail the location and type of the foreign object. The system compares this information with a predefined list of safety risk targets to determine whether there are any objects in the gap that may be caught in the car door, such as clothing, backpack straps, or limbs. If such a safety risk target is detected, the system will immediately proceed to the next safety response process to ensure that timely measures are taken to avoid potential safety accidents.

[0087] S50: When the analysis result indicates the existence of a safety risk target, a preset safety response strategy is triggered. The safety response strategy includes sending a real-time alarm command to the driver's console and coordinating with the door control system.

[0088] In this embodiment, the safety response strategy refers to the automated response measures pre-defined by the system for different risk levels. Real-time alarm commands refer to visual, auditory, or tactile warning information pushed to the driver's console. Linkage control refers to the system sending standard control commands to the door electronic control unit via the vehicle bus to trigger corresponding safety protection actions.

[0089] Specifically, when the system analysis confirms the presence of a safety risk target, it will immediately trigger a preset multi-level safety response mechanism. First, a real-time alarm command containing the risk type and location coordinates is sent to the driver's console via the vehicle communication bus. Simultaneously, the system automatically generates a standardized control command containing a timestamp, door number, and risk level, and sends an emergency linkage signal to the corresponding door's electronic control unit (ECU) via the vehicle's CAN bus, triggering protective actions such as door anti-pinch, automatic rebound, or maintaining the current opening degree. For persistent risk targets, the system will initiate a secondary confirmation mechanism, eliminating false alarms by tracking and verifying three consecutive frames of images. If a risk is confirmed, the alarm level will be automatically escalated and simultaneously pushed to the ground control center, ensuring that ground dispatchers can monitor the situation on-site in real time. At the same time, the system will store all relevant data of this alarm event, including original images, analysis logs, and linkage control records, and mark the anomaly level, providing complete data chain support for subsequent accident tracing and system optimization. This multi-level, closed-loop safety response mechanism can not only effectively avoid response failures caused by single-point failures, but also provide comprehensive and traceable technical protection for the safety monitoring of rail transit doors through the synergy of real-time linkage control and remote management functions.

[0090] In one embodiment, step S20, i.e., the preprocessing of the image data, includes:

[0091] S21: Perform illumination effect elimination processing on the acquired continuous images to reduce the impact of ambient lighting changes on image analysis.

[0092] In this embodiment, illumination effect elimination processing refers to processing the image through algorithms to reduce or eliminate the negative impact on image quality caused by uneven ambient light and changes in intensity (such as entering and exiting tunnels, day and night alternation, and direct light), thereby obtaining an image with stable visual performance.

[0093] Specifically, the system receives continuous images from high-definition cameras in the door area in real time. For each frame, it processes the images using algorithms based on histogram equalization or adaptive gamma correction to improve image visibility in low-light or high-contrast scenes. For rapid changes in lighting caused by train operation (such as entering a tunnel), it combines multiple historical images and uses dynamic range adjustment technology to quickly stabilize global brightness and prevent overexposure or underexposure. The processed images have a more balanced brightness distribution and clearer details, ensuring that the image quality input to the deep learning model remains consistent under different lighting conditions, thereby improving the stability and accuracy of the model in identifying door status and foreign objects in gaps.

[0094] S22: Based on the processed image, extract key regions related to door movement and security monitoring to obtain target images for model analysis.

[0095] In this embodiment, the key area refers to the local image area directly related to the safety monitoring of the vehicle door, mainly including the edge of the door leaf, the outline of the sealing strip, the gap between the door seams, and the area near the threshold where passengers are easily trapped; extraction refers to automatically locating and cropping these specific areas of interest from the complete original image through image processing technology; the target image refers to the image that, after extraction processing, contains only the key area, has a uniform size, and has little background interference.

[0096] Specifically, during the system initialization phase, based on the camera installation location and the physical structure of the car door, the approximate range of the key area in the image coordinate system is predefined. For each processed image frame, fine-tuning is performed according to this predefined range, combined with edge detection and other algorithms, to accurately locate the edge line of the car door leaf and the position of the sealing strip. The system extends a preset safety buffer to both sides of the door gap as the center, and crops out a rectangular image block containing the entire gap and the adjacent area. The cropped key area image is uniformly scaled to a fixed size required by the model (such as 640x640 pixels) to form a target image that can be directly input into the model for analysis. This process effectively removes irrelevant information such as the background inside the car and billboards, reduces the computational burden on the model, and improves the accuracy and speed of subsequent state recognition and foreign object detection.

[0097] In one embodiment, such as Figure 2 As shown, in step S20, the deep learning target detection and semantic segmentation model includes a shared feature extraction backbone network, a semantic segmentation branch network for state recognition, and a target detection branch network for foreign object detection.

[0098] The feature extraction backbone network is used to extract multi-scale feature maps from the input image;

[0099] The semantic segmentation branch network processes the multi-scale feature map and outputs pixel-level segmentation masks for the door leaf edge and the sealing strip outline;

[0100] The target detection branch network processes the multi-scale feature map and outputs the bounding box, category, and confidence score of each detected target within the door gap area.

[0101] In this embodiment, the shared feature extraction backbone network is a deep learning convolutional neural network responsible for automatically learning and extracting features at different levels of abstraction from the input image, forming a set of multi-scale feature maps. These feature maps simultaneously serve the subsequent two branch tasks. The semantic segmentation branch network is a network structure that performs pixel-level classification of images. Its output is a segmentation mask of the same size as the input image, where each pixel is labeled as belonging to a specific category (such as "door edge", "seal strip", or "background"). The object detection branch network is a network structure (such as YOLO) used to locate and identify specific objects in an image. Its output is a series of bounding boxes, each representing a detected object, along with the object category (such as "clothing", "backpack strap", "limb") and its confidence level.

[0102] Specifically, the preprocessed target image is fed into a shared feature extraction backbone network. Through multi-layer convolution and pooling operations, a set of multi-scale feature maps is generated, encompassing low-level details (such as edges and textures) to high-level semantics (such as object parts and overall structure). These feature maps are then fed in parallel into two branch networks: the semantic segmentation branch network classifies each pixel in the feature maps through upsampling and feature fusion, accurately outputting the pixel-level positions of the door panel edges and sealing strip contours to form a segmentation mask; simultaneously, the object detection branch network applies anchor box mechanisms and classification regression on the same feature maps to specifically identify and locate foreign objects within the door gap area, outputting the bounding box coordinates, category, and the model's confidence level for each detected object. This achieves simultaneous fine geometric analysis of the door state and efficient object detection of foreign objects in the gap, optimizing computational efficiency while maintaining high accuracy.

[0103] In one embodiment, such as Figure 3 As shown, step S30, which determines whether the car door is currently fully closed, has a gap, or is not closed, specifically includes the following steps:

[0104] S31: Based on the segmentation mask output by the semantic segmentation branch network, calculate the horizontal pixel distance between the center lines of the left and right door edge masks as an estimate of the gap width.

[0105] In this embodiment, the center lines of the left and right door leaf edge mask refer to two representative lines obtained by performing center line fitting or skeletonization on the pixel sets representing the left and right door leaf edges in the segmentation mask, respectively; the horizontal pixel distance refers to the average pixel spacing between corresponding points between these two center lines in the horizontal direction of the image; the gap width estimate refers to the actual width value of the door gap reflected by the calculated horizontal pixel distance, which is in pixels and is used to quantitatively evaluate the gap size between the left and right door leaves when the door is closed.

[0106] Specifically, pixel sets representing the left and right door edges are extracted from the segmentation mask output by the semantic segmentation branch network. These pixel sets are then subjected to centerline fitting, for example, using a skeletonization algorithm to transform the wide lines of the door edges into single-pixel-width centerlines, reducing the impact of edge thickness on distance measurement. In the horizontal direction of the image, the average pixel spacing between corresponding points on these two centerlines is calculated as an estimate of the door gap width. By traversing multiple sampling points along the centerline and taking the average distance, the stability and accuracy of the gap width measurement are ensured. This estimate directly reflects the door's closure state, providing a quantitative basis for subsequent classification judgments.

[0107] S32: Compare the estimated gap width with a first preset threshold;

[0108] S33: If the estimated gap width is less than or equal to the first preset threshold, it is determined to be in a completely closed state;

[0109] S34: If the estimated gap width is greater than the first preset threshold but less than the second preset threshold, it is determined that a gap exists;

[0110] S35: If the estimated gap width is greater than or equal to the second preset threshold, it is determined to be in an unclosed state.

[0111] In this embodiment, the first preset threshold is a small value (such as a pixel distance of 1-2 mm) used to define the boundary between "completely closed" and "with gaps"; the second preset threshold is a large value (such as 10 mm or a pixel distance where the door is obviously not in place) used to define the boundary between "with gaps" and "not closed"; these three states correspond to different safety conditions of the door: complete closure indicates safety; gaps indicate that small foreign objects may be trapped or there is a risk of not closing properly; not closed indicates that the door is obviously not closed properly.

[0112] Specifically, after obtaining the estimated gap width, it is compared with two pre-set thresholds through calibration and safety analysis. If the estimated value is less than or equal to the first threshold, the door is determined to be fully closed, with the door leaf and sealing strip tightly fitted. If the estimated value is greater than the first threshold but less than the second threshold, the door is determined to have a gap, and the risk will be further assessed in conjunction with subsequent foreign object detection results. If the estimated value is greater than or equal to the second threshold, the door is directly determined to be unclosed, requiring immediate attention to this safety anomaly. This three-level quantitative judgment logic elevates the description of the door's closure status from a simple "open" to a refined "safety level," providing a precise basis for subsequent differentiated management and response.

[0113] In one embodiment, such as Figure 4 As shown, in step S40, i.e., analyzing whether there is a predefined safety risk target within the gap, includes:

[0114] S41: Receive the detection results output by the target detection branch network.

[0115] In this embodiment, the detection result refers to the structured data set output by the target detection branch network after completing forward inference.

[0116] Specifically, after the object detection network processes the input image, it reads the raw detection results from the network's memory or output buffer. The system first performs preliminary filtering on the results, usually removing detection entries that the model considers highly unreliable based on a low confidence threshold (e.g., 0.3) to reduce noise in subsequent processing. After filtering, the remaining detection results are organized into a list or array for analysis in subsequent steps. These results represent all potential safety risks posed by the model in the door gap area and its surroundings in the current frame, and are the core data input for subsequent risk assessment.

[0117] S42: Filter out targets whose bounding box center point falls within the preset gap danger zone geometric model.

[0118] In this embodiment, the preset gap danger zone geometric model refers to one or more polygonal regions predefined in the image coordinate system based on the door structure (such as the door edge and the position of the sealing strip). This region accurately covers the actual physical space (i.e., the gap and its adjacent danger zone) where clamping or jamming may occur when the door is closed; the center point of the bounding box refers to the geometric center coordinates of each detection box in the target detection result.

[0119] Specifically, the system first loads a predefined geometric model of the gap hazard zone; this model is typically one or more polygonal contours whose coordinates are mapped to the image plane based on camera calibration and door dimensions; for each bounding box of a detected target received from step S41, the system calculates the coordinates of its center point; using a geometric point and polygon inclusion detection algorithm (such as ray casting), it determines whether the center point is located inside any polygon of the gap hazard zone; only those detected targets whose center points fall within the hazard zone are retained, effectively focusing the analysis from the entire image to the core area where the most likely dangerous accident is to occur, eliminating interference from similar objects in the distant background or other non-risk areas of the door.

[0120] S43: Match the selected target categories with a predefined risk category library, which includes clothing, backpack straps, and body parts.

[0121] In this embodiment, the predefined risk category library is a pre-specified list representing object categories that may pose a safety risk. It is typically determined based on historical accident data and safety regulations, and its categories correspond to some of the categories used when training the target detection model.

[0122] Specifically, for each target selected by S42, the system checks the category label output by the target detection network. The system then performs a precise or semantic match between the label and the entries in the risk category library (for example, the detection result "sleeve" can be matched with the "clothing" category in the risk library). Only those category labels that can be successfully matched with any category of the target in the risk library are considered as potential risk candidates, ensuring that the system only focuses on the types of objects that are likely to cause pinching or operational failures, ignoring other irrelevant objects that may appear in the door area (such as water bottles, newspapers, etc.).

[0123] S44: Based on a pre-set confidence threshold, if there is a detection target that matches successfully and has a confidence level higher than the threshold, it is determined that there is a target with a security risk.

[0124] In this embodiment, the pre-set confidence threshold is a relatively high value (e.g., 0.7 or 0.8) to further filter out highly reliable results determined by the model from the candidate targets that have passed the location and category screening, so as to minimize false alarms.

[0125] Specifically, for a detected target that successfully matches the category in S43, its associated confidence score is checked; this confidence score is compared with a preset high confidence threshold; only when a detected target simultaneously meets the following three conditions: 1. Its bounding box center point is located within the gap danger zone; 2. Its category belongs to the risk category library; 3. Its confidence score is higher than the preset threshold, will the system finally determine that there are (one or more) safety risk targets in the current image frame; this determination is the core decision basis for triggering subsequent alarms and linkage control. If no target meets all the conditions, it is determined that there is currently no safety risk.

[0126] In one embodiment, such as Figure 5 As shown, after step S40, that is, after determining that a target poses a security risk, a target risk confirmation step is also included:

[0127] S45: In a continuously acquired sequence of image data, perform multi-target tracking on targets identified as safety risks and obtain their motion trajectories.

[0128] In this embodiment, multi-target tracking refers to a computer vision technology that can associate the appearance of the same physical target at different times in continuous video frames and assign it a unique ID, forming the target's motion trajectory over time.

[0129] Specifically, after identifying a security risk target in a single frame image, the system immediately activates a multi-target tracker. This tracker uses the bounding boxes of high-risk targets selected in steps S42-S44 as initial observations. In subsequent continuously acquired image sequences, even if the target's appearance changes slightly or is briefly occluded, it can continuously predict its position and associate it with the correct ID. Through tracking, it can not only obtain the target's position (bounding box) in each subsequent frame, but also calculate its movement speed and direction, forming a motion trajectory starting from the moment of risk determination. This provides a data basis for distinguishing between risk targets that "pass by momentarily" and those that "linger continuously".

[0130] S46: Count the number of frames in which the target remains within the preset gap danger zone geometric model within a preset time window.

[0131] In this embodiment, the preset time window refers to a short period of time (e.g., 1-2 seconds, corresponding to 30-60 frames of images) starting from when the target is first determined to be a safety risk; "continuously in" means that the center point of the target's bounding box or most of its area is located inside the geometric model of the gap danger zone in multiple consecutive frames of images, without moving out of the area.

[0132] Specifically, based on the target motion trajectory provided by the multi-target tracker, the center point of the target's bounding box is checked frame by frame to see if it still falls within the geometric model of the gap danger zone. Starting from the frame in which the target is first identified, the number of image frames in which the target is continuously in the danger zone is accumulated within a sliding time window. The statistical process is carried out in real time until the target moves out of the danger zone or reaches the upper limit of the preset time window. By counting the number of consecutive frames, the dwell time of the target in the danger zone can be quantified, effectively filtering out objects that only quickly pass through the gap area (such as a passenger's rapidly waving arm).

[0133] S47: If the number of continuous frames exceeds the preset number of continuous frames threshold, the security risk target is confirmed to be established, and the step of triggering the security response strategy is executed.

[0134] In this embodiment, the preset continuous frame number threshold is a value set according to the security response timeliness requirements (such as 10-15 frames corresponding to 0.5 seconds), which is used to determine whether the target "continuously" poses a risk rather than "instantaneous" interference.

[0135] Specifically, the system compares the number of consecutive frames obtained in real time with a preset threshold. Once the cumulative number of frames in which the target remains in the danger zone exceeds the threshold within the preset time window, the system determines that the safety risk target is "confirmed." This means that the target is not a temporary false alarm or a momentary pass-by, but is highly likely to be actually trapped by the car door or pose a continuous threat. The system will immediately output a final confirmation signal and trigger the preset safety response strategy in the subsequent steps (S50), such as sending an alarm to the driver or linking with the door control system. If the target has moved out of the danger zone before the threshold is reached, the system determines it as a false alarm and resets the tracking and statistics status. This secondary confirmation mechanism greatly enhances the reliability of the system and effectively reduces malfunctions caused by single-frame false alarms or brief interference.

[0136] In one embodiment, such as Figure 6 As shown, in step S50, the linkage control with the door control system includes the following steps:

[0137] S51: Generate standard control instructions corresponding to the anomaly type and risk level.

[0138] In this embodiment, the anomaly type refers to the specific category of safety event identified by the system, such as "flexible foreign object entrapment", "limb being trapped" or "door not fully closed"; the risk level is the severity of danger assessed based on factors such as anomaly type, target duration and / or target size, and is usually divided into multiple levels (such as "warning", "serious" and "emergency"); the standard control command refers to a predefined digital command with a predefined format and content that can be recognized and executed by the door electronic control unit (ECU).

[0139] Specifically, the system first matches the corresponding standard control instruction template from a preset instruction library based on the anomaly type (such as clothing entrapment, limb clamping, or door not closing) determined in step S40 and subsequent steps, combined with the risk level (such as warning, serious, or emergency) confirmed in step S47. The instruction template typically includes key fields such as instruction type (such as "emergency braking," "delayed closing," or "alarm prompt"), target door number, and priority. The system fills the specific parameters of the anomaly type and risk level into the template to generate an executable digital instruction. For example, for the case of "limbs being clamped and the risk level being emergency," the system generates the instruction to "immediately stop the door movement and trigger a vehicle-wide alarm"; for the case of "clothing entrapment and the risk level being warning," the system generates the instruction to "delay closing the door for 3 seconds and send a prompt to the driver's cab."

[0140] S52: Send the standard control command to the corresponding door electronic control unit via the vehicle bus network.

[0141] In this embodiment, the vehicle bus network refers to the standardized network inside the train used to connect various subsystems (such as doors, traction, and braking) for data communication. Common types include CAN bus, MVB (Multi-function Vehicle Bus), or Ethernet. The door electronic control unit (ECU) is an embedded controller that specifically controls the functions of driving, locking, and anti-pinch of a single or group of doors.

[0142] Specifically, the system sends the standard control command generated in step S51 to the address of the electronic control unit (ECU) corresponding to the target door through a preset communication interface. During the transmission process, the system records the transmission time, target ECU number, and command content, and initiates a timeout retransmission mechanism (such as 3 retries) to ensure that the command arrives reliably. After receiving the command, the door ECU parses the command type, priority, and parameters, and immediately executes the corresponding control action (such as emergency braking, delayed closing, or alarm triggering). At the same time, it sends an execution confirmation or error feedback signal to the system through the vehicle bus network, forming a complete closed-loop control link.

[0143] S53: The door electronic control unit executes at least one preset action, such as emergency stop closing, anti-pinch release and retraction, or keeping the door open, according to the received instruction.

[0144] In this embodiment, the emergency stop closing command requires the door drive mechanism to immediately stop the current closing action and remain stationary; the anti-pinch release retraction command requires the door drive mechanism to move a preset safe distance (e.g., a few centimeters) in the opening direction after stopping, in order to release any objects that may be pinched; and the keep door open command requires the door to remain in the current open state and suspend the subsequent automatic closing process.

[0145] Specifically, after receiving the control command from the bus, the ECU of the target door first verifies and parses it to confirm the validity of the command and the target address. Based on the command code, the ECU calls the corresponding control logic stored internally. For example, if the command is "emergency stop and release", the ECU will immediately cut off the power to the door closing motor and control the motor to run in reverse for a short time, causing the door to move back slightly. The execution status and results of all actions can be fed back to the monitoring system through the bus to form a control closed loop. This linkage control mechanism enables intelligent recognition to be directly transformed into timely and effective physical protection actions, maximizing passenger safety and preventing equipment damage.

[0146] In one embodiment, the method further includes a model optimization step, the model optimization step comprising:

[0147] M1: After each triggering of the security response strategy, automatically extract the image sequence and corresponding intermediate model results and final judgment labels within a set time period before and after the event to form incremental training samples;

[0148] M2: Periodically fine-tuning the parameters of the deep learning object detection and semantic segmentation model using the incremental training samples.

[0149] In this embodiment, incremental training samples refer to image data and their annotation information collected from actual operation, containing real security event scenarios. These data can reflect the shortcomings of the model's performance in the real environment. The corresponding intermediate model results include the original detection boxes output by the object detection branch and the pixel-level classification maps output by the semantic segmentation branch. These data help to analyze the internal logic of the model in the event processing process. The final judgment label is the "real label" that the system infers from the trigger result of the security response strategy, which is used to correct the deviation of the model's original prediction.

[0150] Specifically, after each safety response is triggered (such as an alarm or door linkage control), the system automatically extracts a continuous image sequence (with the same frame rate as the original acquisition) from video storage, spanning from 30 seconds before the event to 10 seconds after the event. Simultaneously, it saves the intermediate outputs of the target detection model (such as bounding box coordinates, confidence scores, and category labels) and the intermediate outputs of the semantic segmentation model (such as pixel-level segmentation results of the door gap area) from memory during this period. The extracted image sequence is then manually or through rules to perform secondary annotation, confirming the targets that truly pose a safety risk (such as trapped limbs or clothing) and correcting errors in the original model annotations, forming "real labels." The image sequence, intermediate results, and real labels are aligned by time to construct a structured incremental training sample set, ensuring that each sample contains three parts: input data, model intermediate state, and target output. When periodically fine-tuning the model's parameters using incremental training samples, a mini-batch stochastic gradient descent method is employed, using only the most recently collected increments each time. The model weights are updated based on samples to reduce computational costs. During fine-tuning, only the parameters of the target detection and semantic segmentation branches of the deep learning target detection and semantic segmentation models (such as the feature extraction layer) are adjusted to preserve the model's generalization ability for common scenarios. The fine-tuning frequency is set according to the accumulation rate of new samples (such as once a week), and the number of iterations for each fine-tuning is controlled within 10-20 rounds to prevent overfitting. The fine-tuned model needs to be evaluated for its detection accuracy, recall, and false positive rate using a retained validation set (including historical normal scenarios and extreme scenarios). Only when all indicators meet the preset thresholds (such as detection accuracy ≥ 95% and false positive rate ≤ 2%) will the new model be deployed to the actual system to replace the original model. This continuous optimization mechanism enables the model to dynamically adapt to changes in the door structure (such as changes in the shape of gaps caused by wear), changes in lighting conditions (such as day and night cycles), and new risk targets (such as newly added backpack strap types), always maintaining a high-precision identification capability for safety risks.

[0151] In one embodiment, the method further includes a data management and traceability step, which includes:

[0152] D1: Store all alarm events, corresponding image evidence, model analysis logs, and linkage control records in a structured format on the vehicle storage device;

[0153] D2: During idle periods, the stored data is uploaded in batches to the ground data center via the vehicle-to-ground wireless network for the purpose of generating safety reports and operational analysis.

[0154] In this embodiment, structured format refers to organizing data according to a predefined data pattern to ensure that each field (such as event ID, timestamp, door number, alarm type, image index, control command code, execution result, etc.) has a clear meaning and is easy to parse; on-board storage device refers to the train's local non-volatile memory (such as solid-state drive); vehicle-to-ground wireless network refers to the wireless communication network between the train and the ground control center for data transmission; ground data center refers to the data processing and storage platform located in the operation control center.

[0155] Specifically, upon each alarm event (including real alarms and false alarms cleared after secondary confirmation), the system automatically associates and packages all relevant data: including basic metadata of the event (time, location, type), original image sequences or screenshots within key time windows before and after the trigger moment, intermediate logs during model analysis (such as coordinates and confidence levels of each detection box, contour data of the segmentation mask), and instructions and execution feedback records issued to the door control system; this data is packaged into a structured data packet, with a unique event ID and timestamp, and stored in a dedicated partition of the onboard storage device, forming a complete event record; the system utilizes the train's stops at the terminal station, During periods of network idleness and sufficient bandwidth, such as when vehicles return to the depot at night, the data synchronization task is automatically initiated. The vehicle-mounted communication module securely and in batches transmits the accumulated unuploaded event data packets from the storage device to the ground data center via the vehicle-to-ground wireless network. After receiving the data, the ground data center parses and stores it, and automatically generates various safety operation reports based on this detailed data (such as event statistics, type distribution, and high-risk period / door analysis in daily and weekly reports). The data also provides a solid data foundation for operational analysis (such as driver operation habit assessment and identification of risk points in specific route environments) and further system optimization (such as providing a wider range of training sample sources for the above model optimization steps).

[0156] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0157] In one embodiment, an intelligent rail transit door monitoring and anomaly recognition system is provided, which corresponds one-to-one with the intelligent rail transit door monitoring and anomaly recognition method described in the above embodiments. For example... Figure 7 As shown, this intelligent rail transit vehicle door monitoring and anomaly recognition system includes:

[0158] The image acquisition module is deployed at each door to collect image data of the area around the door in real time.

[0159] The intelligent analysis module is communicatively connected to the image acquisition module and is used to process the image data. The intelligent analysis module includes a target detection and processing unit and a door status monitoring unit. The target detection and processing unit is used to run the deep learning target detection and semantic segmentation model to perform door closure status recognition and foreign object detection in the gap area. The door status monitoring unit is connected to the target detection and processing unit and is used to determine the door status and analyze whether the safety risk target exists based on the recognition and detection results.

[0160] The alarm and notification module is connected to the door status monitoring unit and is used to trigger the safety response strategy when the analysis result indicates that there is a safety risk target, send an alarm to the driver's console and notify the background system.

[0161] The data storage and analysis module is connected to the intelligent analysis module and the alarm and notification module, and is used to store monitoring data and abnormal records.

[0162] The background management and monitoring module is connected to the data storage and analysis module to remotely monitor the system status and perform data backtracking and system optimization.

[0163] As described above, it is understood that each component of the intelligent rail transit door monitoring and anomaly identification system proposed in this application can realize the function of any of the intelligent rail transit door monitoring and anomaly identification methods described above, and the specific structure will not be repeated.

[0164] The present invention and its embodiments have been described above. This description is not restrictive. The accompanying drawings are only one embodiment of the present invention. The actual structure is not limited to this. In short, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the present invention, such design should fall within the protection scope of the present invention.

Claims

1. A method for monitoring and identifying anomalies in intelligent rail transit vehicle doors, characterized in that, The intelligent rail transit door monitoring and anomaly detection method includes the following steps: Real-time acquisition of image data for each door area; The image data is preprocessed and input into the built-in deep learning object detection and semantic segmentation model to perform door closure status recognition and foreign object detection in the gap area in parallel. Based on the door closure status recognition result, it is determined whether the door is currently fully closed, has a gap, or is not closed. If it is determined that there is a gap or that the gap is not closed, then based on the foreign object detection results in the gap area, it is analyzed whether there is a predefined safety risk target in the gap; When the analysis results indicate the existence of a safety risk target, a preset safety response strategy is triggered. The safety response strategy includes sending a real-time alarm command to the driver's console and coordinating with the door control system.

2. The intelligent rail transit door monitoring and anomaly identification method according to claim 1, characterized in that, The preprocessing of image data includes: Illumination effects are eliminated from the acquired continuous images to reduce the impact of changes in ambient lighting on image analysis. Based on the processed image, key regions related to door movement and security monitoring are extracted to obtain target images for model analysis.

3. The intelligent rail transit door monitoring and anomaly identification method according to claim 1, characterized in that, The deep learning-based object detection and semantic segmentation model includes a shared feature extraction backbone network, a semantic segmentation branch network for state recognition, and an object detection branch network for foreign object detection. The feature extraction backbone network is used to extract multi-scale feature maps from the input image; The semantic segmentation branch network processes the multi-scale feature map and outputs pixel-level segmentation masks for the door leaf edge and the sealing strip outline; The target detection branch network processes the multi-scale feature map and outputs the bounding box, category, and confidence score of each detected target within the door gap area.

4. The intelligent rail transit door monitoring and anomaly identification method according to claim 3, characterized in that, The determination of whether the car door is currently fully closed, has a gap, or is not closed specifically includes: Based on the segmentation mask output by the semantic segmentation branch network, the horizontal pixel distance between the center lines of the left and right door edge masks is calculated as an estimated value of the gap width. The estimated gap width is compared with a first preset threshold. If the estimated gap width is less than or equal to the first preset threshold, it is determined to be in a completely closed state; If the estimated gap width is greater than the first preset threshold but less than the second preset threshold, it is determined that a gap exists. If the estimated gap width is greater than or equal to the second preset threshold, it is determined to be in an unclosed state.

5. The intelligent rail transit door monitoring and anomaly identification method according to claim 3, characterized in that, The analysis of whether there are predefined security risk targets within the gap includes: Receive the detection results output by the target detection branch network; Targets whose bounding box center point falls within the preset gap danger zone geometric model are selected; The categories of the selected detection targets are matched with a predefined risk category library, which includes clothing, backpack straps, and body parts; Based on a pre-set confidence threshold, if there is a detection target that matches successfully and has a confidence level higher than the threshold, it is determined to be a target with a security risk.

6. The intelligent rail transit door monitoring and anomaly identification method according to claim 5, characterized in that, After determining that a target poses a security risk, the process also includes a target risk confirmation step: In a continuously acquired sequence of image data, multi-target tracking is performed on targets identified as safety risks to obtain their motion trajectories; The number of frames in which the target remains within the preset gap danger zone geometric model within a preset time window is counted. If the number of consecutive frames exceeds a preset threshold, the security risk target is confirmed, and the step of triggering the security response strategy is executed.

7. The intelligent rail transit door monitoring and anomaly identification method according to claim 1, characterized in that, The linkage control with the vehicle door control system includes: Generate standard control instructions corresponding to anomaly types and risk levels; The standard control commands are sent to the corresponding door electronic control unit via the vehicle bus network; The door electronic control unit executes at least one preset action based on the received command: emergency stop closing, anti-pinch release and retraction, or keeping the door open.

8. The intelligent rail transit door monitoring and anomaly identification method according to claim 1, characterized in that, The method further includes a model optimization step, which includes: After each security response strategy is triggered, image sequences and corresponding intermediate model results and final judgment labels within a set time period before and after the event are automatically captured to form incremental training samples. The parameters of the deep learning object detection and semantic segmentation model are periodically fine-tuned using the incremental training samples.

9. The intelligent rail transit door monitoring and anomaly identification method according to claim 1, characterized in that, The method further includes a data management and traceability step, which includes: All alarm events, corresponding image evidence, model analysis logs, and linkage control records are stored in a structured format on the vehicle's storage device. During off-peak hours, stored data is uploaded in batches to the ground data center via vehicle-to-ground wireless network to generate safety reports and operational analysis.

10. An intelligent rail transit vehicle door monitoring and anomaly recognition system, used to implement the intelligent rail transit vehicle door monitoring and anomaly recognition method as described in any one of claims 1 to 9, characterized in that, include: The image acquisition module is deployed at each door to collect image data of the area around the door in real time. The intelligent analysis module is communicatively connected to the image acquisition module and is used to process the image data. The intelligent analysis module includes a target detection and processing unit and a door status monitoring unit. The target detection and processing unit is used to run the deep learning target detection and semantic segmentation model to perform door closure status recognition and foreign object detection in the gap area. The door status monitoring unit is connected to the target detection and processing unit and is used to determine the door status and analyze whether the safety risk target exists based on the recognition and detection results. The alarm and notification module is connected to the door status monitoring unit and is used to trigger the safety response strategy when the analysis result indicates that there is a safety risk target, send an alarm to the driver's console and notify the background system. The data storage and analysis module is connected to the intelligent analysis module and the alarm and notification module, and is used to store monitoring data and abnormal records. The background management and monitoring module is connected to the data storage and analysis module to remotely monitor the system status and perform data backtracking and system optimization.

Images