Safety monitoring method and system for subway platform screen doors

Abstract

This application relates to the field of rail transit technology and discloses a safety monitoring method and system for subway platform screen doors. The method includes: detecting the current motion state of the platform screen door; when the current motion state is the closed operating state, obtaining point cloud data and image data around the door body; preprocessing the point cloud data to obtain first point cloud data; fusing the first point cloud data with the image data to obtain three-dimensional door body data; identifying and analyzing the three-dimensional door body data to detect whether there is a foreign object in the door gap; if so, identifying and locating the foreign object, issuing a foreign object alarm signal, and triggering the door to stop or reverse open. This application improves the precision, adaptability, and accuracy of safety monitoring and reduces false alarms.

Description

Technical Field

[0001] The present application relates to the field of rail transportation technology, and in particular to a safety monitoring method and system for subway platform screen doors. Background Art

[0002] Subway platform screen doors are protective devices installed between subway stations and trains to ensure passenger safety. They play a vital role in preventing misconduct, avoiding accidents, and protecting passengers. Due to their importance, the operating status of platform screen doors requires constant monitoring, especially when potential failures or safety hazards arise. Effective monitoring methods are needed to promptly identify and address any issues. Existing monitoring methods often rely on manual inspections or simple sensor alarms, resulting in low efficiency, poor real-time performance, and high false alarm rates. Therefore, developing an efficient, real-time, and reliable method for monitoring subway platform screen door safety is crucial.

[0003] Similar prior art includes Chinese patent publication number CN108520243B, which discloses a PSD foreign object detection and alarm system. This system pre-stores a baseline PSD image and uses an inter-door data capture device to obtain multiple, temporally continuous frames of live PSD images. Each frame is analyzed to obtain C, M, and Y channel data for each pixel. The images are then converted into C, M, and Y color matrices. The system then checks whether a value at a specific location in each color matrix deviates from the mean of the values ​​at surrounding locations by more than a certain threshold. Based on the analysis results, multiple images to be analyzed are obtained and output. Each image to be analyzed is then matched with the baseline PSD image. If the degree of match is less than or equal to a preset percentage threshold, the image to be analyzed is determined to be a foreign object image. This method, which relies on image matching and color analysis to detect PSD foreign objects, suffers from low detection accuracy and poor environmental adaptability. Another Chinese patent application, publication number CN109447090A, discloses a method and system for detecting obstacles in a platform screen door. The system captures an image of the area between the platform screen door and the vehicle door. The captured image is compared with a scene change calibration image to determine whether a significant scene change has occurred. If not, the captured image is fed into an obstacle recognition model for obstacle recognition, which then uses the model to determine whether an obstacle is present. If so, the image of the detection area in the captured image is compared with the image of the corresponding detection area in the training sample image to determine whether an obstacle is present. This method, which relies on image comparison and model recognition for obstacle detection, suffers from low detection accuracy and poor environmental adaptability.

[0004] Therefore, it is an urgent problem to provide a safety monitoring method and system for subway platform screen doors to improve the precision, adaptability and accuracy of safety monitoring. Summary of the Invention

[0005] The present application provides a safety monitoring method and system for subway platform screen doors. By adopting multimodal data fusion technology, the platform screen door area can be comprehensively and accurately monitored, thereby reducing false alarms caused by environmental changes and improving the precision and accuracy of detecting objects stuck in door gaps and foreign objects. The system has strong adaptability and can ensure the safe operation of the subway in different complex environments.

[0006] In a first aspect, the present application provides a safety monitoring method for a subway platform screen door, the method comprising:

[0007] Step 1: Detect the current motion state of the screen door. When the current motion state is the closed running state, obtain the point cloud data and image data around the door body;

[0008] Step 2: Use the 3D point cloud self-learning background algorithm to pre-process the point cloud data to obtain the first point cloud data;

[0009] Step 3: Fusing the first point cloud data and the image data to obtain three-dimensional data of the door body;

[0010] Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is any foreign object in the door gap. If so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse open control.

[0011] In conjunction with the first aspect, in a first implementation of the first aspect of the present application, step 3 includes:

[0012] Step 31: Define the target object in the first point cloud data as a first object, identify all first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate a first geometric feature array corresponding to each first key point;

[0013] Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array;

[0014] Step 33: define the target object in the image data as a second object, identify all second objects in the image data, extract any second object, and extract N2 second key points of any second object, and calculate a second visual feature array corresponding to each second key point;

[0015] Step 34: extract a first visual feature array corresponding to any first key point of any first object, calculate the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, and when the Euclidean distance is less than a first preset value, determine that any second key point is a corresponding point of any first key point, extract other first key points of any first object, and other second key points of the reference second object, and determine whether other first key points have one-to-one corresponding points in other second key points. If so, establish a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points. If not, repeat step 34 until a second object having an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array.

[0016] Step 35: After traversing all first objects, based on the first association relationship and the second association relationship, map the point cloud data of any first object to the image data of the corresponding second object to generate three-dimensional data of the door body.

[0017] In combination with the first aspect, in a second implementation of the first aspect of the present application, the first geometric feature array includes curvature, normal vector, point cloud density, and geometric shape, the visual feature array includes color features, texture features, edge features, and shape features, and the first geometric feature array and the second visual feature array are generated by:

[0018] For any first key point, taking any first key point as the origin, obtaining point cloud data within a first preset range, defining it as reference point cloud data, and calculating a first geometric feature array based on the reference point cloud data;

[0019] For any second key point, take any second key point as the origin, obtain image data within a second preset range, define it as reference image data, and calculate the second visual feature array based on the reference image data.

[0020] In combination with the first aspect, in a third implementation of the first aspect of the present application, the method for generating the first preset model is:

[0021] Building a simulation model based on the operating environment of the subway platform screen door, and collecting second point cloud data and first image data of the simulation model at a preset collection point;

[0022] The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated;

[0023] The second geometric feature array of any third key point and the third visual feature array of the fourth key point corresponding to any third key point are used as a pair of training data to construct a training data set;

[0024] Construct a neural network model, train the neural network model using a training data set, and generate a first preset model.

[0025] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, step 4 includes:

[0026] Step 41: extract any first object, calculate the average distance between the object point cloud data and the point cloud acquisition device, and if the average value is less than or equal to a second preset value, define the first object as a specific target object and mark the pixel corresponding to the object point cloud data as a first pixel; otherwise, define the first object as a to-be-filtered object and mark the pixel corresponding to the object point cloud data as a second pixel, wherein the object point cloud data is the point cloud data corresponding to the first object;

[0027] Step 42: extracting any pixel point adjacent to the reference pixel point from the image data, defining it as a third pixel point; calculating a correlation between the reference pixel point and the third pixel point based on the pixel value and / or pixel coordinates; and marking the third pixel point as the reference pixel point when the correlation degree is greater than or equal to a third preset value, wherein the reference pixel point is the first pixel point or the second pixel point;

[0028] Step 43: Extract unlabeled pixels and define them as fourth pixels. Define the minimum distance between the fourth pixel and an adjacent first pixel as a first distance, and define the minimum distance between the fourth pixel and an adjacent second pixel as a second distance. If the first distance is greater than the second distance, label the fourth pixel as the first pixel; otherwise, label it as the second pixel.

[0029] Step 44: Set the pixel value of the first pixel to 255, set the pixel value of the second pixel to 0, obtain third image data, extract the edge line of the specific target object based on the third image data, and obtain an edge feature vector of the edge line;

[0030] Step 45: Input the edge feature vector of any specific target object into the second preset model to obtain the category of any specific target object.

[0031] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, the edge line is encoded to obtain an edge feature vector, and the edge feature vector includes Fourier transform, Zernike moment, contour histogram and / or shape context.

[0032] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, a first monitoring device and a second monitoring device are respectively provided in the horizontal direction and the vertical direction of the screen door, monitoring data collected by the first monitoring device and the second monitoring device are respectively obtained, and whether there is a foreign object in the door gap is detected based on the monitoring data;

[0033] The analysis device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, any one of the monitoring devices is determined to be abnormal.

[0034] In combination with the first aspect, in a seventh implementation of the first aspect of the present application, when the number of times any monitoring device is judged to be abnormal exceeds a fourth preset value, any monitoring device is judged to be faulty and a device fault alarm message is sent.

[0035] In a second aspect, the present application provides a safety monitoring system for subway platform screen doors, the system comprising: a state detection module, a data acquisition module, a preprocessing module, a data fusion module and a foreign object detection module;

[0036] A state detection module is used to detect the current motion state of the shielding door;

[0037] The data acquisition module is used to obtain point cloud data and image data around the door body when the current motion state is the closed operation state;

[0038] A preprocessing module, configured to preprocess the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data;

[0039] A data fusion module is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body;

[0040] The foreign object detection module is used to identify and analyze the three-dimensional data of the door body, detect whether there is any foreign object in the door gap, and if so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse opening control.

[0041] Compared with the prior art, the beneficial effects of the technical solution of this application are at least as follows:

[0042] 1. Through the fusion processing of 3D point cloud data and image data, it can accurately detect foreign objects in door gaps under complex backgrounds, avoiding safety accidents caused by foreign objects being clamped. It has high detection accuracy and strong adaptability.

[0043] 2. Use 3D point cloud self-learning background algorithm to pre-process point cloud data, automatically distinguish point cloud changes caused by vehicle body shaking from real foreign object signals, and automatically adapt to different environmental changes, reducing the need for manual intervention and improving the accuracy, intelligence and automation level of detection.

[0044] 3. Feature extraction and classification recognition are performed through preset models and neural network models, which improves the speed and efficiency of data processing, enabling the system to complete the monitoring and analysis of the environment around the door in a short time, ensuring real-time performance.

[0045] 4. By setting up multiple monitoring devices in different directions of the shielding door and performing data cross-verification, the reliability of the system is improved. Even if a device fails, other devices can still work normally, ensuring the continuity of monitoring and being able to quickly identify monitoring device failures.

[0046] 5. When a foreign object is detected in the door gap, an alarm signal is issued and the door is triggered to stop or reverse open, effectively preventing potential safety accidents. This not only ensures the safety of passengers, but also improves their travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of an embodiment of a safety monitoring method for subway platform screen doors in an embodiment of the present application;

[0049] Figure 2 This is a schematic diagram of an embodiment of edge line extraction of a specific target object in an embodiment of the present application;

[0050] Figure 3 This is a schematic diagram of an embodiment of setting a monitoring device on a shielding door in an embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of an embodiment of a safety monitoring system for subway platform screen doors in an embodiment of the present application;

[0052] Among them, SD is a screen door, ME1 is a first monitoring device, and ME2 is a second monitoring device. DETAILED DESCRIPTION

[0053] The embodiments of the present application provide a safety monitoring method and system for subway platform screen doors. The terms "first," "second," "third," "fourth," and so on (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0054] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In one embodiment of the present application, a safety monitoring method for a subway platform door includes:

[0055] Step 1: Detect the current motion state of the shielding door. When the current motion state is the closed running state, obtain the point cloud data and image data around the door body.

[0056] Specifically, laser sensors are distributed around the shielding door at a certain angle and distance to cover all possible gap areas. They scan the surrounding environment of the door at a high frequency and obtain three-dimensional point cloud data of the surrounding environment. After obtaining the three-dimensional point cloud data of each laser sensor, the above three-dimensional point cloud data are fused and spliced ​​to obtain point cloud data of the entire shielding door monitoring area; 2D cameras are installed on both sides or above the shielding door to obtain image data of the door gap and objects around the door in real time. After obtaining the image data of each camera, the above image data are spliced ​​to obtain image data of the entire shielding door monitoring area.

[0057] The current motion state includes static, open, and closed states. The closed state is essential for ensuring safety monitoring is activated at critical moments, as foreign objects trapped in the door gap only pose a threat to passenger safety during the closing process. When the platform screen door is in the closed state, the system automatically activates the data acquisition device to acquire point cloud and image data around the door. This ensures timely and targeted data collection, avoids invalid data collection, and improves system efficiency.

[0058] Step 2: Use the 3D point cloud self-learning background algorithm to preprocess the point cloud data to obtain the first point cloud data.

[0059] Specifically, through the self-learning mechanism, the algorithm can adapt to dynamic changes in the environment, such as vehicle body shaking, effectively remove point cloud changes caused by vehicle body shaking or other dynamic factors, and extract normal point clouds when the object is stationary. These point clouds reflect the static characteristics of the environment around the shield door, providing a basis for subsequent foreign object detection and safety warnings.

[0060] Step 3: Fuse the first point cloud data and the image data to obtain three-dimensional data of the door body.

[0061] Specifically, point cloud data provides spatial geometric information about objects (such as position, shape, size, and distance), but its sparse distribution at long distances and in the vertical direction can result in incomplete target outlines. It also lacks semantic features like color and texture, making it difficult to accurately identify obstacles. Image data provides visual information about objects (such as color, texture, and edges), but lacks depth information, making it difficult to accurately determine the three-dimensional position and distance of obstacles. Image data is also susceptible to factors such as lighting, shadows, and reflections. By combining information from both data sources, we can overcome the shortcomings of a single data type and leverage the strengths of each to achieve a more comprehensive and accurate description of objects. This improves the system's adaptability to environmental changes, more accurately identifies and locates foreign objects in door gaps, reduces the likelihood of false positives and missed negatives, and enhances robustness.

[0062] Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is any foreign object in the door gap. If so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse open control.

[0063] Specifically, based on the fusion of 2D and 3D data, the system detects whether there is any foreign object (human body or object) trapped during the closing process of the shield door. If a foreign object is trapped, the system will send an alarm signal through the alarm indicator light or communication interface, and transmit the signal to the shield door control system, which will promptly start the door to stop or reverse opening to avoid accidents and improve safety.

[0064] In a specific embodiment, the process of executing step 3 may specifically include the following steps:

[0065] Step 31: Define the target object in the first point cloud data as a first object, identify all first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate the first geometric feature array corresponding to each first key point.

[0066] Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array.

[0067] Step 33: define the target object in the image data as a second object, identify all second objects in the image data, extract any second object, and extract N2 second key points of any second object, and calculate the second visual feature array corresponding to each second key point.

[0068] Step 34: extract the first visual feature array corresponding to any first key point of any first object, calculate the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, and when the Euclidean distance is less than a first preset value, determine that any second key point is the corresponding point of any first key point, extract other first key points of any first object, and other second key points of the reference second object, and determine whether the other first key points have one-to-one corresponding points in the other second key points. If so, establish a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points. If not, repeat step 34 until a second object with an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array.

[0069] Step 35: After traversing all first objects, based on the first association relationship and the second association relationship, map the point cloud data of any first object to the image data of the corresponding second object to generate three-dimensional data of the door body.

[0070] Specifically, the first point cloud data and image data originate from different data acquisition devices, potentially leading to inconsistent coordinate systems and resolution differences, necessitating data alignment and fusion. The target objects are objects contained in the first point cloud data / image data, including the screen door body, obstacles between the door gaps, and objects in its surroundings. Exemplarily, a clustering algorithm (such as DBSCAN, K-Means, or Euclidean clustering) is used to partition the first point cloud data into clusters, each representing a target object. The contours of each target object in the image data are extracted using edge detection algorithms, contour extraction algorithms, or semantic segmentation models to identify all second objects.

[0071] Key points are feature points with significant geometric variations related to the three-dimensional shape of the target object, such as corners, edges, or points in areas of high curvature. Key points in the first object are defined as first key points, and key points in the second object are defined as second key points. The geometric characteristic array corresponding to the first key point is defined as a first geometric feature array. The first geometric feature array is an array representing the geometric or topological features of the three-dimensional point cloud data of the target object at the first key point. The visual feature array corresponding to the second key point is defined as a second visual feature array. The second visual feature array is an array representing the features of the two-dimensional image data of the target object at the second key point. The first geometric feature array represents the features of the three-dimensional shape, while the second visual feature array represents the features of the two-dimensional shape. They cannot be directly compared. The first preset model is used to convert the first geometric feature array into a first visual feature array that can be compared with the two-dimensional feature array.

[0072] The first preset value is a preset value used to determine whether the first key point and the second key point have a corresponding relationship. It is set based on the experience of those skilled in the art or based on actual application scenarios, and is not limited in the embodiments of the present application. For example, the first preset value is 0.1 or 0.3.

[0073] Exemplarily, the first preset value is set to 0.2, the first object A has 3 first key points (a1, a2, a3), the first visual feature array corresponding to a1 is [0.8, 0.6, 0.7], the first visual feature array corresponding to a2 is [0.5, 0.9, 0.3], and the first visual feature array corresponding to a3 is [0.4, 0.7, 0.5]. The second object B has 3 second key points (b1, b2, b3), the second visual feature array corresponding to b1 is [0.7, 0.5, 0.6], the second visual feature array corresponding to b2 is [0.4, 0.8, 0.2], and the second visual feature array corresponding to b3 is [0.3, 0.6, 0.4]. First, it is determined that a1 and b1 are corresponding points, and then a2 and a3, b2 and b3 are extracted for judgment. The Euclidean distance d between a2 and b2 can be obtained by calculation. 22 is 0.173, and the Euclidean distance d between a2 and b3 23 is 0.374, d 22 is less than the first preset value, a2 and b2 are corresponding points, and the Euclidean distance d between a3 and b3 33 is 0.173, d 33 is less than the first preset value, a3 and b3 are corresponding points, then the first object A and the second object B have an association relationship, a1 and b1, a2 and b2, a3 and b3 have an association relationship.

[0074] According to the above technical solution, even if there are deviations in the installation positions of the point cloud acquisition equipment and the image acquisition equipment, the matching relationship between each target object and each key point can be quickly identified, and the three-dimensional data of the door body can be automatically generated.

[0075] In a specific embodiment, the first geometric feature array includes curvature, normal vector, point cloud density, and geometric shape, and the visual feature array includes color features, texture features, edge features, and shape features. The first geometric feature array and the second visual feature array are generated by:

[0076] (1) For any first key point, take any first key point as the origin, obtain point cloud data within a first preset range, define it as reference point cloud data, and calculate a first geometric feature array based on the reference point cloud data.

[0077] (2) For any second key point, take any second key point as the origin, obtain image data within a second preset range, define it as reference image data, and calculate the second visual feature array based on the reference image data.

[0078] Specifically, the first geometric feature array is an array of quantized values ​​used to describe the geometric or topological features of the local area around the first key point. It is a multidimensional vector that contains detailed information about the point cloud data around the first key point; the visual feature array is an array of feature parameters used to describe the local area around the key point. It is a multidimensional vector that contains visual information about the image data around the key point.

[0079] The first preset range is the range of local point cloud data that needs to be considered when calculating the first geometric feature array, and the second preset range is the range of local image data that needs to be considered when calculating the second visual feature array. Both are set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiments of the present application do not limit this. For example, the first preset range is a sphere with the first key point as the center and a radius of 0.1 meters, which is specifically selected based on the density of the point cloud data and the size of the target object; the second preset range is a circle with the second key point as the center and a radius of 10 pixels, which is specifically selected based on the resolution of the image and the size of the target object. The first preset range can also be a cube, a cuboid, etc., and the second preset range can also be a square, a rectangle, etc.

[0080] In a specific embodiment, the method for generating the first preset model is:

[0081] (1) A simulation model is constructed based on the operating environment of the subway platform screen door, and the second point cloud data and the first image data of the simulation model are collected at the preset collection points.

[0082] (2) The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated.

[0083] (3) The second geometric feature array of any third key point and the third visual feature array of the fourth key point corresponding to any third key point are used as a pair of training data to construct a training data set.

[0084] (4) Construct a neural network model, use the training data set to train the neural network model, and generate a first preset model.

[0085] Specifically, to generate a training dataset, a 3D simulation model of the subway platform door and its surrounding environment is constructed using 3D modeling software (such as Blender, 3ds Max, etc.). The simulation model should reflect the actual situation as realistically as possible, including the geometric shape, material, lighting, etc. of the target object. Point cloud data and image data similar to the real environment are generated through the simulation model for model training.

[0086] Preset collection points are pre-set data collection points used to collect point cloud and image data. Their locations are determined based on the simulated operating environment and monitoring requirements of the subway platform screen doors, ensuring coverage of critical areas such as door gaps and door edges. Multiple preset collection points can be set within the simulation model to simulate the installation positions and angles of data collection equipment in real-world applications. Since point cloud and image acquisition devices can be set up at the same data collection points within the simulation space, point cloud and image data can be obtained from common data collection points scanning and capturing images in common directions. This ensures the generation of high-quality training data, improving the model's generalization capabilities and the reliability of the monitoring system.

[0087] Pair the geometric features in the point cloud data with the visual features in the image data to form a training dataset. Use a neural network model (such as a fully connected neural network, convolutional neural network, deep neural network, etc.) to learn the training dataset and generate a preset model that can map the geometric feature array to the visual feature array.

[0088] Since the point cloud acquisition device and the image acquisition device are set at the same position, there is no deviation between the point cloud data and image data of the same target object. A large amount of training data can be obtained in the simulation model. By generating training data through the simulation model, the demand for real data collection is reduced, the cost is reduced, and the tediousness and errors of manual data annotation are avoided. The training efficiency and accuracy of the preset model are improved, the robustness and generalization ability of the system are enhanced, and reliable technical support is provided for the safety monitoring of subway platform doors.

[0089] In a specific embodiment, the process of executing step 4 may specifically include the following steps:

[0090] Step 41: Extract any first object, calculate the average value of the distance between the object point cloud data and the point cloud acquisition device, and if the average value is less than or equal to a second preset value, define any first object as a specific target object, and mark the pixel corresponding to the object point cloud data as a first pixel; otherwise, define it as an object to be filtered, and mark the pixel corresponding to the object point cloud data as a second pixel, wherein the object point cloud data is the point cloud data corresponding to any first object.

[0091] Step 42: extract any pixel point adjacent to the reference pixel point from the image data, define it as the third pixel point, calculate the correlation between the reference pixel point and the third pixel point based on the pixel value and / or pixel coordinates, and when the correlation is greater than or equal to a third preset value, mark the third pixel point as the reference pixel point, wherein the reference pixel point is the first pixel point or the second pixel point.

[0092] Step 43: Extract unlabeled pixel points and define them as the fourth pixel points. Define the minimum distance between the fourth pixel point and the adjacent first pixel point as the first distance, and define the minimum distance between the fourth pixel point and the adjacent second pixel point as the second distance. If the first distance is greater than the second distance, label the fourth pixel point as the first pixel point; otherwise, label it as the second pixel point.

[0093] Step 44: Set the pixel value of the first pixel to 255, set the pixel value of the second pixel to 0, obtain third image data, extract the edge line of the specific target object based on the third image data, and obtain the edge feature vector of the edge line.

[0094] Step 45: Input the edge feature vector of any specific target object into the second preset model to obtain the category of any specific target object.

[0095] Specifically, the specific target object is the object to be detected and identified, i.e., an obstacle located in the gap of the shielding door, while the objects to be filtered are objects that do not need to be detected and identified, such as background objects, the door body, etc. Since fixed objects such as the door body are at a certain distance from the point cloud acquisition device, the door body, background, etc., and obstacles in the door gap can be classified by distance judgment, distinguishing between objects that require special attention (such as obstacles) and objects that can be ignored. The second preset value is a preset value used to determine whether the first object is a specific target object. It is set based on the experience of those skilled in the art or according to actual application scenarios, and is not limited to this in the embodiments of this application.

[0096] When mapping point cloud data to image data, the pixels on the image data that correspond to the point cloud data (i.e., reference pixels) can be annotated to determine whether these pixels belong to a specific target object or an object to be filtered. Subsequently, based on the correlation between other pixels and the reference pixels, the other pixels can be classified as first pixels or second pixels. For example, there are three connected pixels C1, C2, and C3, where C1 is the first pixel, C3 is the second pixel, and C2 has not been annotated. Since C2 is adjacent to C1 and C3, the correlation R between C2 and C1 is calculated. 21 , the correlation R between C2 and C3 23 , calculate the correlation R 21 When , the Euclidean distance between C2 and C1 is calculated based on the pixel value, and the reciprocal of the Euclidean distance is used as the color similarity. Then the spatial distance between C2 and C1 is calculated based on the pixel coordinates, and the reciprocal of the spatial distance is used as the distance similarity. Finally, the color similarity and distance similarity are weighted averaged to obtain the correlation R. 21 Calculate the correlation R using the same method 23 , compare the correlation R 21 and correlation R 23 If the correlation R 21 Greater than the correlation R 23 , then the correlation R 21 As a reference correlation, when the reference correlation is greater than the third preset value, the pixel C2 is marked as the first pixel. When there is only one reference pixel adjacent to any unmarked pixel, only one correlation calculation and comparison is required. The third preset value is a preset value used to determine whether the third pixel is to be marked as a reference pixel. It is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiment of the present application does not limit this. Some pixels may not be successfully marked due to insufficient color information or sparse point cloud. By considering the spatial distance, these pixels are further marked, and the pixels to be identified are classified into specific target objects or objects to be filtered, further improving the regional division of foreign body detection, which can effectively fill the gaps in regional division and improve the integrity of detection.

[0097] By setting the pixel value of the first pixel point to 255 and the pixel value of the second pixel point to 0, a specific target object (such as an obstacle) is highlighted, making it easier to extract the contour line of the specific target object, such as Figure 2 As shown, high contour extraction accuracy can be maintained even in complex scenes.

[0098] The second preset model is a pre-trained neural network model that analyzes the edge feature vectors of the target object to obtain the category of the target object (such as a passenger's hand, plastic bag, bag or luggage, etc.). By learning the edge features of different categories of foreign objects, it realizes automatic classification and recognition of specific target objects.

[0099] In a specific embodiment, edge lines are encoded to obtain edge feature vectors, where the edge feature vectors include Fourier transform, Zernike moments, contour histograms, and / or shape contexts.

[0100] Specifically, the edge feature vector is a mathematical description of the edge of an object. Fourier transform is used to perform a Fourier transform on the edge line to extract its frequency domain features; Zernike moments are used to calculate the Zernike moments of the edge line to extract its shape features; contour histograms are used to generate contour histograms of the edge line to extract its contour distribution features; and shape context is used to characterize the shape by calculating the relative positional relationship between edge points and other points, extracting the local detail information it provides. By combining multiple features, the edge feature vector can provide a more comprehensive and accurate description of the edge. While improving the ability to distinguish different types of foreign objects, it also improves the edge feature vector's adaptability to environmental changes and reduces the possibility of false positives and false negatives. For example, the edge feature vector can also include geometric features (area, perimeter, aspect ratio, convex hull, etc.), shape matching features (such as polygonal approximation), etc.

[0101] Preferably, the edge feature vector can be a one-dimensional vector of any one of the Fourier transform, Zernike moment, contour histogram, or shape context. Although simplifying the edge feature vector to a one-dimensional vector of a single feature will lose some information, it has obvious advantages in terms of computational efficiency, model complexity, implementation difficulty, and applicability to specific scenarios.

[0102] In a specific embodiment, a first monitoring device and a second monitoring device are respectively set in the horizontal direction and the vertical direction of the shielding door, and monitoring data collected by the first monitoring device and the second monitoring device are respectively obtained, and whether there is foreign matter in the door gap is detected based on the monitoring data.

[0103] The analysis device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, any one of the monitoring devices is determined to be abnormal.

[0104] like Figure 3 As shown, a first monitoring device ME1 is set in the horizontal direction of the shielding door SD, which is used to monitor the door gap area in the horizontal direction of the shielding door and detect whether there are foreign objects. The first monitoring device is an ultrasonic sensor, a laser beam detector, a photoelectric sensor or an infrared detector, etc. Based on the working principles of various types of monitoring devices, transmitters and receivers can be installed on the left and right sides of the shielding door; a second monitoring device ME2 is set in the vertical direction of the shielding door SD, which is used to monitor the door gap area in the vertical direction of the shielding door and detect whether there are foreign objects. The second monitoring device is a laser ranging detector, an ultrasonic sensor, a lidar, an infrared array sensor, etc. Based on the working principles of various types of monitoring devices, transmitters and receivers can be installed on the upper and lower sides of the shielding door.

[0105] Through collaborative monitoring of multiple devices, not only the accuracy of foreign object detection is improved, but also when a monitoring device fails, other devices can continue to work, avoiding complete failure of the monitoring system. The monitoring results of different devices can be verified with each other. While improving the accuracy of foreign object detection, it can quickly and accurately identify monitoring equipment failures, timely discover and deal with equipment failures, and provide more reliable safety protection for subway operations.

[0106] In a specific embodiment, when the number of times any monitoring device is determined to be abnormal exceeds a fourth preset value, any monitoring device is determined to be faulty, and a device fault alarm message is sent.

[0107] The fourth preset value is set based on the experience of those skilled in the art or based on actual application scenarios, and is not limited in the present embodiment.

[0108] By counting the number of times a device is judged to be abnormal, device failures can be identified more accurately. For example, if a device occasionally exhibits an abnormality, it may be due to instantaneous interference or environmental factors; and if the number of abnormalities exceeds the fourth preset value, it is more likely that there is a problem with the device itself. Based on this, false alarms can be reduced and the accuracy of fault detection can be improved.

[0109] The above describes the safety monitoring method for subway platform screen doors in the embodiment of the present application. The following describes the safety monitoring system for subway platform screen doors in the embodiment of the present application. Figure 4In an embodiment of the present application, a safety monitoring system for subway platform doors includes: a status detection module 10, a data acquisition module 20, a preprocessing module 30, a data fusion module 40 and a foreign object detection module 50.

[0110] The state detection module 10 is used to detect the current motion state of the platform shielding door.

[0111] The data acquisition module 20 is used to obtain point cloud data and image data around the door body when the current motion state is the closed operation state.

[0112] The preprocessing module 30 is used to preprocess the point cloud data using the 3D point cloud self-learning background algorithm to obtain first point cloud data.

[0113] The data fusion module 40 is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body.

[0114] The foreign object detection module 50 is used to identify and analyze the three-dimensional data of the door body, detect whether there is a foreign object in the door gap, and if so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse open control.

[0115] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0117] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A safety monitoring method for subway platform doors, characterized in that: The method comprises: Step 1: Detect the current motion state of the screen door. When the current motion state is a closed operating state, obtain point cloud data and image data around the door body. Step 2: Preprocess the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data, and adapt to dynamic changes in the environment through a self-learning mechanism; Step 3: Fusing the first point cloud data and the image data to obtain three-dimensional data of the door body; Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is any foreign object in the door gap. If so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse open control; The target object in the first point cloud data is defined as a first object, and the step 4 includes: Step 41: Extract any of the first objects, calculate the average distance between the object point cloud data and the point cloud acquisition device, and if the average value is less than or equal to a second preset value, define any of the first objects as a specific target object and mark the pixel corresponding to the object point cloud data as a first pixel; otherwise, define any of the first objects as a to-be-filtered object and mark the pixel corresponding to the object point cloud data as a second pixel, wherein the object point cloud data is the point cloud data corresponding to any of the first objects; Step 42: extracting any pixel point adjacent to the reference pixel point from the image data, defining it as a third pixel point; calculating a correlation between the reference pixel point and the third pixel point based on pixel values ​​and / or pixel coordinates; and when the correlation degree is greater than or equal to a third preset value, marking the third pixel point as the reference pixel point, wherein the reference pixel point is the first pixel point or the second pixel point; Step 43: Extract unlabeled pixels and define them as fourth pixels. Define the minimum distance between the fourth pixel and the adjacent first pixel as a first distance, and define the minimum distance between the fourth pixel and the adjacent second pixel as a second distance. If the first distance is greater than the second distance, label the fourth pixel as the first pixel; otherwise, label it as the second pixel. Step 44: Set the pixel value of the first pixel point to 255, set the pixel value of the second pixel point to 0, obtain third image data, extract the edge line of the specific target object based on the third image data, and obtain an edge feature vector of the edge line; Step 45: Input the edge feature vector of any of the specific target objects into a second preset model to obtain the category of any of the specific target objects.

2. The safety monitoring method for subway platform doors according to claim 1, characterized in that: The step 3 comprises: Step 31: Define the target object in the first point cloud data as a first object, identify all first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate a first geometric feature array corresponding to each first key point; Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array; Step 33: defining the target object in the image data as a second object, identifying all second objects in the image data, extracting any second object, and extracting N2 second key points of any second object, and calculating a second visual feature array corresponding to each second key point; Step 34: extract the first visual feature array corresponding to any first key point of any first object, calculate the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, and when the Euclidean distance is less than a first preset value, determine that any second key point is a corresponding point of any first key point, extract other first key points of any first object, and other second key points of the reference second object, and determine whether other first key points have one-to-one corresponding points in other second key points. If so, establish a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points. If not, repeat step 34 until a second object having an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array. Step 35: After traversing all the first objects, based on the first association relationship and the second association relationship, map the point cloud data of any first object to the image data of the corresponding second object to generate the three-dimensional data of the door body.

3. The safety monitoring method for subway platform doors according to claim 2, characterized in that: The first geometric feature array includes curvature, normal vector, point cloud density and geometric shape, and the visual feature array includes color features, texture features, edge features and shape features. The first geometric feature array and the second visual feature array are generated by: For any of the first key points, taking any of the first key points as an origin, obtaining point cloud data within a first preset range, defining the data as reference point cloud data, and calculating the first geometric feature array based on the reference point cloud data; For any of the second key points, image data within a second preset range is obtained with any of the second key points as the origin, defined as reference image data, and the second visual feature array is calculated based on the reference image data.

4. The safety monitoring method for subway platform doors according to claim 2, characterized in that: The method for generating the first preset model is: Building a simulation model based on the operating environment of the subway platform screen door, and collecting second point cloud data and first image data of the simulation model at a preset collection point; The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated; constructing a training data set by taking the second geometric feature array of any of the third key points and the third visual feature array of the fourth key point corresponding to any of the third key points as a pair of training data; Construct a neural network model, use the training data set to train the neural network model, and generate the first preset model.

5. The safety monitoring method for subway platform screen doors according to claim 1, characterized in that: The edge line is encoded to obtain the edge feature vector, where the edge feature vector includes Fourier transform, Zernike moment, contour histogram and / or shape context.

6. The safety monitoring method for subway platform doors according to claim 1, characterized in that: The method further comprises: A first monitoring device and a second monitoring device are respectively arranged in the horizontal direction and the vertical direction of the screen door, monitoring data collected by the first monitoring device and the second monitoring device are respectively obtained, and whether there is a foreign object in the door gap based on the monitoring data; The analysis device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, any one of the monitoring devices is determined to be abnormal.

7. The safety monitoring method for subway platform doors according to claim 6, characterized in that: When the number of times any of the monitoring devices is determined to be abnormal exceeds a fourth preset value, it is determined that any of the monitoring devices is faulty, and a device fault alarm message is sent.

8. The safety monitoring system for subway platform door safety monitoring method according to claim 1, characterized in that: The safety monitoring system includes a status detection module, a data acquisition module, a pre-processing module, a data fusion module and a foreign object detection module; The state detection module is used to detect the current movement state of the platform screen door; The data acquisition module is used to obtain point cloud data and image data around the door body when the current motion state is the closed operation state; The preprocessing module is used to preprocess the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data; The data fusion module is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body; The foreign object detection module is used to identify and analyze the three-dimensional data of the door body, detect whether there is a foreign object in the door gap, and if so, identify and locate the foreign object, issue a foreign object alarm signal, and trigger the door to stop or reverse open control.

Images