Safety monitoring method and system for metro platform screen door

Abstract

The invention relates to the technical field of rail transit, and discloses a safety monitoring method and system for a metro platform screen door. The method comprises the steps that the current motion state of the shielding door is detected, and when the current motion state is a closed operation state, point cloud data and image data around a door body are obtained; preprocessing the point cloud data to obtain first point cloud data; performing fusion processing on the first point cloud data and the image data to obtain door body three-dimensional data; and identifying and analyzing the three-dimensional data of the door body, detecting whether foreign matters exist in the door crack, if so, identifying and positioning the foreign matters, sending out a foreign matter alarm signal, and triggering the stop or reverse opening control of the vehicle door, so that the precision, adaptability and accuracy of safety monitoring are improved, and false alarms are reduced.

Description

Technical Field

[0001] The present application relates to the field of rail transit technology, and in particular to a safety monitoring method and system for subway platform shielding doors. Background Art

[0002] Subway platform screen doors are protective facilities installed between subway stations and trains to ensure passenger safety. Platform screen doors play an important role in preventing passengers from misbehaving, avoiding accidents, and protecting passenger safety. Due to their importance, the operating status of platform screen doors needs to be monitored at all times, especially when there may be failures or safety hazards. Effective monitoring methods are needed to promptly detect and deal with problems. Existing monitoring methods mostly rely on manual inspections or simple sensor alarms, which have problems such as low efficiency, poor real-time performance, and high false alarm rates. Therefore, it is of great significance to develop an efficient, real-time, and reliable subway platform screen door safety monitoring method.

[0003] Similar prior art includes a Chinese patent with announcement number CN108520243B, which discloses a shielded door foreign body detection alarm system, pre-stores a reference shielded door image, obtains multiple frames of on-site shielded door images that are continuous in time through a door-to-door data capture device, analyzes each frame, obtains C channel data, M channel data, and Y channel data of each pixel point, and converts the image into a C color matrix, an M color matrix, and a Y color matrix, and detects whether there is a deviation value of a certain position from the average value of the surrounding positions in each color matrix that is greater than a limit, obtains and outputs multiple images to be analyzed based on the analysis results; matches each image to be analyzed with the reference shielded door image, and if the matching degree is less than or equal to a preset percentage threshold, the image to be analyzed is determined to be a foreign body image. This method detects shielded door foreign bodies based on image matching and color analysis, and its detection accuracy is low and its environmental adaptability is weak. There is also a Chinese patent application with publication number CN109447090A, which discloses a method and system for detecting obstacles of shielded doors, which captures an image of the area between the shielded door and the vehicle door; compares the captured image with the scene change calibration image, and determines whether there is a significant change in the scene. If not, the captured image is sent to the obstacle recognition model for obstacle recognition, and the obstacle recognition model is used to calculate and determine whether there is an obstacle; if yes, the image of the detection area in the captured image is compared with the image of the corresponding detection area in the training sample image to determine whether there is an obstacle. This method performs obstacle detection based on image comparison and model recognition, and its detection accuracy is low and its environmental adaptability is weak.

[0004] Therefore, it is an urgent problem to provide a safety monitoring method and system for subway platform screen doors to improve the precision, adaptability and accuracy of safety monitoring. Summary of the invention

[0005] The present application provides a safety monitoring method and system for subway platform screen doors, which adopts multimodal data fusion technology, can comprehensively and accurately monitor the platform screen door area, reduce false alarms caused by environmental changes, and improve the precision and accuracy of detecting objects and foreign objects in door gaps. It has strong adaptability and can ensure the safe operation of the subway in different complex environments.

[0006] In a first aspect, the present application provides a safety monitoring method for a subway platform screen door, the method comprising: Step 1: Detect the current motion state of the shielding door. When the current motion state is a closed operation state, obtain point cloud data and image data around the door body; Step 2: Preprocess the point cloud data using the 3D point cloud self-learning background algorithm to obtain the first point cloud data; Step 3: Fusing the first point cloud data and the image data to obtain three-dimensional data of the door body; Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is any foreign object in the door gap. If so, identify and locate the foreign object, send out a foreign object alarm signal, and trigger the door to stop or reverse opening control.

[0007] In combination with the first aspect, in a first implementation of the first aspect of the present application, step 3 includes: Step 31: define the target object in the first point cloud data as the first object, identify all the first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate the first geometric feature array corresponding to each first key point respectively; Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array; Step 33, defining the target object in the image data as a second object, identifying all second objects in the image data, extracting any second object, and extracting N2 second key points of any second object, and respectively calculating a second visual feature array corresponding to each second key point; Step 34, extracting a first visual feature array corresponding to any first key point of any first object, calculating the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, and when the Euclidean distance is less than a first preset value, determining that any second key point is a corresponding point of any first key point, extracting other first key points of any first object, and other second key points of the reference second object, and determining whether other first key points have one-to-one corresponding points in other second key points, and if so, establishing a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points, and if not, repeating step 34 until a second object having an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array; Step 35: After traversing all the first objects, based on the first association relationship and the second association relationship, the point cloud data of any first object is mapped to the image data of the corresponding second object to generate the three-dimensional data of the door body.

[0008] In combination with the first aspect, in a second implementation of the first aspect of the present application, the first geometric feature array includes curvature, normal vector, point cloud density and geometric shape, the visual feature array includes color features, texture features, edge features and shape features, and the first geometric feature array and the second visual feature array are generated by: For any first key point, taking any first key point as the origin, obtaining point cloud data within a first preset range, defining it as reference point cloud data, and calculating a first geometric feature array based on the reference point cloud data; For any second key point, take any second key point as the origin, obtain image data within a second preset range, define it as reference image data, and calculate the second visual feature array based on the reference image data.

[0009] In combination with the first aspect, in a third implementation of the first aspect of the present application, a method for generating the first preset model is: A simulation model is constructed based on the operating environment of the subway platform screen door, and second point cloud data and first image data of the simulation model are collected at a preset collection point; The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated; Taking the second geometric feature array of any third key point and the third visual feature array of the fourth key point corresponding to any third key point as a pair of training data, constructing a training data set; Construct a neural network model, use the training data set to train the neural network model, and generate a first preset model.

[0010] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, step 4 includes: Step 41, extract any first object, calculate the average value of the distance from the object point cloud data to the point cloud acquisition device, if the average value is less than or equal to the second preset value, define any first object as a specific target object, and mark the pixel points corresponding to the object point cloud data as first pixel points, otherwise, define it as an object to be filtered, and mark the pixel points corresponding to the object point cloud data as second pixel points, wherein the object point cloud data is the point cloud data corresponding to any first object; Step 42: extract any pixel point adjacent to the reference pixel point from the image data, define it as a third pixel point, calculate the correlation between the reference pixel point and the third pixel point based on the pixel value and / or pixel coordinate, and when the correlation degree is greater than or equal to a third preset value, mark the third pixel point as the reference pixel point, wherein the reference pixel point is the first pixel point or the second pixel point; Step 43: extract unlabeled pixel points and define them as fourth pixel points; define the minimum distance between the fourth pixel point and the adjacent first pixel point as the first distance; and define the minimum distance between the fourth pixel point and the adjacent second pixel point as the second distance; if the first distance is greater than the second distance, label the fourth pixel point as the first pixel point; otherwise, label it as the second pixel point; Step 44: set the pixel value of the first pixel point to 255, set the pixel value of the second pixel point to 0, obtain third image data, extract the edge line of the specific target object based on the third image data, and obtain an edge feature vector of the edge line; Step 45: Input the edge feature vector of any specific target object into the second preset model to obtain the category of any specific target object.

[0011] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, the edge line is encoded to obtain an edge feature vector, and the edge feature vector includes Fourier transform, Zernike moment, contour histogram and / or shape context.

[0012] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, a first monitoring device and a second monitoring device are respectively arranged in the horizontal direction and the vertical direction of the shielding door, monitoring data collected by the first monitoring device and the second monitoring device are respectively obtained, and whether there is a foreign object in the door gap is detected based on the monitoring data; The analyzing device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, any one of the monitoring devices is determined to be abnormal.

[0013] In combination with the first aspect, in a seventh implementation of the first aspect of the present application, when the number of times any monitoring device is judged to be abnormal exceeds a fourth preset value, any monitoring device is judged to be faulty and a device fault alarm message is sent.

[0014] In a second aspect, the present application provides a safety monitoring system for subway platform screen doors, the system comprising: a state detection module, a data acquisition module, a preprocessing module, a data fusion module and a foreign body detection module; A state detection module is used to detect the current motion state of the shielding door; A data acquisition module, used to obtain point cloud data and image data around the door body when the current motion state is a closed operation state; A preprocessing module, used to preprocess the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data; A data fusion module is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body; The foreign object detection module is used to identify and analyze the three-dimensional data of the door body, detect whether there is a foreign object in the door gap, and if so, identify and locate the foreign object, send out a foreign object alarm signal, and trigger the door to stop or reverse opening control.

[0015] Compared with the prior art, the beneficial effects of the technical solution of the present application are at least as follows: 1. Through the fusion processing of 3D point cloud data and image data, it can accurately detect foreign objects in the door gap under complex backgrounds, avoiding safety accidents caused by foreign objects being clamped, with high detection accuracy and strong adaptability.

[0016] 2. Use 3D point cloud self-learning background algorithm to pre-process point cloud data, automatically distinguish point cloud changes caused by vehicle body shaking from real foreign object signals, and automatically adapt to different environmental changes, reduce the need for manual intervention, and improve the accuracy, intelligence and automation level of detection.

[0017] 3. Feature extraction and classification recognition are performed through preset models and neural network models, which improves the speed and efficiency of data processing, enabling the system to complete the monitoring and analysis of the environment around the door in a short time, ensuring real-time performance.

[0018] 4. By setting up multiple monitoring devices in different directions of the shielding door and performing data cross-verification, the reliability of the system is improved. Even if a device fails, other devices can still work normally, ensuring the continuity of monitoring and quickly identifying monitoring equipment failures.

[0019] 5. When a foreign object is detected in the door gap, an alarm signal is issued and the door is triggered to stop or reverse open, effectively preventing potential safety accidents. It not only ensures the safety of passengers, but also improves their travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0021] Figure 1 A schematic diagram of an embodiment of a safety monitoring method for a subway platform screen door in an embodiment of the present application; Figure 2 A schematic diagram of an embodiment of edge line extraction of a specific target object in an embodiment of the present application; Figure 3 This is a schematic diagram of an embodiment of setting a monitoring device on a shielding door in an embodiment of the present application; Figure 4 This is a schematic diagram of an embodiment of a safety monitoring system for subway platform screen doors in an embodiment of the present application; Among them, SD is a shielding door, ME1 is a first monitoring device, and ME2 is a second monitoring device. DETAILED DESCRIPTION

[0022] The embodiment of the present application provides a safety monitoring method and system for subway screen doors. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the safety monitoring method for subway platform screen doors in the embodiment of the present application includes: Step 1: Detect the current motion state of the shielding door. When the current motion state is a closed operation state, obtain point cloud data and image data around the door body.

[0024] Specifically, laser sensors are distributed around the shielding door at a certain angle and distance to cover all possible gap areas. They scan the surrounding environment of the door at a high frequency and obtain three-dimensional point cloud data of the surrounding environment. After obtaining the three-dimensional point cloud data of each laser sensor, the above three-dimensional point cloud data are fused and spliced ​​to obtain the point cloud data of the entire shielding door monitoring area; 2D cameras are installed on both sides or above the shielding door to obtain image data of door gaps and objects around the door in real time. After obtaining the image data of each camera, the above image data are spliced ​​to obtain image data of the entire shielding door monitoring area.

[0025] The current motion state includes static state, open operation state and closed operation state. The closed operation state is a necessary condition to ensure that safety monitoring is activated at a critical moment, because only during the closing process will foreign objects that may exist in the door gap pose a threat to passenger safety. When the shield door is in the closed operation state, the system automatically starts the data acquisition device to obtain point cloud data and image data around the door body, ensuring the timeliness and pertinence of data acquisition, avoiding the collection of invalid data, and improving the operation efficiency of the system.

[0026] Step 2: Use the 3D point cloud self-learning background algorithm to pre-process the point cloud data to obtain the first point cloud data.

[0027] Specifically, through the self-learning mechanism, the algorithm can adapt to dynamic changes in the environment, such as vehicle body swaying, effectively remove point cloud changes caused by vehicle body swaying or other dynamic factors, and extract normal point clouds when the object is stationary. These point clouds reflect the static characteristics of the environment around the shield door, providing a basis for subsequent foreign object detection and safety warnings.

[0028] Step 3: Fuse the first point cloud data and the image data to obtain three-dimensional data of the door body.

[0029] Specifically, point cloud data provides spatial geometric information of objects (such as position, shape, size, distance, etc.), but it is sparsely distributed at long distances and in the vertical direction, which may lead to incomplete target contours, and lacks semantic features such as color and texture, making it difficult to accurately identify obstacles; image data provides visual information of objects (color, texture, edges, etc.), but it lacks depth information, making it difficult to accurately judge the three-dimensional position and distance of obstacles, and is easily affected by factors such as lighting, shadows, and reflections. By combining the information of the two data sources, making up for the shortcomings of a single data type, and making full use of their respective advantages, a more comprehensive and accurate description of objects can be achieved, which can improve the system's ability to adapt to environmental changes, more accurately identify and locate foreign objects in door gaps, reduce the possibility of false alarms and missed alarms, and enhance robustness.

[0030] Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is any foreign object in the door gap. If so, identify and locate the foreign object, send out a foreign object alarm signal, and trigger the door to stop or reverse opening control.

[0031] Specifically, based on the fusion of 2D and 3D data, it detects whether there is a foreign object (human body and object) trapped in the shield door during closing. If a foreign object is trapped, the system sends an alarm signal through the alarm indicator light or communication interface, and transmits the signal to the shield door control system, so as to start the door to stop or open in reverse in time to avoid accidents and improve safety.

[0032] In a specific embodiment, the process of executing step 3 may specifically include the following steps: Step 31: define the target object in the first point cloud data as a first object, identify all first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate the first geometric feature array corresponding to each first key point respectively.

[0033] Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array.

[0034] Step 33: define the target object in the image data as a second object, identify all second objects in the image data, extract any second object, and extract N2 second key points of any second object, and calculate the second visual feature array corresponding to each second key point.

[0035] Step 34, extract the first visual feature array corresponding to any first key point of any first object, calculate the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, when the Euclidean distance is less than the first preset value, determine that any second key point is the corresponding point of any first key point, extract other first key points of any first object, and other second key points of the reference second object, and determine whether other first key points have one-to-one corresponding points in other second key points. If so, establish a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points. If not, repeat step 34 until a second object with an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array.

[0036] Step 35: After traversing all the first objects, based on the first association relationship and the second association relationship, the point cloud data of any first object is mapped to the image data of the corresponding second object to generate the three-dimensional data of the door body.

[0037] Specifically, the first point cloud data and image data come from different data acquisition devices, and there may be problems such as inconsistent coordinate systems and resolution differences, which require data alignment and fusion. The target object is the object contained in the first point cloud data / image data, including the shielding door body, obstacles between door gaps, and objects in its surrounding environment. Exemplarily, the first point cloud data is divided into different clusters by a clustering algorithm (such as DBSCAN, K-Means or Euclidean clustering), each cluster represents a target object; the contours of each target object in the image data are extracted by an edge detection algorithm, a contour extraction algorithm or a semantic segmentation model, so as to identify all second objects.

[0038] Key points are feature points with significant geometric changes related to the three-dimensional shape of the target object, such as corner points, edge points, or points in areas with high curvature. The key points in the first object are defined as first key points, and the key points in the second object are defined as second key points. The geometric characteristic array corresponding to the first key point is defined as a first geometric feature array, which is an array representing the geometric or topological features of the three-dimensional point cloud data of the target object at the first key point; the visual feature array corresponding to the second key point is defined as a second visual feature array, which is an array representing the features of the two-dimensional image data of the target object at the second key point. The first geometric feature array is a value representing the features of the three-dimensional shape, and the second visual feature array is a value representing the features of the two-dimensional shape. They cannot be directly compared. The first preset model is used to convert the first geometric feature array into a first visual feature array that can be compared with the two-dimensional feature array.

[0039] The first preset value is a preset value used to determine whether the first key point and the second key point have a corresponding relationship, and is set according to the experience of those skilled in the art or according to actual application scenarios, and is not limited in the embodiments of the present application. For example, the first preset value is 0.1 or 0.3.

[0040] Exemplarily, the first preset value is set to 0.2, the first object A has three first key points (a1, a2, a3), the first visual feature array corresponding to a1 is [0.8, 0.6, 0.7], the first visual feature array corresponding to a2 is [0.5, 0.9, 0.3], and the first visual feature array corresponding to a3 is [0.4, 0.7, 0.5]. The second object B has three second key points (b1, b2, b3), the second visual feature array corresponding to b1 is [0.7, 0.5, 0.6], the second visual feature array corresponding to b2 is [0.4, 0.8, 0.2], and the second visual feature array corresponding to b3 is [0.3, 0.6, 0.4]. First, it is determined that a1 and b1 are corresponding points to each other, and then a2 and a3, b2 and b3 are extracted for judgment. The Euclidean distance d between a2 and b2 can be obtained by calculation. 22 is 0.173, and the Euclidean distance d between a2 and b3 23 is 0.374, d 22 is less than the first preset value, a2 and b2 are corresponding points, and the Euclidean distance d between a3 and b3 33 is 0.173, d 33 is less than the first preset value, a3 and b3 are corresponding points, then the first object A and the second object B are associated, a1 and b1, a2 and b2, a3 and b3 are associated.

[0041] According to the above technical solution, even if there are deviations in the installation positions of the point cloud acquisition equipment and the image acquisition equipment, the matching relationship between each target object and each key point can be quickly identified, and the three-dimensional data of the door body can be automatically generated.

[0042] In a specific embodiment, the first geometric feature array includes curvature, normal vector, point cloud density and geometric shape, the visual feature array includes color features, texture features, edge features and shape features, and the first geometric feature array and the second visual feature array are generated by: (1) For any first key point, take any first key point as the origin, obtain point cloud data within a first preset range, define it as reference point cloud data, and calculate a first geometric feature array based on the reference point cloud data.

[0043] (2) For any second key point, take any second key point as the origin, obtain image data within a second preset range, define it as reference image data, and calculate the second visual feature array based on the reference image data.

[0044] Specifically, the first geometric feature array is an array of quantized values ​​used to describe the geometric or topological features of the local area around the first key point. It is a multidimensional vector that contains detailed information about the point cloud data around the first key point; the visual feature array is an array of feature parameters used to describe the local area around the key point. It is a multidimensional vector that contains visual information about the image data around the key point.

[0045] The first preset range is the range of local point cloud data that needs to be considered when calculating the first geometric feature array, and the second preset range is the range of local image data that needs to be considered when calculating the second visual feature array. Both are set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiments of the present application do not limit this. Exemplarily, the first preset range is a sphere with the first key point as the center and a radius of 0.1 meters, which is specifically selected based on the density of the point cloud data and the size of the target object; the second preset range is a circle with the second key point as the center and a radius of 10 pixels, which is specifically selected based on the resolution of the image and the size of the target object. The first preset range can also be a cube, a cuboid, etc., and the second preset range can also be a square, a rectangle, etc.

[0046] In a specific embodiment, the method for generating the first preset model is: (1) A simulation model is constructed based on the operating environment of the subway platform screen door, and second point cloud data and first image data of the simulation model are collected at preset collection points.

[0047] (2) The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated.

[0048] (3) The second geometric feature array of any third key point and the third visual feature array of the fourth key point corresponding to any third key point are used as a pair of training data to construct a training data set.

[0049] (4) Construct a neural network model, use the training data set to train the neural network model, and generate a first preset model.

[0050] Specifically, to generate a training data set, a 3D simulation model of the subway platform screen door and its surrounding environment is constructed using 3D modeling software (such as Blender, 3ds Max, etc.). The simulation model should reflect the actual situation as realistically as possible, including the geometry, material, lighting, etc. of the target object. The simulation model is used to generate point cloud data and image data similar to the real environment for model training.

[0051] The preset collection points are pre-set data collection points for collecting point cloud data and image data. Their positions can be determined according to the simulated operating environment and monitoring requirements of the subway platform screen door to ensure that the key areas of the platform screen door, such as the door gap and the edge of the door body, can be covered. Multiple preset collection points can be set in the simulation model to simulate the installation position and angle of the data collection equipment in actual applications. Since point cloud collection equipment and image collection equipment can be set at the same data collection point in the simulation space, point cloud data and image data scanned and photographed in a common direction from a common data collection point can be obtained. Based on this, it is possible to ensure the generation of high-quality training data, improve the generalization ability of the model and the reliability of the monitoring system.

[0052] The geometric features in the point cloud data and the visual features in the image data are paired to form a training data set. A neural network model (such as a fully connected neural network, a convolutional neural network, a deep neural network, etc.) is used to learn the training data set to generate a preset model that can map the geometric feature array to the visual feature array.

[0053] Since the point cloud acquisition device and the image acquisition device are set at the same position, there is no deviation between the point cloud data and image data of the same target object. A large amount of training data can be obtained in the simulation model. The training data is generated by the simulation model, which reduces the need for real data collection and reduces costs. It also avoids the tediousness and errors of manual data annotation, improves the training efficiency and accuracy of the preset model, enhances the robustness and generalization ability of the system, and provides reliable technical support for subway platform door safety monitoring.

[0054] In a specific embodiment, the process of executing step 4 may specifically include the following steps: Step 41, extract any first object, calculate the average value of the distance from the object point cloud data to the point cloud acquisition device, if the average value is less than or equal to the second preset value, define any first object as a specific target object, and mark the pixel points corresponding to the object point cloud data as first pixel points; otherwise, define it as an object to be filtered, and mark the pixel points corresponding to the object point cloud data as second pixel points, wherein the object point cloud data is the point cloud data corresponding to any first object.

[0055] Step 42, extract any pixel point adjacent to the reference pixel point from the image data, define it as the third pixel point, calculate the correlation between the reference pixel point and the third pixel point based on the pixel value and / or pixel coordinate, and when the correlation degree is greater than or equal to a third preset value, mark the third pixel point as the reference pixel point, wherein the reference pixel point is the first pixel point or the second pixel point.

[0056] Step 43, extract the unlabeled pixel points, define them as the fourth pixel points, define the minimum distance between the fourth pixel point and the adjacent first pixel point as the first distance, and define the minimum distance between the fourth pixel point and the adjacent second pixel point as the second distance. If the first distance is greater than the second distance, the fourth pixel point is labeled as the first pixel point, otherwise, it is labeled as the second pixel point.

[0057] Step 44, setting the pixel value of the first pixel point to 255, setting the pixel value of the second pixel point to 0, acquiring third image data, extracting the edge line of the specific target object based on the third image data, and acquiring the edge feature vector of the edge line.

[0058] Step 45: Input the edge feature vector of any specific target object into the second preset model to obtain the category of any specific target object.

[0059] Specifically, the specific target object is the object to be detected and identified, that is, the obstacle located in the gap of the shielding door, and the object to be filtered is the object that does not need to be detected and identified, such as the background object, the door body, etc. Since the distance between the fixed object such as the door body and the point cloud acquisition device is certain, the door body, the background, etc. and the obstacles in the door gap can be classified by distance judgment, and the objects that need to be focused on (such as obstacles) and the objects that can be ignored can be distinguished. The second preset value is a preset value for judging whether the first object is a specific target object, which is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiment of the present application does not limit this.

[0060] When mapping point cloud data to image data, the pixels on the image data corresponding to the point cloud data (i.e., reference pixels) can be annotated to determine whether these pixels belong to a specific target object or an object to be filtered. Then, based on the correlation between other pixels and the reference pixels, other pixels can be classified as first pixels or second pixels. For example, there are three connected pixels C1, C2, and C3, where C1 is the first pixel, C3 is the second pixel, and C2 has not been annotated. Since C2 is adjacent to C1 and C3, the correlation R between C2 and C1 is calculated. 21 , the correlation between C2 and C3 23 , calculate the correlation R 21 When , the Euclidean distance between C2 and C1 is calculated based on the pixel value, and the reciprocal of the Euclidean distance is used as the color similarity. Then, the spatial distance between C2 and C1 is calculated based on the pixel coordinates, and the reciprocal of the spatial distance is used as the distance similarity. Finally, the color similarity and distance similarity are weighted averaged to obtain the correlation R. 21 , calculate the correlation R in the same way 23 , compare the correlation R 21 and the correlation R 23If the correlation R 21 Greater than the correlation R 23 , then the correlation R 21 As a reference correlation, when the reference correlation is greater than the third preset value, the pixel C2 is marked as the first pixel. When there is only one reference pixel adjacent to any unmarked pixel, only one correlation calculation and comparison is required. The third preset value is a preset value used to determine whether the third pixel is to be marked as a reference pixel. It is set according to the experience of technicians in this field or according to the actual application scenario. The embodiment of the present application does not limit this. Some pixels may not be successfully marked due to insufficient color information or sparse point cloud. By considering the spatial distance, these pixels are further marked, and the pixels to be identified are classified into specific target objects or objects to be filtered. The regional division of foreign body detection is further improved, which can effectively fill the gaps in regional division and improve the integrity of detection.

[0061] By setting the pixel value of the first pixel point to 255 and the pixel value of the second pixel point to 0, a specific target object (such as an obstacle) is highlighted, making it easier to extract the contour line of the specific target object, such as Figure 2 As shown, high contour extraction accuracy can be maintained even in complex scenes.

[0062] The second preset model is a pre-trained neural network model that analyzes the edge feature vector of the target object to obtain the category of the target object (such as a passenger's hand, plastic bag, bag or luggage, etc.), and realizes automatic classification and recognition of specific target objects by learning the edge features of different categories of foreign objects.

[0063] In a specific embodiment, the edge line is encoded to obtain an edge feature vector, where the edge feature vector includes Fourier transform, Zernike moment, contour histogram and / or shape context.

[0064] Specifically, the edge feature vector is a mathematical description of the edge of an object, wherein the Fourier transform is to perform a Fourier transform on the edge line to extract its frequency domain features; the Zernike moment is to calculate the Zernike moment of the edge line to extract its shape features; the contour histogram is to generate the contour histogram of the edge line to extract its contour distribution features; the shape context is to characterize the shape by calculating the relative position relationship between the edge point and other points, and extract the local detail information it provides. By combining multiple features, the edge feature vector can provide a more comprehensive and accurate edge description, while improving the ability to distinguish different types of foreign objects, improving the adaptability of the edge feature vector to environmental changes, and reducing the possibility of false alarms and missed reports. Exemplarily, the edge feature vector can also include geometric features (area, perimeter, aspect ratio, convex hull, etc.), shape matching features (such as polygon approximation), etc.

[0065] Preferably, the edge feature vector can be a one-dimensional vector of any one of Fourier transform, Zernike moment, contour histogram or shape context. Although some information will be lost by simplifying the edge feature vector to a one-dimensional vector of a single feature, it has obvious advantages in terms of computational efficiency, model complexity, implementation difficulty and applicability to specific scenarios.

[0066] In a specific embodiment, a first monitoring device and a second monitoring device are respectively arranged in the horizontal direction and the vertical direction of the shielding door, monitoring data collected by the first monitoring device and the second monitoring device are respectively obtained, and whether there is foreign matter in the door gap is detected based on the monitoring data.

[0067] The analyzing device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, any one of the monitoring devices is determined to be abnormal.

[0068] like Figure 3 As shown, a first monitoring device ME1 is set in the horizontal direction of the shielding door SD, which is used to monitor the door gap area in the horizontal direction of the shielding door to detect whether there are foreign objects. The first monitoring device is an ultrasonic sensor, a laser beam detector, a photoelectric sensor or an infrared detector, etc. Based on the working principles of various types of monitoring devices, transmitters and receivers can be installed on the left and right sides of the shielding door; a second monitoring device ME2 is set in the vertical direction of the shielding door SD, which is used to monitor the door gap area in the vertical direction of the shielding door to detect whether there are foreign objects. The second monitoring device is a laser ranging detector, an ultrasonic sensor, a lidar, an infrared array sensor, etc. Based on the working principles of various types of monitoring devices, transmitters and receivers can be installed on the upper and lower sides of the shielding door.

[0069] Through collaborative monitoring of multiple devices, not only the accuracy of foreign object detection is improved, but also when a monitoring device fails, other devices can continue to work, avoiding complete failure of the monitoring system. The monitoring results of different devices can be verified with each other. While improving the accuracy of foreign object detection, monitoring equipment failures can be quickly and accurately identified, and equipment failures can be discovered and handled in a timely manner, providing more reliable safety guarantees for subway operations.

[0070] In a specific embodiment, when the number of times any monitoring device is judged to be abnormal exceeds a fourth preset value, any monitoring device is judged to be faulty, and device fault alarm information is sent.

[0071] The fourth preset value is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiment of the present application does not limit this.

[0072] By counting the number of times a device is judged to be abnormal, device failures can be identified more accurately. For example, if a device occasionally becomes abnormal, it may be due to instantaneous interference or environmental factors. If the number of abnormalities exceeds the fourth preset value, it is more likely that there is a problem with the device itself. Based on this, false alarms can be reduced and the accuracy of fault detection can be improved.

[0073] The above describes the safety monitoring method for subway platform screen doors in the embodiment of the present application. The following describes the safety monitoring system for subway platform screen doors in the embodiment of the present application. Figure 4 In an embodiment of the present application, a safety monitoring system for subway platform screen doors includes: a state detection module 10, a data acquisition module 20, a preprocessing module 30, a data fusion module 40 and a foreign object detection module 50.

[0074] The state detection module 10 is used to detect the current motion state of the shielding door.

[0075] The data acquisition module 20 is used to obtain point cloud data and image data around the door body when the current motion state is the closed operation state.

[0076] The preprocessing module 30 is used to preprocess the point cloud data using the 3D point cloud self-learning background algorithm to obtain first point cloud data.

[0077] The data fusion module 40 is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body.

[0078] The foreign object detection module 50 is used to identify and analyze the three-dimensional data of the door body, detect whether there is a foreign object in the door gap, and if so, identify and locate the foreign object, send out a foreign object alarm signal, and trigger the door to stop or reverse opening control.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0081] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A safety monitoring method for subway platform screen doors, characterized in that: The method comprises: Step 1: Detect the current motion state of the shielding door, and when the current motion state is a closed operation state, obtain point cloud data and image data around the door body; Step 2: pre-process the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data; Step 3: fusing the first point cloud data and the image data to obtain three-dimensional data of the door body; Step 4: Identify and analyze the three-dimensional data of the door body to detect whether there is a foreign object in the door gap. If so, identify and locate the foreign object, send a foreign object alarm signal, and trigger the door to stop or reverse open control.

2. The safety monitoring method for subway platform screen doors according to claim 1, characterized in that: The step 3 comprises: Step 31: define the target object in the first point cloud data as a first object, identify all first objects in the first point cloud data, extract any first object, and extract N1 first key points of any first object, and calculate the first geometric feature array corresponding to each first key point respectively; Step 32: extract any first geometric feature array, input any first geometric feature array into a first preset model, and obtain a first visual feature array corresponding to any first geometric feature array; Step 33, defining the target object in the image data as a second object, identifying all second objects in the image data, extracting any second object, and extracting N2 second key points of any second object, and respectively calculating a second visual feature array corresponding to each second key point; Step 34, extracting the first visual feature array corresponding to any first key point of any first object, calculating the Euclidean distance between the first visual feature array and the second visual feature array of any second key point, and when the Euclidean distance is less than a first preset value, determining that any second key point is a corresponding point of any first key point, extracting other first key points of any first object, and other second key points of the reference second object, and determining whether other first key points have one-to-one corresponding points in other second key points, and if so, establishing a first association relationship between any first object and the reference second object, and a second association relationship between each group of corresponding points, and if not, repeating step 34 until the second object having an association relationship with any first object is found, wherein the reference second object is the second object corresponding to any second visual feature array; Step 35: After traversing all the first objects, based on the first association relationship and the second association relationship, map the point cloud data of any first object to the corresponding image data of the second object to generate the three-dimensional data of the door body.

3. The safety monitoring method for subway platform screen doors according to claim 2, characterized in that: The first geometric feature array includes curvature, normal vector, point cloud density and geometric shape, the visual feature array includes color feature, texture feature, edge feature and shape feature, and the first geometric feature array and the second visual feature array are generated by: For any of the first key points, taking any of the first key points as the origin, obtaining point cloud data within a first preset range, defining it as reference point cloud data, and calculating the first geometric feature array based on the reference point cloud data; For any of the second key points, take any of the second key points as the origin, obtain image data within a second preset range, define it as reference image data, and calculate the second visual feature array based on the reference image data.

4. The safety monitoring method for subway platform screen doors according to claim 2, characterized in that: The method for generating the first preset model is: Building a simulation model based on the operating environment of the subway platform screen door, and collecting second point cloud data and first image data of the simulation model at a preset collection point; The target object in the second point cloud data is defined as a third object, and according to the method of step 31, a second geometric feature array of any third key point of any third object in the second point cloud data is calculated; the target object in the first image data is defined as a fourth object, and according to the method of step 33, a third visual feature array of any fourth key point of any fourth object in the first image data is calculated; Taking the second geometric feature array of any of the third key points and the third visual feature array of the fourth key point corresponding to any of the third key points as a pair of training data, constructing a training data set; Construct a neural network model, use the training data set to train the neural network model, and generate the first preset model.

5. The safety monitoring method for subway platform screen doors according to claim 2, characterized in that: The step 4 comprises: Step 41, extract any of the first objects, calculate the average value of the distance from the object point cloud data to the point cloud acquisition device, and if the average value is less than or equal to a second preset value, define any of the first objects as a specific target object, and mark the pixel points corresponding to the object point cloud data as first pixel points; otherwise, define it as an object to be filtered, and mark the pixel points corresponding to the object point cloud data as second pixel points, wherein the object point cloud data is the point cloud data corresponding to any of the first objects; Step 42: extract any pixel point adjacent to the reference pixel point from the image data, define it as a third pixel point, calculate the correlation between the reference pixel point and the third pixel point based on the pixel value and / or pixel coordinate, and when the correlation degree is greater than or equal to a third preset value, mark the third pixel point as the reference pixel point, wherein the reference pixel point is the first pixel point or the second pixel point; Step 43: extract unlabeled pixel points and define them as fourth pixel points; define the minimum distance between the fourth pixel point and the adjacent first pixel point as the first distance; and define the minimum distance between the fourth pixel point and the adjacent second pixel point as the second distance; if the first distance is greater than the second distance, label the fourth pixel point as the first pixel point; otherwise, label it as the second pixel point; Step 44: setting the pixel value of the first pixel point to 255, setting the pixel value of the second pixel point to 0, acquiring third image data, extracting the edge line of the specific target object based on the third image data, and acquiring an edge feature vector of the edge line; Step 45: Input the edge feature vector of any of the specific target objects into a second preset model to obtain the category of any of the specific target objects.

6. The safety monitoring method for subway platform screen doors according to claim 5, characterized in that: The edge line is encoded to obtain the edge feature vector, where the edge feature vector includes Fourier transform, Zernike moment, contour histogram and / or shape context.

7. The safety monitoring method for subway platform screen doors according to claim 1, characterized in that: The method further comprises: A first monitoring device and a second monitoring device are respectively arranged in the horizontal direction and the vertical direction of the shielding door, monitoring data collected by the first monitoring device and the second monitoring device are respectively acquired, and whether there is a foreign object in the door gap is detected based on the monitoring data; The analysis device of the door body three-dimensional data is defined as a three-dimensional monitoring device. When the monitoring result of any one of the three-dimensional monitoring device, the first monitoring device and the second monitoring device is different from the monitoring results of the other two monitoring devices, it is determined that any one of the monitoring devices is abnormal.

8. The safety monitoring method for subway platform screen doors according to claim 7, characterized in that: When the number of times that any of the monitoring devices is determined to be abnormal exceeds a fourth preset value, any of the monitoring devices is determined to be faulty, and device fault alarm information is sent.

9. A safety monitoring system for subway platform doors, characterized in that: The system includes a state detection module, a data acquisition module, a preprocessing module, a data fusion module and a foreign body detection module; The state detection module is used to detect the current motion state of the shielding door; The data acquisition module is used to obtain point cloud data and image data around the door body when the current motion state is a closed operation state; The preprocessing module is used to preprocess the point cloud data using a 3D point cloud self-learning background algorithm to obtain first point cloud data; The data fusion module is used to fuse the first point cloud data and the image data to obtain three-dimensional data of the door body; The foreign object detection module is used to identify and analyze the three-dimensional data of the door body, detect whether there is a foreign object in the door gap, and if so, identify and locate the foreign object, send out a foreign object alarm signal, and trigger the door to stop or reverse opening control.

Images