Subway platform screen door safety monitoring method and monitoring system based on multi-source data analysis

Abstract

The invention provides a metro platform screen door safety monitoring method and monitoring system based on multi-source data analysis, and the method comprises the steps: obtaining a multi-source data set of a target metro platform, carrying out the time-space alignment processing of the multi-source data set, generating a multi-modal monitoring data set fusing the time-space characteristics, and carrying out the monitoring of the metro platform screen door safety. And performing foreign matter identification processing based on the multi-modal monitoring data set, generating a foreign matter monitoring result and foreign matter position distribution information of the shielding door area, performing density prediction processing according to the foreign matter monitoring result and a passenger flow data sequence in the multi-source data set, generating a passenger linear density prediction result and density change trend information, and performing density prediction processing according to the passenger linear density prediction result and the density change trend information. And generating a safety early warning strategy based on the foreign matter position distribution information and the density change trend information, and transmitting the safety early warning strategy to a subway platform control system to trigger emergency response operation. According to the invention, the emergency processing efficiency and passenger guidance accuracy of the subway platform in a sudden foreign matter blocking scene are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a subway shield door safety monitoring method and system based on multi-source data analysis. BACKGROUND

[0002] In the field of subway operation safety, foreign matter intrusion monitoring and passenger density prediction are important technologies to ensure platform order. The current mainstream method usually only relies on visual camera to identify foreign matter in the shield door area, or uses ultrasonic ranging device to determine the existence of obstacles. Due to the limitation of sensor characteristics, such methods are easily disturbed by the environment, resulting in insufficient foreign matter positioning accuracy. In terms of passenger density prediction, existing technologies are mostly based on historical passenger flow statistical models or real-time video number estimation to generate density distribution, but they do not effectively integrate the real-time impact of foreign matter blockage on passenger flow path, resulting in significant deviation between prediction results and real scene. In addition, the existing early warning mechanism relies on fixed threshold to trigger emergency response, which is difficult to adapt to the density mutation risk caused by dynamic movement of foreign matter, resulting in insufficient timeliness and accuracy of the early warning scheme. SUMMARY

[0003] Therefore, the present application provides a subway shield door safety monitoring method and system based on multi-source data analysis. The technical scheme of the embodiment of the present application is as follows: On the one hand, the present application provides a subway shield door safety monitoring method based on multi-source data analysis, comprising: acquiring a multi-source data set of a target subway platform, the multi-source data set comprising a visual data sequence of the shield door area captured by an image acquisition device and an obstacle distance data sequence collected by an ultrasonic ranging device; performing spatio-temporal alignment processing on the multi-source data set to generate a multi-modal monitoring data set that integrates spatio-temporal features; performing foreign matter identification processing based on the multi-modal monitoring data set to generate foreign matter monitoring results and foreign matter position distribution information of the shield door area; performing density prediction processing based on the foreign matter monitoring results and passenger flow data sequence in the multi-source data set to generate passenger linear density prediction results and density change trend information; generating a safety warning strategy based on the foreign matter position distribution information and the density change trend information, and transmitting the safety warning strategy to the subway platform control system to trigger emergency response operation.

[0004] On the other hand, the present application provides a monitoring system comprising a memory and a processor, the memory storing a computer program executable on the processor, and the processor implementing the steps of the above method when executing the program.

[0005] The application provides a subway shield door safety monitoring method based on multi-source data analysis, which acquires a visual data sequence of a shield door area captured by an image acquisition device and an obstacle distance data sequence collected by an ultrasonic ranging device, constructs a multi-source data set, and generates multi-modal monitoring data with fused space-time characteristics by performing space-time alignment processing on the multi-source data set, thereby solving the problem of inaccurate foreign object positioning caused by time difference and spatial deviation of visual data and ranging data in traditional monitoring methods; based on the multi-modal monitoring data, a foreign object dynamic recognition result and position distribution information are extracted, dynamic density prediction is performed in combination with a passenger flow data sequence, an initial density prediction value is nonlinearly corrected by embedding a foreign object blocking influence factor, the limitation of traditional density prediction models that do not consider real-time interference of physical obstacles on passenger paths is broken, and the prediction result is more in line with the complex changes of passenger flow in an actual scene; further, a safety warning strategy including a time dimension fluctuation curve is generated according to density change trend information, a density peak node and a change direction identifier are dynamically labeled, real-time linkage response from foreign object monitoring to risk warning is realized, the emergency handling efficiency and passenger evacuation accuracy of a subway platform in a sudden foreign object blocking scene are effectively improved, and the risk of misjudgment of a single sensor data or a static threshold warning mechanism in a complex environment is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 An implementation flowchart of a subway shield door safety monitoring method based on multi-source data analysis provided by the embodiment of the application is shown in the figure.

[0007] Figure 2 A hardware entity diagram of a monitoring system provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0008] In order to make the purpose, technical scheme and advantages of the application clearer, the technical scheme of the application is further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the application, and all other embodiments obtained by a person skilled in the art without creative labor belong to the protection scope of the application.

[0009] The embodiment of the application provides a subway shield door safety monitoring method based on multi-source data analysis, which can be executed by a processor of a monitoring system.

[0010] Figure 1 An implementation flowchart of a subway shield door safety monitoring method based on multi-source data analysis provided by the embodiment of the application is shown in the figure. Figure 1As shown, the method comprises: step S100: acquiring a multi-source data set of a target subway platform, the multi-source data set comprising a shield door area visual data sequence captured by an image acquisition device and an obstacle distance data sequence collected by an ultrasonic ranging device.

[0011] The multi-source data set is a set composed of multiple different sources and different types of data. In the embodiment of the present application, it mainly includes a shield door area visual data sequence and an obstacle distance data sequence. The shield door area visual data sequence is a series of visual data about the shield door area captured by the image acquisition device. These data exist in the form of a sequence, reflecting the visual state of the shield door area at different times, such as the opening and closing state of the shield door, whether there are people or objects around, etc. The obstacle distance data sequence is a series of data about the distance between the shield door and the obstacle collected by the ultrasonic ranging device, also presented in the form of a sequence, embodying the change of the distance between the obstacle and the shield door over time.

[0012] Acquiring the multi-source data set of the target subway platform requires the comprehensive use of multiple data acquisition devices and technologies. For the shield door area visual data sequence, image acquisition devices such as high-definition cameras deployed in the shield door area can be used to periodically collect visual data from different angles. These cameras can be installed at different positions above and beside the shield door to obtain comprehensive visual information of the shield door area. The collected visual data from different angles will be time-synchronized and labeled to ensure consistency in the time dimension and facilitate subsequent processing. For example, in the shield door area of the subway platform, three high-definition cameras are installed, one above the shield door and two on both sides. They collect visual data every 1 second, and each collected data is labeled with an accurate time stamp.

[0013] For the obstacle distance data sequence, the ultrasonic ranging device is used to obtain dynamic distance data between the shield door and the obstacle in real time. The ultrasonic ranging device uses the reflection principle of ultrasonic waves to measure the distance. It emits ultrasonic signals, and when the signals are reflected back by the obstacle, the device calculates the distance based on the propagation time of the signals. In order to more accurately locate the position of the obstacle, the dynamic distance data will also be position-encoded based on the spatial coordinates of the ranging sensor. For example, an ultrasonic ranging device is installed on each leaf of the shield door, and the spatial coordinates of the device are known. By combining the collected distance data with the spatial coordinates of the device, the specific position information of the obstacle relative to the shield door can be obtained.

[0014] As an implementation, step S100 can specifically comprise steps S110-S140: step S110: periodically collecting visual data from different angles by multiple image acquisition devices deployed in the shield door area, and time-synchronizing and labeling the visual data from different angles to generate a shield door area visual data sequence.

[0015] The image acquisition device is a device for acquiring visual data, for example, a high-definition camera. The visual data of different perspectives is image or video data collected from multiple different positions and angles about the shield door area, and the data of multiple perspectives can more comprehensively reflect the actual situation of the shield door area. The time synchronization mark is to ensure the consistency of the data collected from different perspectives in time, by marking an accurate timestamp for each data frame, to facilitate subsequent analysis and processing of the data.

[0016] In specific implementation, multiple image acquisition devices are deployed reasonably in the shield door area, for example, high-definition cameras are installed at the top, both sides and other positions of the shield door. The acquisition period is set, for example, data is collected every 0.5 seconds. During the acquisition process, each image acquisition device independently acquires visual data of its own perspective, and adds an accurate timestamp to each frame of data.

[0017] Step S120: Real-time acquisition of dynamic distance data between the shield door and the obstacle by the ultrasonic ranging device, and position coding of the dynamic distance data based on the spatial coordinates of the ranging sensor to generate an obstacle distance data sequence.

[0018] The ultrasonic ranging device is a device that measures distance using the reflection principle of ultrasonic waves. It emits ultrasonic signals, and when the signals are reflected back after encountering an obstacle, the distance between the obstacle and the device is calculated based on the propagation time of the signals. Dynamic distance data is real-time distance data between the shield door and the obstacle at different times, which changes with the movement of the obstacle or the opening and closing of the shield door. The spatial coordinates of the ranging sensor are the specific position information of the ultrasonic ranging device in the shield door area, and through the known spatial coordinates, the specific position of the obstacle relative to the shield door can be determined. Position coding is to combine the dynamic distance data with the spatial coordinates of the ranging sensor to assign specific position information to each distance data, so as to more accurately locate the obstacle.

[0019] In actual operation, the ultrasonic ranging device is installed at a suitable position of the shield door, such as the middle position of each door leaf. The device will emit ultrasonic signals in real time and receive the reflected signals, and the dynamic distance data between the shield door and the obstacle is obtained by calculating the propagation time of the signals. At the same time, the spatial coordinates of each ranging sensor are recorded, for example, in a three-dimensional coordinate system, the coordinates of each sensor are (x, y, z). The collected dynamic distance data is associated with the corresponding sensor spatial coordinates, for example, the distance data "d" is combined with the sensor coordinates (x, y, z) to form a data pair (d, x, y, z), thus completing the position coding. Through continuous acquisition and coding, an obstacle distance data sequence reflecting the change of distance between the shield door and the obstacle with time and position is generated.

[0020] Step S130: Obtain passenger entry and exit record data and real-time video monitoring data from the subway platform control system, and extract flow trajectory from the entry and exit record data and video monitoring data to generate passenger flow data sequence.

[0021] The subway platform control system is responsible for managing and monitoring various facilities and operations of the subway platform, which includes passenger entry and exit record data and real-time video monitoring data. Passenger entry and exit record data is the relevant information recorded by the system when passengers pass through the gate to enter or exit the platform, such as entry and exit time, gate number, etc. These data can reflect the entry and exit behavior and time regularity of passengers. Real-time video monitoring data is the video data collected by the monitoring camera in the platform in real time, which can directly show the activities of passengers in the platform. Flow trajectory extraction is to analyze and extract the moving path and time information of passengers in the platform from the entry and exit record data and real-time video monitoring data.

[0022] When obtaining data from the subway platform control system, data interface can be connected with the system to obtain the required passenger entry and exit record data and real-time video monitoring data according to the set rules and protocols. For entry and exit record data, the entry and exit position and time of passengers can be preliminarily determined according to the time and gate number information in the record. For real-time video monitoring data, target detection and tracking algorithms such as YOLO (You Only Look Once) algorithm based on deep learning and KCF (Kernelized Correlation Filters) algorithm can be used to detect and track passengers in the video and extract their moving trajectories. For example, in the subway platform, passenger entry and exit record data and real-time video monitoring data within a day are obtained from the control system through data interface. YOLO algorithm is used to detect passengers in the video and identify the position of each passenger, and KCF algorithm is used to track passengers and record their moving path and time in the platform. Combined with the entry and exit record data, passenger flow data sequence reflecting the flow of passengers in the platform can be generated.

[0023] Step S140: Standardize the data format of the platform screen door area visual data sequence, obstacle distance data sequence and passenger flow data sequence to generate a multi-source data set containing unified time identifier and space identifier.

[0024] Data format standardization processing is to convert data of different sources and formats into a unified format to facilitate subsequent analysis and processing. Unified time identifier is to ensure the consistency of all data in time dimension, facilitating synchronous analysis of different types of data. Unified space identifier is to uniformly manage and locate data in spatial dimension, so that different types of data can be associated and compared in the same spatial coordinate system.

[0025] In the data format standardization process, first, a unified data format and specification need to be determined. For the shield door area visual data sequence, obstacle distance data sequence, and passenger flow data sequence, format conversion and adjustment are performed respectively. For example, the image or video data in the visual data sequence is uniformly converted to a set image format (such as JPEG) and resolution, and the data in the obstacle distance data sequence and passenger flow data sequence is arranged according to a unified table format, including time, location, and other key information. At the same time, a unified time identifier and space identifier are added to each data point. The time identifier can use a unified timestamp format, such as "YYYY-MM-DD HH:MM:SS.sss", and the space identifier can use a three-dimensional coordinate system of the station for positioning. Through these processes, the three data sequences are integrated into a multi-source data set containing unified time identifiers and space identifiers, facilitating subsequent spatio-temporal alignment and analysis processing.

[0026] Step S200: Spatio-temporal alignment processing is performed on the multi-source data set to generate a multi-modal monitoring data set that fuses spatio-temporal features.

[0027] Spatio-temporal alignment processing is to match and align different types of data in the multi-source data set in time and space dimensions, so that different data have consistency and correlation in time and space. Fusion of spatio-temporal features is to extract and fuse features of data after spatio-temporal alignment processing to obtain information that can comprehensively reflect the spatio-temporal features of the shield door area. The multi-modal monitoring data set is a data set containing multiple different types of data (such as visual data, ranging data, etc.) and their fused features, which is used for subsequent foreign object identification and safety monitoring.

[0028] Spatio-temporal alignment processing of the multi-source data set needs to consider the time and space characteristics of different data. For the time dimension, it is necessary to ensure that the data points in different data sequences correspond in time, for example, aligning the visual data sequence and the obstacle distance data sequence at the same time point. For the spatial dimension, the spatial coordinates of different data need to be unified and matched so that they have accurate correspondence in the same spatial coordinate system. Through spatio-temporal alignment processing, the spatio-temporal differences between data can be eliminated, providing a basis for subsequent feature fusion and analysis. In the fusion of spatio-temporal features, a variety of feature extraction and fusion algorithms can be used, such as principal component analysis (PCA), linear discriminant analysis (LDA), etc., to fuse the features of different types of data, obtaining a multi-modal monitoring data set that can comprehensively reflect the spatio-temporal features of the shield door area.

[0029] As an implementation, in step S200, the multi-source data set is processed for spatio-temporal alignment to generate a multi-modal monitoring data set with fused spatio-temporal features, which can specifically include the following steps S210-S250: in step S210, the shield door area visual data sequence is processed for blur compensation, blur parameters of each frame of image are extracted, and the motion trajectory between adjacent frames of image is compensated and corrected based on the blur parameters to generate an enhanced visual data sequence with continuous motion trajectory.

[0030] The blur compensation processing is a processing for the image blur problem that can exist in the shield door area visual data sequence. The image blur can be caused by camera shaking, too fast object motion, etc. The blur parameter is a parameter for describing the blur degree and characteristics of the image. By extracting these parameters, the blur condition of the image can be understood. The motion trajectory compensation and correction is to adjust and correct the motion trajectory of the object between adjacent frames of image according to the extracted blur parameters, so as to eliminate the problem of discontinuous motion trajectory caused by image blur. The enhanced visual data sequence is the visual data sequence obtained after the blur compensation processing and the motion trajectory correction, in which the motion trajectory of the object is more continuous and accurate.

[0031] In the blur compensation processing, first, an image blur detection algorithm, such as an algorithm based on spectral analysis, is used to analyze each frame of image in the shield door area visual data sequence to extract the blur parameters of the image. These parameters can include the direction and degree of blur. Then, the motion trajectory of the object between adjacent frames of image is compensated and corrected according to the blur parameters. For example, the optical flow method is used to calculate the motion vector of the object between adjacent frames of image, and the motion vector is adjusted in combination with the blur parameters to make the motion trajectory of the object smoother and more continuous. For example, in the shield door area visual data sequence of the subway station platform, some images are blurred due to camera shaking. The blur direction and degree of the image are detected by the spectral analysis algorithm, and then the motion vector of the object is calculated by the optical flow method, and the motion vector is corrected according to the blur parameters to finally generate an enhanced visual data sequence with continuous motion trajectory.

[0032] In step S220, the multi-dimensional interference source analysis is performed on the obstacle distance data sequence to decompose the environmental vibration interference component and the temperature drift interference component in the obstacle distance data sequence, and to generate a calibrated ranging data sequence after interference suppression according to the distribution characteristics of the decomposed interference components.

[0033] Multi-dimensional interference source analysis is to analyze and identify possible interference sources in the obstacle distance data sequence from multiple angles and dimensions. Environmental vibration interference components are interference components generated by the vibration of the surrounding environment, such as the vibration generated by the running of the subway train, the vibration caused by the movement of the platform personnel, etc. Temperature drift interference components are interference components generated by the change of the environmental temperature, which causes the change of the ultrasonic wave propagation speed in the air, thereby affecting the ranging result. Interference suppression is used to reduce or eliminate the influence of these interference components on the ranging data. The calibrated ranging data sequence is a more accurate obstacle distance data sequence obtained after interference suppression processing.

[0034] When performing multi-dimensional interference source analysis, signal processing and data analysis methods are used to analyze the obstacle distance data sequence in depth. For environmental vibration interference components, the vibration of the platform can be monitored, such as using a vibration sensor to collect vibration data, and then the vibration data is analyzed in association with the obstacle distance data sequence to identify the influence degree and regularity of vibration on the ranging result. For temperature drift interference components, a temperature sensor can be used to monitor the environmental temperature in real time, and according to the propagation speed formula of ultrasonic wave at different temperatures, the influence of temperature on the ranging result can be calculated. Then, according to the distribution characteristics of the decomposed interference components, filtering, compensation and other methods are used for interference suppression. For example, a Kalman filter is used to filter the obstacle distance data sequence to remove the influence of environmental vibration interference and temperature drift interference, and a calibrated ranging data sequence after interference suppression is generated.

[0035] Step S230: The timestamps of the enhanced visual data sequence and the collection timestamps of the calibrated ranging data sequence are bidirectionally synchronized and aligned to generate the spatio-temporal mapping relationship between the visual data and the ranging data in the multi-source data set.

[0036] Timestamp is an identifier used to mark the data collection time, and through the timestamp, the position of the data in the time dimension can be determined. Bidirectional synchronization alignment is not only to match the timestamps of the enhanced visual data sequence and the collection timestamps of the calibrated ranging data sequence, but also to consider the sequence and time interval of data collection to ensure accurate correspondence of the two in time. Spatio-temporal mapping relationship is the correspondence relationship between visual data and ranging data in time and space dimensions, through which different types of data can be associated and integrated in space and time.

[0037] In the bidirectional synchronization alignment, first, the timestamp format and precision of the enhanced visual data sequence and the calibration ranging data sequence need to be determined. Then, a time synchronization algorithm, such as a timestamp matching-based algorithm, is used to compare and adjust the timestamps of the two data sequences. For example, find the data points in the two data sequences that are closest in time, match them, and then synchronize and adjust other data points according to the time interval and order of data collection. Through bidirectional synchronization alignment, the spatiotemporal mapping relationship between visual data and ranging data in the multi-source data set is established. For example, the timestamps of the enhanced visual data sequence are "2024-01-01 10:00:00.000", "2024-01-01 10:00:01.000", etc., and the collection timestamps of the calibration ranging data sequence are "2024-01-01 10:00:00.100", "2024-01-01 10:00:01.100", etc. Through the time synchronization algorithm, they are matched and adjusted to generate the spatiotemporal mapping relationship between visual data and ranging data.

[0038] Step S240: Cross-verify the spatial resolution of the enhanced visual data sequence and the spatial coordinates of the calibration ranging data sequence according to the spatiotemporal mapping relationship, and generate the verified visual data sequence and the verified ranging data sequence with spatiotemporal consistency constraints.

[0039] The spatial resolution is the spatial detail representation capability of the image in the enhanced visual data sequence, usually represented by pixel size or the number of pixels per unit distance. The spatial coordinates are the three-dimensional coordinate information of the obstacle position in the calibration ranging data sequence. Cross-verification is to use the spatiotemporal mapping relationship to mutually verify and compare the spatial resolution information of the enhanced visual data sequence and the spatial coordinate information of the calibration ranging data sequence to ensure their consistency in space and time. The spatiotemporal consistency constraint is to limit the verified data sequence in time and space to ensure the accuracy and reliability of the data. The verified visual data sequence and the verified ranging data sequence are data sequences obtained after cross-verification and spatiotemporal consistency constraint processing, which have higher consistency and accuracy in space and time.

[0040] When performing cross-validation, the image at a certain time point in the enhanced visual data sequence is compared with the spatial coordinates of the obstacle at the corresponding time point in the calibration ranging data sequence according to the spatio-temporal mapping relationship. For example, the position of the obstacle is identified in the image of the enhanced visual data sequence through an image recognition algorithm, and then the position is compared with the spatial coordinates in the calibration ranging data sequence. If there is a difference between the two, adjustment and correction are needed. At the same time, the verified data sequence is subjected to spatio-temporal consistency constraints to ensure the continuity and accuracy of the data in time and space. For example, at a certain time, the image of the enhanced visual data sequence shows that the obstacle is located on the left side of the shielding door, while the calibration ranging data sequence shows that the spatial coordinates of the obstacle are also on the left side of the shielding door, and the positions are roughly consistent, so the data is considered to pass the cross-validation. After such processing, the verified visual data sequence and the verified ranging data sequence with spatio-temporal consistency constraints are generated.

[0041] Step S250: Multi-modal feature cross-fusion is performed on the edge contour features in the verified visual data sequence and the distance fluctuation features in the verified ranging data sequence to generate a multi-modal monitoring data set containing visual-ranging joint feature vectors, wherein the visual-ranging joint feature vectors adjust the fusion proportion of the edge contour features and the distance fluctuation features through dynamic weight parameters, and the dynamic weight parameters are adaptively adjusted according to the environmental interference residual amount in the calibration ranging data sequence.

[0042] The edge contour features are the edge shape and contour information of the objects in the verified visual data sequence, which can be extracted through image edge detection algorithms such as the Canny edge detection algorithm. The distance fluctuation features are the changes of the distance of the obstacle over time in the verified ranging data sequence, reflecting the motion state and position change of the obstacle. Multi-modal feature cross-fusion is the fusion of different types of features (such as edge contour features and distance fluctuation features) to obtain more comprehensive and accurate information. The visual-ranging joint feature vector is a feature vector obtained by fusing the edge contour features and the distance fluctuation features, which can comprehensively reflect the visual and distance features of the objects in the shielding door area. The dynamic weight parameter is a parameter used to adjust the fusion proportion of the edge contour features and the distance fluctuation features, which will be adaptively adjusted according to the environmental interference residual amount in the calibration ranging data sequence. The environmental interference residual amount is the degree of environmental interference remaining in the calibration ranging data sequence after interference suppression processing.

[0043] When performing multimodal feature cross-fusion, the edge contour features from the validated visual data sequence and the distance fluctuation features from the validated ranging data sequence are first extracted separately. Then, dynamic weight parameters are calculated based on the residual environmental interference in the calibrated ranging data sequence. If the residual environmental interference is large, it indicates that the reliability of the ranging data is relatively low; in this case, the weight of the distance fluctuation feature in the fusion can be appropriately reduced, while the weight of the edge contour feature can be increased. Conversely, if the residual environmental interference is small, it indicates that the ranging data is relatively reliable; in this case, the weight of the distance fluctuation feature can be appropriately increased. The fusion ratio of the edge contour feature and the distance fluctuation feature is adjusted by the dynamic weight parameters to generate a visual-ranging joint feature vector. Finally, multiple visual-ranging joint feature vectors are combined into a multimodal monitoring dataset containing the visual-ranging joint feature vector. For example, in subway platform safety monitoring, the Canny edge detection algorithm is used to extract the edge contour features of obstacles in the validated visual data sequence, and the distance fluctuation of obstacles in the validated ranging data sequence is calculated. Based on the residual amount of environmental interference in the calibration ranging data sequence, the fusion ratio of edge contour features and distance fluctuation features is dynamically adjusted to generate a visual-ranging joint feature vector, thus obtaining a multimodal monitoring data set.

[0044] As one implementation method, step S200, which involves spatiotemporal alignment of the multi-source data set to generate a multimodal monitoring data set fused with spatiotemporal features, may also include steps S260-S290: Step S260: Denoising and edge enhancement processing is performed on each frame of the visual data sequence of the shielded door area to eliminate interference from illumination changes and motion blur, generating a high-definition standard visual data sequence. Denoising removes noise present in the image, which may be caused by electronic noise from the camera itself, ambient light interference, etc. Edge enhancement highlights the edge information of objects in the image, making the object's outline clearer. Illumination change interference is the impact of changes in ambient lighting conditions, such as flickering lights and changes in the intensity of natural light, on image quality. Motion blur interference is caused by objects moving too fast or camera shaking, resulting in blurred edges of objects in the image. The standard visual data sequence is a high-definition visual data sequence obtained after denoising and edge enhancement processing, in which the edges of objects are clear and the image quality is significantly improved. When performing denoising, various denoising algorithms can be used, such as median filtering and Gaussian filtering. Median filtering replaces the gray value of each pixel in the image with the median of the gray values ​​of its neighboring pixels, effectively removing impulse noise such as salt-and-pepper noise. Gaussian filtering, on the other hand, performs convolution operations on the image using a Gaussian function to smooth noise. For edge enhancement, edge detection operators such as the Sobel and Prewitt operators can be used to perform convolution operations on the image, highlighting the edge information of objects. For example, for a frame in a visual data sequence of a screen door area, median filtering is first used to remove noise, and then the Sobel operator is used for edge enhancement to eliminate interference from lighting changes and motion blur, generating a high-resolution standard visual data sequence.

[0045] Step S270: Filter and time-frequency convert the obstacle distance data sequence to extract the spectral and spatial distribution features of the effective ranging signal and generate a standardized ranging data sequence with environmental noise eliminated.

[0046] Filtering involves filtering the obstacle distance data sequence to remove noise and interference signals, retaining only the valid ranging signal. Time-frequency conversion transforms the time-domain obstacle distance data sequence into the frequency domain to analyze the signal's spectral characteristics. Spectral characteristics represent the signal's frequency distribution in the frequency domain; analyzing these characteristics reveals the signal's frequency components and energy distribution. Spatial distribution characteristics indicate the spatial distribution of obstacles; extracting these characteristics allows for more accurate obstacle location. The standardized ranging data sequence, obtained after filtering and time-frequency conversion, eliminates environmental noise, resulting in more accurate and reliable data.

[0047] During filtering, low-pass filters and band-pass filters can be used to filter the obstacle distance data sequence. Low-pass filters remove high-frequency noise while retaining the effective low-frequency ranging signal; band-pass filters retain only signals within a set frequency range, further improving signal quality. For time-frequency conversion, methods such as Fourier transform and wavelet transform can be used to convert the time-domain signal to the frequency domain. For example, the Fast Fourier Transform (FFT) can be used to convert the obstacle distance data sequence to the frequency domain and analyze its spectral characteristics. By extracting the spectral and spatial distribution characteristics of the effective ranging signal, the influence of environmental noise can be eliminated, generating a standardized ranging data sequence. For instance, if high-frequency environmental noise exists in the obstacle distance data sequence, a low-pass filter can remove this noise, and then FFT can be used for time-frequency conversion to extract the spectral and spatial distribution characteristics of the effective ranging signal, resulting in a standardized ranging data sequence.

[0048] Step S280: Register the spatial resolution of the standard visual data sequence with the spatial coordinates of the standardized ranging data sequence to establish a spatial mapping relationship between the visual data and the ranging data.

[0049] Spatial resolution refers to the ability of an image in a standard visual data sequence to represent spatial detail, typically expressed in pixels or the number of pixels per unit distance. Spatial coordinates are the three-dimensional coordinate information of an obstacle's position in a standardized ranging data sequence. Registration is the process of matching and aligning the spatial resolution of a standard visual data sequence with the spatial coordinates of a standardized ranging data sequence, ensuring an accurate spatial correspondence between the two. Spatial mapping is the correspondence between visual data and ranging data in a spatial dimension; by establishing this relationship, different types of data can be spatially associated and integrated.

[0050] During registration, the spatial reference frames for the standard visual data sequence and the standardized ranging data sequence must first be determined. Then, a registration algorithm, such as a feature point matching algorithm, is used to match the feature points of the image in the standard visual data sequence with the spatial coordinates of obstacles in the standardized ranging data sequence. For example, feature points, such as corner points and edge points, are extracted from the image in the standard visual data sequence, and then the corresponding obstacle spatial coordinates are found in the standardized ranging data sequence. Through matching and adjustment, a spatial mapping relationship between the visual data and the ranging data is established. For instance, if a corner point is extracted from the image in the standard visual data sequence, the corresponding obstacle spatial coordinates in the standardized ranging data sequence are found using a registration algorithm, thus establishing a spatial mapping relationship between the two.

[0051] Step S290: Based on the spatial mapping relationship, perform data fusion on the standard visual data sequence and the standardized ranging data sequence to generate a multimodal monitoring data set containing complementary features of multi-source data.

[0052] Data fusion integrates and merges different types of data to obtain more comprehensive and accurate information. Complementary features of multi-source data are unique characteristics inherent in both standard visual data sequences and standardized ranging data sequences. Data fusion combines these complementary features to improve data quality and usability. A multimodal monitoring dataset is a dataset containing fused features from standard visual data sequences and standardized ranging data sequences, used for subsequent foreign object identification and security monitoring.

[0053] During data fusion, data at corresponding positions in standard visual data sequences and standardized ranging data sequences are fused based on the established spatial mapping relationship. Various fusion methods can be employed, such as weighted averaging and feature-level fusion. Weighted averaging assigns different weights to the standard visual data sequences and standardized ranging data sequences based on their reliability and importance, and then averages the data at corresponding positions. Feature-level fusion extracts and fuses features from the standard visual data sequences and standardized ranging data sequences to generate new feature vectors. For example, using weighted averaging to fuse standard visual data sequences and standardized ranging data sequences, weights are assigned to the visual and ranging data based on their reliability, and the data at corresponding positions are averaged to generate a multimodal monitoring dataset containing complementary features from multiple data sources.

[0054] Step S300: Perform foreign object identification processing based on the multimodal monitoring data set to generate foreign object monitoring results and foreign object location distribution information in the shielded door area.

[0055] Foreign object (FOO) identification processing identifies foreign objects within the platform screen door area from a multimodal monitoring dataset. Foreign objects are objects that should not be present within the platform screen door area, such as items dropped by passengers or animals. FEO monitoring results, after FEO identification processing, provide information about the presence and type of foreign objects. FEO location distribution information details the specific location and distribution of foreign objects within the platform screen door area.

[0056] Foreign object (FOO) identification based on multimodal monitoring datasets requires the comprehensive utilization of various feature information within these datasets. Machine learning algorithms and deep learning algorithms can be employed for FEO identification. For example, a convolutional neural network (CNN) can be used to classify the visual-ranging joint feature vectors in the multimodal monitoring dataset to determine the presence and type of FEO. Simultaneously, the spatial coordinate information within the multimodal monitoring dataset can be used to determine the location and distribution of the FEO. For instance, in a subway platform's multimodal monitoring dataset, a CNN algorithm identifies a FEO. Based on the spatial coordinate information in the dataset, the FEO is located in the middle right side of the platform screen door, generating FEO monitoring results and FEO location distribution information that include the FEO's type and location.

[0057] As one implementation method, step S300 involves performing foreign object identification processing based on the multimodal monitoring data set to generate foreign object monitoring results and foreign object location distribution information for the platform screen door area. Specifically, this may include the following steps S310-S340: Step S310: Extracting the spatial distribution features of the platform screen door area and the temporal fluctuation features of the foreign object's movement trajectory from the multimodal monitoring data set. Spatial distribution features refer to the spatial distribution of objects (including foreign objects) within the platform screen door area, such as the object's position, size, and shape. Temporal fluctuation features reflect the change in the foreign object's movement trajectory over time, indicating the object's movement state and trend. When extracting spatial distribution features, the position and shape of objects within the platform screen door area can be determined by analyzing the spatial coordinate information and visual features in the multimodal monitoring data set. For example, an image segmentation algorithm can be used to segment the visual data in the multimodal monitoring data set, identifying the object's boundaries, thereby obtaining the object's shape and location information. When extracting the temporal fluctuation characteristics of foreign object movement trajectories, parameters such as the object's velocity and acceleration can be calculated based on the object's location information at different time points in the multimodal monitoring dataset, and the changes in the object's trajectory can be analyzed. For example, in the multimodal monitoring dataset, the shape and location of a foreign object within the shielded door area can be identified using an image segmentation algorithm. Then, based on the location information at different time points, the object's velocity and acceleration can be calculated, yielding the temporal fluctuation characteristics of the object's trajectory.

[0058] Step S320: Construct a foreign object discrimination model based on spatial distribution characteristics and temporal fluctuation characteristics, and update the parameters of the foreign object discrimination model through a preset foreign object feature library.

[0059] The foreign object detection model is used to determine the presence and type of foreign objects within the shielded door area. Spatial distribution characteristics and temporal fluctuation characteristics are important bases for constructing the foreign object detection model, as they reflect the spatial location and movement state of the foreign objects. The preset foreign object feature library is a database that pre-stores various foreign object feature information, including the shape, size, and movement pattern of the foreign objects. Parameter updates are performed by adjusting and optimizing the parameters of the foreign object detection model based on the information in the preset foreign object feature library, thereby improving the model's detection accuracy.

[0060] When constructing a foreign object discrimination model, deep learning models, such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), can be used. Taking CNNs as an example, their model architecture typically includes an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The input layer receives spatial distribution features and temporal fluctuation features from a multimodal monitoring dataset. The convolutional layers extract features from the data using convolutional kernels, the pooling layers reduce the dimensionality of the features, the fully connected layers combine and classify the extracted features, and the output layer outputs the discrimination result. During model training, data from a pre-defined foreign object feature library is used to train the model, and the model parameters are adjusted using the backpropagation algorithm. For example, data from a pre-defined foreign object feature library can be used as training samples and input into the CNN model for training. The model's weights and biases can be adjusted based on the training results to update the parameters.

[0061] As one implementation method, step S320 may specifically include the following steps S321~S324: Step S321: Extract spatial texture features and foreign object shape features from the multimodal monitoring dataset based on a convolutional neural network, and construct an initial discrimination model containing a multi-layer feature fusion structure. A convolutional neural network (CNN) automatically extracts features from the data through structures such as convolutional layers and pooling layers. Spatial texture features are the texture information of the object surface within the shielding gate area, such as roughness and fineness. Foreign object shape features are the outline and geometric shape of the foreign object. The multi-layer feature fusion structure integrates feature information from different levels through feature extraction and fusion at multiple levels in the model to improve the model's discrimination ability. The initial discrimination model is a model initially formed during the construction process, and further training and optimization are required. When extracting spatial texture features and foreign object shape features based on a convolutional neural network, the visual data from the multimodal monitoring dataset is first input into the input layer of the CNN. Then, convolution operations are performed between the convolutional kernels in the convolutional layer and the input data to extract local features of the data. Different convolutional kernels can extract different types of features, such as edge features and texture features. Pooling layers reduce the dimensionality of the feature maps output by convolutional layers, decreasing the data's dimensionality while preserving important feature information. Through a combination of multiple convolutional and pooling layers, features at different levels are continuously extracted and fused to obtain spatial texture features and object shape features. When constructing an initial discriminant model containing a multi-layer feature fusion structure, the extracted features are input into a fully connected layer for combination and classification, forming the initial discriminant model.

[0062] Step S322: Train the initial discrimination model using a pre-set foreign object sample library, and dynamically optimize the model parameters using a transfer learning algorithm.

[0063] The pre-collected foreign object sample library is a database containing various foreign object samples with different shapes, sizes, colors, and other characteristics. Model training involves using data from the pre-collected foreign object sample library to train an initial discrimination model. By adjusting the model's parameters, the model can accurately identify foreign objects. Transfer learning algorithms transfer the knowledge of a model trained on one task to another related task. Transfer learning can accelerate model training and improve model performance. Dynamic optimization involves continuously adjusting the model's parameters based on the training results during the model training process, thereby continuously improving the model's performance.

[0064] When training the initial discrimination model using a pre-defined foreign object sample library, the data in the library is divided into a training set and a validation set. The training set is used to train the model, and the validation set is used to evaluate the model's performance. During training, the backpropagation algorithm is used to calculate the model's loss function, and the model's parameters are adjusted based on the gradient of the loss function. When using transfer learning algorithms, a pre-trained CNN model on a large-scale image dataset, such as ResNet or VGG, can be selected, and the parameters of some layers can be transferred to the initial discrimination model. Then, the initial discrimination model is fine-tuned on the pre-defined foreign object sample library, dynamically optimizing the model's parameters. For example, a pre-trained ResNet model can be selected, and the parameters of its convolutional layers can be transferred to the initial discrimination model. Then, the initial discrimination model can be fine-tuned using the training set from the pre-defined foreign object sample library, and the model's parameters can be dynamically adjusted based on the evaluation results of the validation set.

[0065] Step S323: Introduce temporal fluctuation features as a time dimension constraint during model training to enhance the model's ability to identify the continuity of the trajectory of foreign objects.

[0066] Temporal fluctuation characteristics represent the changes in the trajectory of a foreign object over time, reflecting its motion state and trend. The time dimension constraint is incorporated into the model during training, allowing it to account for the temporal continuity of the object's motion. Enhancing the model's ability to identify the continuity of the object's trajectory is achieved by introducing temporal fluctuation characteristics, enabling the model to more accurately identify the trajectory and avoid misjudgments caused by discontinuous trajectories.

[0067] When introducing temporal fluctuation features as a time-dimensional constraint during model training, these features can be input into the model along with spatial distribution features and object shape features. For example, in a CNN model, a time-series processing layer, such as a Long Short-Term Memory (LSTM) layer, can be added before the fully connected layers, and the temporal fluctuation features can be processed in the LSTM layer. The LSTM layer can remember long-term dependencies in the time series, thereby enhancing the model's ability to continuously identify the trajectory of foreign objects. During training, spatial distribution features, object shape features, and temporal fluctuation features are considered simultaneously, and the model parameters are adjusted to better adapt the model to changes in the movement of foreign objects. For example, in training a foreign object recognition model for subway platforms, temporal fluctuation features are input into the LSTM layer and trained together with spatial distribution features and object shape features. By adjusting the model parameters, the model's ability to continuously identify the trajectory of foreign objects is enhanced.

[0068] Step S324: Based on the error feedback between the trained model output and the real-time monitoring data, dynamically adjust the classification threshold of the foreign object discrimination model to generate an optimized foreign object discrimination model.

[0069] The trained model output is the result obtained by the trained foreign object discrimination model after judging real-time monitoring data, such as the presence and type of foreign object. Real-time monitoring data refers to data collected in real time from a multimodal monitoring dataset. Error feedback compares the trained model output with the actual situation of the real-time monitoring data and calculates the error between the two. The classification threshold is a threshold used by the model when making classification judgments. When the probability value output by the model is greater than the threshold, it is judged that a foreign object exists; when the probability value is less than the threshold, it is judged that no foreign object exists. Dynamic adjustment adjusts the classification threshold in real time based on the error feedback results to improve the model's discrimination accuracy. The optimized foreign object discrimination model is a higher-performing model obtained after dynamic adjustment of the classification threshold.

[0070] When dynamically adjusting the classification threshold of a foreign object detection model based on the error feedback between the trained model output and real-time monitoring data, the error between the trained model output and the real-time monitoring data is first calculated. Mean squared error (MSE) and cross-entropy loss can be used to measure this error. Then, the classification threshold is adjusted according to the magnitude and direction of the error. If the error is large, it indicates that the current classification threshold may be inappropriate and needs adjustment. For example, if the model misclassifies a foreign object frequently, the classification threshold can be appropriately increased; if the model misclassifies a foreign object frequently, the classification threshold can be appropriately decreased. By continuously adjusting the classification threshold, the model's discrimination results become more accurate, generating an optimized foreign object detection model. For example, in foreign object recognition on subway platforms, the trained model had a high misclassification rate for real-time monitoring data. Error calculation revealed that the classification threshold was set unreasonably. Therefore, the classification threshold was dynamically adjusted based on the error feedback, ultimately generating an optimized foreign object detection model.

[0071] Step S330: Use the updated foreign object discrimination model to perform multimodal feature fusion analysis on the multimodal monitoring data set to determine the presence status of foreign objects and their spatial coverage within the shielded door area.

[0072] The updated foreign object detection model is a model that has undergone parameter updates and optimization, resulting in higher detection accuracy. Multimodal feature fusion analysis integrates and analyzes different types of features (such as spatial distribution features, temporal fluctuation features, and visual-ranging joint features) from the multimodal monitoring dataset to obtain more comprehensive and accurate information. Foreign object presence status refers to the presence and state of foreign objects within the platform screen door area, such as whether they are stationary or in motion. Spatial coverage range refers to the spatial extent occupied by the foreign object within the platform screen door area.

[0073] When performing multimodal feature fusion analysis on a multimodal monitoring dataset using the updated foreign object discrimination model, various features from the dataset are input into the updated model. The model fuses and analyzes these features, determining the presence and type of foreign objects within the platform screen door area based on preset discrimination rules. Simultaneously, it determines the spatial coverage area of ​​the foreign object based on the spatial coordinate information in the multimodal monitoring dataset. For example, by inputting spatial distribution features, temporal fluctuation features, and visual-ranging joint features from the multimodal monitoring dataset into the updated model, the model analyzes these features to determine the presence of a foreign object within the platform screen door area, and then determines, based on the spatial coordinate information in the dataset, that the spatial coverage area of ​​the foreign object is a portion of the left side of the platform screen door.

[0074] Step S340: Generate foreign object monitoring results containing foreign object type identifiers and location coordinate sets based on the foreign object's existence status and spatial coverage, and generate foreign object location distribution information based on the location coordinate set.

[0075] Foreign object type identification is used to identify the type of foreign object, such as luggage or animals. The location coordinate set is the specific location coordinate information of the foreign object within the platform screen door area. The foreign object monitoring result includes both the foreign object type identification and the location coordinate set, used to indicate the status of foreign objects within the platform screen door area. Foreign object location distribution information is generated based on the location coordinate set and is used to describe the distribution of foreign objects within the platform screen door area.

[0076] When generating foreign object (FOO) monitoring results based on the presence and spatial coverage of FEOs, the type of FEO is first determined according to the output of the updated FEO discrimination model, and a corresponding FEO type identifier is added. Then, the location coordinates of the FEO are extracted from the multimodal monitoring dataset to form a location coordinate set. The FEO type identifier and the location coordinate set are combined to generate the FEO monitoring result. When generating FEO location distribution information based on the location coordinate set, map visualization technology can be used to mark the location coordinate set of FEOs on a map of the platform screen door area, visually displaying the distribution of FEOs. For example, if the updated FEO discrimination model determines that there is a piece of luggage in the platform screen door area, a "luggage" FEO type identifier is added, and the location coordinates of the luggage are extracted from the multimodal monitoring dataset to form a location coordinate set, generating a FEO monitoring result containing "luggage" and the location coordinate set. Then, the location coordinate set is marked on a map of the platform screen door area to generate FEO location distribution information.

[0077] Step S400: Based on the foreign object monitoring results and the passenger flow data sequence in the multi-source dataset, perform density prediction processing to generate passenger line density prediction results and density change trend information.

[0078] Density prediction processing involves forecasting the passenger linear density within the platform screen doors area based on foreign object monitoring results and passenger flow data sequences. Passenger linear density is the number of passengers per unit length within the platform screen doors area. The passenger linear density prediction result is a predicted value for the passenger linear density over a future period, obtained after density prediction processing. Density change trend information describes the trend of passenger linear density over time, such as increasing, decreasing, or remaining stable.

[0079] When performing density prediction, it is necessary to comprehensively consider information from foreign object monitoring results and passenger flow data sequences. The location distribution information of foreign objects in the monitoring results affects passenger flow paths and aggregation, thus influencing passenger linear density. The flow rate and direction distribution characteristics in the passenger flow data sequences reflect passenger flow patterns and are important bases for density prediction. By establishing a suitable density prediction model and combining information from foreign object monitoring results and passenger flow data sequences, passenger linear density is predicted, and its changing trends are analyzed. For example, in a subway platform, foreign object monitoring results indicate the presence of a foreign object within the platform screen door area; its location affects passenger flow paths. Simultaneously, passenger flow rate and direction distribution information are obtained from the passenger flow data sequences. Using this information, a density prediction model is used to predict the passenger linear density in the platform screen door area over a future period, and its changing trends are analyzed, generating passenger linear density prediction results and density change trend information.

[0080] As one implementation method, step S400 involves performing density prediction processing based on the foreign object monitoring results and the passenger flow data sequence in the multi-source dataset to generate passenger line density prediction results and density change trend information. Specifically, this may include the following steps S410 to S440: Step S410: Extract the flow rate characteristics and direction distribution characteristics of the passenger flow data sequence from the multi-source dataset, and establish the correlation mapping relationship between the flow rate characteristics and the foreign object location distribution information.

[0081] Flow rate characteristics refer to the speed at which passengers move within the platform screen doors area, reflecting the speed of passenger movement. Directional distribution characteristics refer to the direction of passenger movement within the platform screen doors area, such as left, right, or forward. The correlation mapping relationship is the correspondence between flow rate characteristics and the location distribution information of foreign objects. By establishing this relationship, the impact of foreign objects on passenger flow rate can be analyzed.

[0082] When extracting flow rate and directional distribution features from passenger flow data sequences in a multi-source dataset, object detection and tracking algorithms can be used to analyze video data within the passenger flow data sequences. For example, the YOLO algorithm can be used to detect passengers in the video, and the KCF algorithm can be used to track the passengers, calculating their movement speed and direction to obtain flow rate and directional distribution features. When establishing the correlation between flow rate features and foreign object location distribution information, the area where the foreign object is located is determined based on the foreign object location distribution information, and then the flow rate features of passengers near that area are analyzed. If the presence of a foreign object causes a change in the passenger flow path, then the flow rate of passengers near that area will also change accordingly. In this way, a correlation between flow rate features and foreign object location distribution information is established. For example, in a multi-source dataset of a subway platform, the YOLO and KCF algorithms can be used to extract flow rate and directional distribution features from the passenger flow data sequence. Based on the information on the location of the foreign object, it was found that there was a foreign object on the left side of the platform screen door. Analyzing the flow rate of passengers in the vicinity of the area, it was found that the flow rate of passengers was significantly reduced due to the presence of the foreign object, thus establishing a correlation mapping relationship between the flow rate characteristics and the location distribution information of the foreign object.

[0083] Step S420: Analyze the flow rate characteristics and directional distribution characteristics based on the preset density prediction model to generate the initial density prediction value of the passenger gathering area.

[0084] The preset density prediction model is a pre-established model for predicting passenger line density, which can be based on machine learning or deep learning. Flow rate characteristics and directional distribution characteristics are important input information for density prediction, reflecting passenger flow patterns. Passenger gathering areas are areas where passengers are relatively concentrated within the platform screen doors. The initial density prediction value is a preliminary density prediction value for passenger gathering areas obtained after analyzing the flow rate characteristics and directional distribution characteristics based on the preset density prediction model.

[0085] When analyzing flow rate and directional distribution characteristics based on a pre-defined density prediction model, these characteristics are input into the model. The model then predicts the density of passenger gathering areas based on the input features, combined with its internal algorithms and parameters. For example, the pre-defined density prediction model can be a multilayer perceptron (MLP) model. Its input layer receives flow rate and directional distribution characteristics, the hidden layer performs nonlinear transformations and feature extraction on the input features, and the output layer outputs the initial density prediction value for the passenger gathering area. When training the pre-defined density prediction model, historical passenger flow data and corresponding density data can be used for training, and the model's parameters can be adjusted to accurately predict passenger density. For example, in a subway platform, the extracted flow rate and directional distribution characteristics are input into the pre-defined MLP density prediction model, and the model predicts the initial density value of the passenger gathering area based on these characteristics.

[0086] As one implementation method, step S420 may specifically include the following steps S421-S426: Step S421: Perform time series decomposition processing on the flow rate characteristics, extract the periodic trend component and random disturbance component of passenger flow, and perform spatiotemporal correlation mapping between the periodic trend component and the directional distribution characteristics to generate an initial density distribution feature set. Time series decomposition processing treats the flow rate characteristics as a time series and decomposes it into different components, such as the periodic trend component and the random disturbance component. The periodic trend component is the periodic change trend of passenger flow rate over time, for example, the passenger flow rate is higher during the morning and evening peak hours and lower during the midday period. The random disturbance component is the fluctuation in passenger flow rate caused by random factors (such as sudden events, special behaviors of individual passengers, etc.). Spatiotemporal correlation mapping is to associate and match the periodic trend component and the directional distribution characteristics in the time and space dimensions to generate an initial density distribution feature set reflecting the passenger flow and distribution. When performing time series decomposition processing on the flow rate characteristics, seasonal decomposition algorithms, such as additive models or multiplicative models, can be used. Additive models, for example, involve adding a trend component, a seasonal component, and a random disturbance component to a time series, while multiplicative models involve multiplying these components. Taking the additive model as an example, the flow rate characteristics are first smoothed to remove random noise, yielding the trend component. Then, the trend component is removed through differencing to obtain the seasonal component. Finally, the trend and seasonal components are subtracted from the original series to obtain the random disturbance component. When mapping the periodic trend component to directional distribution characteristics in a spatiotemporal relationship, the flow trend of passengers in different time periods is determined based on the periodic trend component, and then the spatial distribution of passengers is determined by combining it with the directional distribution characteristics. For example, during morning and evening rush hours, the periodic trend component shows a higher passenger flow rate; combined with the directional distribution characteristics, it is known that passengers mainly flow from the platform entrance to the train, thus generating an initial density distribution feature set.

[0087] Step S422: Configure a multilayer learning network for the density prediction model based on the initial density distribution feature set. In the model training phase, introduce random perturbation components as input noise parameters to enhance the robustness of the multilayer learning network to abnormal flow patterns.

[0088] Multilayer learning networks are multilayer neural network structures in density prediction models, such as Multilayer Perceptrons (MLPs) and Long Short-Term Memory (LSTM) networks. The initial density distribution feature set, obtained after spatiotemporal correlation mapping, reflects passenger flow and distribution and is used to configure the input and hidden layers of the multilayer learning network. Random perturbation components are the random fluctuations in passenger flow rates extracted through time-series decomposition. Input noise parameters are random noise introduced during model training to simulate uncertainties and anomalies in real-world scenarios. Enhancing the robustness of the multilayer learning network to abnormal flow patterns involves introducing random perturbation components as input noise parameters, enabling the network to better handle abnormal passenger flow patterns and improving model stability and accuracy. When configuring the multilayer learning network for the density prediction model based on the initial density distribution feature set, the features in the initial set are used as input to the network. The number and type of neurons in the input layer are determined based on the quantity and type of features. In the hidden layers, nonlinear activation functions are used to transform and extract the input features, increasing the model's expressive power. During model training, random perturbation components are introduced as input noise parameters. Random perturbation components can be added to the input features at a set ratio, enabling the model to learn features of abnormal flow patterns during training. For example, in an MLP-based density prediction model, the initial density distribution feature set is used as the input to the input layer. During the training phase, random perturbation components are added to the input features at a ratio of 10%, and the model parameters are adjusted through the backpropagation algorithm to enhance the robustness of the multi-layer learning network to abnormal flow patterns.

[0089] Step S423: Couple the influence range characteristics of the obstruction area and the directional distribution characteristics in the foreign object location distribution information to generate passenger path offset parameters and path adjustment frequency parameters caused by foreign object obstruction.

[0090] The obstruction area influence range characteristic refers to the area occupied by the foreign object in the foreign object location distribution information and the range of its influence on passenger flow. The directional distribution characteristic refers to the flow direction of passengers within the platform screen door area. Coupled analysis integrates the obstruction area influence range characteristic and the directional distribution characteristic, considering their interaction and influence. The passenger path offset parameter is the degree to which the passenger flow path deviates due to foreign object obstruction, such as the offset distance or angle. The path adjustment frequency parameter is the frequency at which passengers adjust their flow path to avoid foreign objects.

[0091] When coupling the impact range characteristics of the obstruction area with the directional distribution characteristics in the foreign object location distribution information, the scope of the obstruction area is first determined based on the foreign object location distribution information. Then, the impact of this area on passenger flow in different directions is analyzed. If a foreign object blocks the normal flow path of passengers, passengers will be forced to adjust their paths, resulting in path deviation and path adjustment. By analyzing a large amount of passenger flow data, the path deviation and path adjustment frequency of passengers are statistically analyzed to generate passenger path deviation parameters and path adjustment frequency parameters. For example, in a subway platform, the foreign object location distribution information indicates that there is a foreign object in the middle of the platform screen door, and its obstruction area has a large impact range. Analyzing the directional distribution characteristics of passengers reveals that passengers who would normally pass through this area will shift to both sides. The offset distance and path adjustment frequency of these passengers are statistically analyzed to generate passenger path deviation parameters and path adjustment frequency parameters.

[0092] Step S424: Dynamically correct the output of the multilayer learning network based on the path offset parameter and the path adjustment frequency parameter to generate a density prediction correction value that incorporates the influence of foreign object blockage.

[0093] The output of the multi-layer learning network in the density prediction model is the passenger density prediction value calculated by the multi-layer learning network based on input features (such as the initial density distribution feature set). Dynamic feedback correction adjusts and corrects the output of the multi-layer learning network in real time based on path offset parameters and path adjustment frequency parameters, considering the impact of obstruction on passenger density. The density prediction correction value, incorporating the impact of obstruction, is a more accurate passenger density prediction value obtained after dynamic feedback correction, comprehensively considering the impact of obstruction on passenger flow and density.

[0094] When dynamically correcting the output of a multilayer learning network based on path offset and path adjustment frequency parameters, the relationship between these parameters and passenger density is first analyzed. A large path offset parameter indicates a significant change in passenger flow paths, potentially leading to increased passenger density in some areas and decreased density in others. A high path adjustment frequency parameter indicates frequent path adjustments by passengers, also affecting passenger distribution and density. Based on these relationships, the output of the multilayer learning network is corrected. For example, if the path offset parameter indicates passengers are shifting towards a certain area, the predicted density value for that area can be appropriately increased; if the path adjustment frequency parameter is high, the predicted density value can be smoothed to reflect dynamic changes in passenger distribution. Through this dynamic feedback correction, a corrected density prediction value incorporating the effects of foreign object blockage is generated.

[0095] Step S425: The density prediction correction value is adjusted by the iterative optimization module of the multi-layer learning network, and the network weight parameters are updated in combination with the changing pattern of the periodic trend component to generate an initial density prediction value containing the foreign object influence factor.

[0096] The iterative optimization module of the multilayer learning network is used to optimize model parameters in the density prediction model, employing algorithms such as stochastic gradient descent (SGD) and Adam. The density prediction correction value is a passenger density prediction value that incorporates the impact of foreign object blockages, obtained after dynamic feedback correction. Parameter adjustment is achieved by adjusting the density prediction correction value through the iterative optimization module, making the model's output more accurate. The periodic trend component reflects the periodic variation trend of passenger flow rate, such as daily peak and off-peak hours. Network weight parameters are the connection weights between neurons in the multilayer learning network, determining the model's computational and expressive capabilities. The foreign object impact factor reflects the degree of influence of foreign objects on passenger density; by considering the foreign object impact factor, passenger density can be predicted more accurately.

[0097] When adjusting the parameters of the density prediction correction value through the iterative optimization module of the multi-layer learning network, the density prediction correction value is used as the target value, and the network weight parameters are adjusted using an iterative optimization algorithm. During the adjustment process, the changing patterns of the periodic trend component are incorporated to enable the model to better adapt to passenger flow conditions at different times. For example, during morning and evening peak hours, the model's predicted passenger density value is appropriately increased based on the changing patterns of the periodic trend component. Simultaneously, the foreign object impact factor is introduced into the model; by adjusting the magnitude of the foreign object impact factor, the degree of influence of foreign objects on passenger density is reflected. Through continuous iterative optimization, an initial density prediction value incorporating the foreign object impact factor is generated.

[0098] Step S426: Compare the initial density prediction value with the validation set in the historical density data to determine the error. Based on the comparison result, calibrate the feature fusion threshold of the multilayer learning network to generate the optimized density prediction model output.

[0099] The initial density prediction is the passenger density prediction obtained after parameter adjustment and the introduction of foreign object influence factors. Historical density data refers to passenger density data recorded over a past period for the platform screen door area; the validation set is a subset of this data used to evaluate model performance. Error comparison compares the initial density prediction with the validation set from the historical density data, calculating the error between the two. The feature fusion threshold is a threshold used in a multi-layer learning network to fuse different features; it determines the weight and role of different features in the model. Calibration adjusts the feature fusion threshold based on the error comparison results, enabling the model to better fuse different features and improve prediction accuracy. The optimized density prediction model outputs a more accurate passenger density prediction result after feature fusion threshold calibration.

[0100] When comparing the initial density prediction with the validation set in historical density data, metrics such as mean squared error (MSE) and mean absolute error (MAE) can be used to measure the error. Based on the magnitude and direction of the error, the feature fusion threshold of the multi-layer learning network is adjusted. If the error is large, it indicates that the current feature fusion threshold may be inappropriate and needs adjustment. For example, if a feature has too large a weight in the model, leading to inaccurate predictions, the fusion threshold for that feature can be appropriately reduced. By continuously calibrating the feature fusion threshold, the model can better fuse different features, generating an optimized density prediction model output. For example, in the density prediction of a subway platform, comparing the initial density prediction with the validation set in historical density data yields a large mean squared error. Analysis reveals that the fusion threshold for a certain feature is set unreasonably. Therefore, based on the error comparison results, the fusion threshold for that feature is adjusted, ultimately generating an optimized density prediction model output.

[0101] Step S430: Correct the initial density prediction value based on the foreign object location distribution information to generate the passenger line density prediction result and its corresponding density change rate.

[0102] Foreign object location distribution information refers to the specific location and distribution of foreign objects within the platform screen door area. This affects passenger flow paths and aggregation, thus influencing passenger linear density. The initial density prediction value is a preliminary density prediction of the passenger aggregation area obtained through density prediction model analysis. Correction processing adjusts and corrects the initial density prediction value based on the foreign object location distribution information, making the prediction results more accurate. The passenger linear density prediction result is the predicted value of passenger linear density within the platform screen door area after correction processing. The density change rate is the rate at which passenger linear density changes over time, reflecting the dynamic changes in passenger density.

[0103] When correcting the initial density prediction value based on the location distribution information of foreign objects, the impact of this information on passenger flow is first analyzed. If a foreign object is located on the main passenger flow path, it will obstruct passenger flow, thus increasing the passenger linear density in that area; if the foreign object is located in the peripheral area, its impact on passenger flow is smaller. Based on this relationship, the initial density prediction value is corrected. For example, if a foreign object blocks a passage, the initial density prediction value for the area near that passage can be appropriately increased. After generating the passenger linear density prediction result, the difference between the passenger linear density prediction values ​​at adjacent time points is calculated and divided by the time interval to obtain the corresponding density change rate. For example, if the passenger linear density prediction value is d1 at time t1 and d2 at time t2, with a time interval Δt = t2 - t1, then the density change rate r = (d2 - d1) / Δt. In this way, the initial density prediction value is corrected by comprehensively considering the location distribution information of foreign objects, generating an accurate passenger linear density prediction result and its corresponding density change rate.

[0104] As one implementation method, step S430 may specifically include the following steps S431~S436: Step S431: Extract the obstruction area influence range characteristics from the foreign object location distribution information, and analyze the degree of obstruction of passenger flow paths within the platform screen door area based on the obstruction area influence range characteristics. The obstruction area influence range characteristics describe the location of the foreign object and the range of its obstructive influence on the surrounding space. It can be defined by information such as the spatial coordinates, size, and shape of the foreign object. For example, if the foreign object is a large suitcase placed in the middle of the platform screen door, its obstruction area influence range may be a spatial area extending outwards at a predetermined distance from the suitcase. Analyzing the degree of obstruction of passenger flow paths within the platform screen door area requires combining the layout of the subway platform and the normal flow direction of passengers. A topological model of the subway platform can be established, abstracting the passenger flow path as edges in the graph, with nodes representing key locations on the platform, such as turnstiles, platform screen doors, and stairwells. Then, based on the obstruction area influence range characteristics, determine which edges are obstructed and the degree of obstruction. For example, image processing techniques can be used to analyze visual data of the shielded door area to determine the location and extent of foreign objects in the image. Then, the object can be mapped into a topological model to calculate the length of the obstructed path and its proportion of the total path length, thereby quantifying the degree of obstruction.

[0105] Step S432: Calculate the passenger dwell time parameter and spatial distribution offset parameter corresponding to the blocked area based on the degree of obstruction, and generate a set of correction coefficients that reflect the impact of foreign object obstruction.

[0106] Passenger dwell time is the time passengers spend near a blockage area due to obstruction, and it is closely related to the degree of obstruction. Generally, the higher the degree of obstruction, the longer the passenger dwell time. It can be calculated by analyzing passenger flow data sequences and using target tracking algorithms to track passenger movements near the blockage area. For example, when the obstruction level reaches 80%, most passengers may need to wait a longer time to pass, resulting in a corresponding increase in dwell time. Spatial distribution offset is the spatial movement of passengers to avoid the blockage area. It can be calculated by comparing the spatial distribution of passengers with and without obstruction. For example, density clustering algorithms (such as DBSCAN) can be used to cluster passenger location data to obtain the positional changes of different cluster centers, thus measuring the spatial distribution offset parameter. The correction coefficient set is a set of coefficients used to correct the initial density prediction value, determined based on the passenger dwell time parameter and the spatial distribution offset parameter. A functional relationship can be established, taking the passenger dwell time parameter and the spatial distribution offset parameter as input and outputting the correction coefficients. For example, using a linear weighting method, the correction coefficient C = α × T + β × S, where T is the passenger dwell time parameter, S is the spatial distribution offset parameter, and α and β are weighting coefficients, which are obtained through training with historical data.

[0107] Step S433: Perform a multi-dimensional correlation analysis between the set of correction coefficients and the initial density prediction values ​​to determine the density correction weight and direction for passenger gathering areas.

[0108] Multidimensional correlation analysis examines the relationship between the set of correction coefficients and the initial density prediction value from multiple perspectives and levels. It can be conducted from temporal and spatial dimensions. In the temporal dimension, it considers the impact of changes in correction coefficients over different time periods on the initial density prediction value; in the spatial dimension, it analyzes the correlation between the correction coefficients and the initial density prediction value of different regions. For example, during peak hours, passenger flow is high, and the impact of correction coefficients on the initial density prediction value may be greater. Through multidimensional correlation analysis, the density correction weight and correction direction for each passenger-concentrated area can be determined. The density correction weight represents the degree of influence of the correction coefficient on the initial density prediction value, and the correction direction indicates whether the initial density prediction value is increased or decreased. Regression analysis and other methods can be used to determine the density correction weight and correction direction. For example, a linear regression model D can be established. new =D old +w×C×sign, where D new This is the corrected density prediction value, D. old is the initial density prediction value, w is the density correction weight, C is the correction coefficient, and sign is the correction direction (+1 indicates increase, -1 indicates decrease). Appropriate w and sign values ​​are obtained by fitting historical data.

[0109] Step S434: Perform nonlinear superposition correction on the initial density prediction value based on the density correction weight and correction direction to generate passenger line density prediction results containing density correction values ​​of different time segments.

[0110] Nonlinear superposition correction combines density correction weights, correction directions, and initial density predictions in a nonlinear manner to obtain more accurate correction results. This is because, in reality, passenger density changes are often not linear and may be influenced by a combination of factors. Nonlinear functions, such as the sigmoid function and ReLU function, can be used for correction. For example, for the initial density prediction value D of each time segment... old Based on the corresponding density-adjusted weights w and adjustment direction sign, the sigmoid function is used for correction: D new =σ(w×C×sign)×D old +(1-σ(w×C×sign))×D old , where σ is the sigmoid function. By making such corrections to the initial density predictions for different time segments, passenger line density predictions containing density corrections for different time segments are generated. This result can more accurately reflect the change of passenger line density over time under foreign object blockage conditions.

[0111] Step S435: Combining the density correction value differences between adjacent time segments in the passenger line density prediction results, analyze the fluctuation pattern of the density correction value with the change of the foreign object position, and generate the density change rate reflecting the magnitude and direction of density change.

[0112] Analyzing the fluctuation pattern of density correction values ​​with changes in the location of foreign objects requires comparing the density correction values ​​of adjacent time segments in the passenger line density prediction results. Changes in the location of foreign objects will alter passenger flow paths and aggregation patterns, causing fluctuations in the density correction values. The difference in density correction values ​​between adjacent time segments can be calculated as ΔD = D. i+1 -D i D i and D i+1 These are the density correction values ​​for the i-th and (i+1)-th time segments, respectively. The magnitude of density change can be represented by |ΔD|, reflecting the size of the change in the density correction value. The direction of density change can be determined by the sign of ΔD; if ΔD > 0, it indicates an increase in density, and if ΔD < 0, it indicates a decrease in density. By analyzing the density correction values ​​of multiple adjacent time segments in this way, a sequence of density change rates can be obtained. For example, within a certain period, the density correction value is recorded every minute, and the density change rate between adjacent minutes is calculated, thereby understanding the dynamic changes in density with time and the location of the foreign object.

[0113] Step S436: Perform a spatiotemporal correlation mapping between the density change rate and the passenger line density prediction results to generate a final passenger line density prediction result set containing density prediction values, change rates, and foreign object impact indicators. The foreign object impact indicators are used to indicate the correction area and correction intensity in the density prediction results affected by the distribution of foreign object locations.

[0114] Spatiotemporal correlation mapping associates and matches the density change rate with the passenger line density prediction results in both time and space dimensions. In the time dimension, it ensures that the density change rate corresponds to the passenger line density prediction results for the corresponding time segment; in the spatial dimension, it associates the density change rate with specific areas in the passenger line density prediction results. Foreign object impact indicators can be determined through analysis of the correction coefficient set and density correction weights. For example, areas with correction coefficients greater than a certain threshold are marked as correction areas affected by foreign objects, with the magnitude of the correction coefficient indicating the correction intensity. The final passenger line density prediction result set integrates the density prediction value, change rate, and foreign object impact indicator, providing more comprehensive and accurate information for subsequent safety warnings. For example, on a subway platform map, the density prediction value, density change rate, and foreign object impact indicator for each area are marked, allowing staff to intuitively understand the dynamic changes in passenger density within the platform and the impact of foreign objects on different areas.

[0115] Step S440: Perform trend extrapolation processing on the passenger line density prediction results based on the density change rate to generate density change trend information that includes the density fluctuation curve in the time dimension.

[0116] Trend extrapolation is a process that predicts the trend of passenger line density over a future period based on known rates of density change and passenger line density forecasts. A time-dimensional density fluctuation curve is a visual representation that shows how passenger line density changes over time, including trends such as increases, decreases, and fluctuations. Trend extrapolation helps subway station managers prepare in advance, such as adjusting passenger flow management plans and scheduling train operations.

[0117] As one implementation method, step S440 may specifically include the following steps S441-S446: Step S441: Extract the rate change direction feature and rate duration feature from the density change rate, and analyze the cumulative growth or decline trend of passenger density in different time segments based on the rate change direction feature. The rate change direction feature is the positive or negative value of the density change rate; positive indicates an increase in density, and negative indicates a decrease in density. The rate duration feature is the duration for which the density change rate remains in a certain direction (increasing or decreasing). By extracting these two features, a deeper understanding of the passenger density change trend can be obtained. For example, if the density change rate is positive for 5 consecutive minutes and the rate value gradually increases, it indicates that the passenger density shows a cumulative growth trend within these 5 minutes; if it is negative for 3 consecutive minutes and the rate value is stable, it indicates that the passenger density shows a cumulative decline trend within these 3 minutes. A sliding window method can be used to extract the rate duration feature, and the direction and duration of the density change rate can be statistically analyzed within each window. When analyzing the cumulative growth or decline trend of passenger density in different time segments based on the rate change direction feature, the density change rate of each time segment can be integrated to obtain the cumulative change in passenger density within that time segment. For example, within a 10-minute time segment, the rate of density change per minute is summed to obtain the cumulative increase or decrease in passenger density over those 10 minutes.

[0118] Step S442: Construct a trend prediction model associated with the time dimension based on the cumulative growth or decline trend, and dynamically initialize the trend prediction model based on the distribution characteristics of the density correction value in the passenger line density prediction results.

[0119] Trend prediction models are used to predict future passenger line density trends. They are time-dependent and can predict future density changes based on current cumulative growth or decline trends. Suitable time series prediction models can be chosen, such as ARIMA (Autoregressive Integral Moving Average) and LSTM (Long Short-Term Memory). Taking the ARIMA model as an example, it consists of three parts: autoregression (AR), differencing (I), and moving average (MA). The model order is determined based on the cumulative growth or decline trend, such as the autoregression order p, the differencing order d, and the moving average order q. Dynamic parameter initialization involves adjusting the initial parameters of the trend prediction model based on the distribution characteristics of the density correction values ​​in the passenger line density prediction results. For example, analyzing the statistical characteristics of the density correction values, such as the mean and variance, and using these characteristics as initial parameters input into the trend prediction model allows it to better adapt to actual conditions. When using the LSTM model, the time series of density correction values ​​can be used as input, and appropriate weights and bias parameters can be obtained through training for dynamic parameter initialization.

[0120] Step S443: Use a trend prediction model to perform multi-period extrapolation calculations on the density change rate to generate a preliminary density fluctuation curve covering a preset future time range.

[0121] Multi-period extrapolation calculation uses a trend prediction model to predict the rate of density change over multiple time periods. The preset time range is the length of the future time to be predicted, such as the next 30 minutes or 1 hour. The current rate of density change and the passenger line density prediction results are input into the trend prediction model. The model then predicts the rate of density change within the preset time range based on its internal algorithm and parameters. Next, based on the predicted rate of density change and the current passenger line density prediction results, the predicted passenger line density value for each future time point is calculated. Connecting these predicted values ​​generates a preliminary density fluctuation curve covering the preset future time range. For example, using a trained ARIMA model to predict the rate of density change over the next 30 minutes, with each 5-minute period, yields prediction values ​​for 6 time periods. These prediction values ​​are then used to calculate the corresponding passenger line density prediction values, and the preliminary density fluctuation curve is plotted.

[0122] Step S444: Perform feature matching verification between the preliminary density fluctuation curve and the periodic fluctuation pattern in historical density data to identify abnormal fluctuation segments in the preliminary density fluctuation curve that deviate from the historical pattern.

[0123] The periodic fluctuations in historical density data represent the periodic changes in passenger line density over a past period, such as higher passenger density during morning and evening rush hours and lower density during midday. Feature matching verification compares and matches the characteristics of the preliminary density fluctuation curve (such as the location of peaks and troughs, and the amplitude of fluctuations) with the periodic fluctuations in historical density data. Methods such as correlation analysis and pattern recognition can be used for verification. For example, the correlation coefficient between the preliminary density fluctuation curve and historical density data can be calculated. A low correlation coefficient indicates that the preliminary density fluctuation curve may deviate from historical patterns. Through feature matching verification, abnormal fluctuation segments in the preliminary density fluctuation curve that deviate from historical patterns can be identified. These abnormal fluctuation segments may be caused by foreign object obstruction, sudden events, etc., and require further analysis and processing.

[0124] Step S445: Adjust the extrapolation parameters of the trend prediction model based on the abnormal fluctuation segments to generate an optimized trend prediction model that eliminates deviation features.

[0125] Extrapolation parameters are parameters used in trend prediction models for extrapolation calculations, such as the autoregressive coefficients and moving average coefficients in the ARIMA model. Feedback adjustment involves adjusting the extrapolation parameters of the trend prediction model based on the characteristics of abnormal fluctuation segments, enabling the model to better predict future passenger line density trends. For example, if the fluctuation amplitude in a certain period of the initial density fluctuation curve is found to be significantly greater than historical patterns, it indicates that the trend prediction model may have underestimated the changes in that period. The extrapolation parameters can be appropriately adjusted to increase the predicted fluctuation amplitude for that period. By continuously analyzing and adjusting the extrapolation parameters for abnormal fluctuation segments, an optimized trend prediction model that eliminates deviation features is generated. Optimization algorithms such as gradient descent can be used to adjust the extrapolation parameters, making the model's prediction results more consistent with historical patterns.

[0126] Step S446: The density change rate is extrapolated twice by the optimized trend prediction model to generate density change trend information containing density peak nodes and change direction indicators in the time dimension. The density fluctuation curve in the time dimension in the density change trend information reflects the critical state and trend persistence of density change by dynamically annotating the peak nodes and change direction indicators.

[0127] Secondary extrapolation involves using the optimized trend prediction model to extrapolate the density change rate again, resulting in a more accurate trend of future passenger line density changes. The time-dimensional density peak node is the position of the peak in the time-dimensional density fluctuation curve, representing the time point when the passenger line density reaches its maximum value. The change direction indicator indicates the direction of density change, such as increase or decrease. By dynamically labeling peak nodes and change direction indicators, the critical state and trend persistence of density changes can be reflected more intuitively. For example, by labeling the time and density values ​​of each peak node on the time-dimensional density fluctuation curve and using arrows to indicate the direction of density change, staff can clearly see the trend and critical state of passenger line density changes. The generated density change trend information provides important information for the safety management and emergency decision-making of subway platforms.

[0128] Step S500: Generate a safety warning strategy based on the location distribution information and density change trend information of foreign objects, and transmit the safety warning strategy to the subway platform control system to trigger emergency response operations.

[0129] Safety early warning strategies are a series of countermeasures formulated based on information on the location and density trends of foreign objects, aiming to ensure the safe operation of subway platforms and the safety of passengers. Information on the location and distribution of foreign objects indicates the specific location and impact range of objects within the platform screen doors area, while information on density trends reflects the changing trends in passenger linear density and potential hazards. By comprehensively considering these two pieces of information, a reasonable safety early warning strategy can be developed. The safety early warning strategy is transmitted to the subway platform control system, which can trigger corresponding emergency response operations based on the strategy's content, such as adjusting train operation plans, activating emergency lighting, and issuing broadcast announcements.

[0130] As one implementation method, step S500, generating a safety warning strategy based on the foreign object location distribution information and density change trend information, may specifically include the following steps S510~S540: Step S510: Determine the obstruction impact area of ​​the platform screen door area based on the foreign object location distribution information, and extract the passenger number and dwell time distribution parameters within the obstruction impact area. Determining the obstruction impact area of ​​the platform screen door area based on the foreign object location distribution information requires considering the size, shape, and location of the foreign object, as well as the layout of the platform screen door area and the passenger flow path. A three-dimensional model of the platform screen door area can be established, mapping the location information of the foreign object into the model to simulate the obstruction effect of the foreign object on passenger flow, thereby determining the obstruction impact area. For example, computer simulation software can be used to place the three-dimensional model of the foreign object within the three-dimensional model of the platform screen door area to simulate passenger flow, and the obstruction impact area can be determined based on the simulation results. Extracting the passenger number and dwell time distribution parameters within the obstruction impact area can be achieved by analyzing the visual data of the platform screen door area and passenger flow data sequences. Target detection and tracking algorithms can be used to identify and track passengers in the visual data, and the number of passengers within the obstruction impact area can be counted. At the same time, the dwell time of each passenger in the congestion-affected area is recorded, and the distribution parameters of the dwell time, such as mean and variance, are obtained through data analysis.

[0131] Step S520: Combine the time dimension density fluctuation curve in the density change trend information to calculate the density risk score and risk diffusion rate of the blockage-affected area.

[0132] Density risk score is an indicator that measures the degree of risk posed by passenger density within a congestion-affected area. It is closely related to the density fluctuation curve over time. The density risk score can be calculated based on characteristics such as density peaks and fluctuation amplitudes in the time-dimensional density fluctuation curve. For example, when the density peak exceeds a certain safety threshold, it indicates that the passenger density in that area is too high, posing a high safety risk, and a higher density risk score can be assigned. Risk diffusion rate is the speed at which risk spreads from the congestion-affected area to surrounding areas. It can be calculated by analyzing the propagation of the time-dimensional density fluctuation curve across different areas. For example, by observing the changes in the time-dimensional density fluctuation curve in adjacent areas and calculating the gradient of density change, the risk diffusion rate can be measured. A risk assessment model can be established, taking the characteristics of the time-dimensional density fluctuation curve as input and outputting the density risk score and risk diffusion rate. This model allows for a more accurate assessment of the safety risks in congestion-affected areas.

[0133] Step S533: Generate a safety warning strategy that includes warning level indicators and emergency evacuation routes based on the density risk score and risk diffusion rate.

[0134] Warning level indicators are used to distinguish different levels of safety risk, generally categorized into low, medium, and high levels. The warning level indicator is determined based on the density risk score and the risk diffusion rate. For example, a low warning level indicates a low density risk score and a low risk diffusion rate, while a high warning level indicates a high density risk score and a high risk diffusion rate. Emergency evacuation routes are paths used to guide passengers during a safety hazard. These routes can be planned based on the layout of the platform screen door area, passenger flow, and the location and distribution of obstructions. For example, shortest path algorithms (such as Dijkstra's algorithm) can be used to find the shortest path from the congestion area to the safety exit in the topological model of the platform screen door area. Combining the warning level indicators and emergency evacuation routes generates a safety warning strategy. This strategy provides clear response guidance for subway platform staff.

[0135] Step S540: Dynamically match the emergency evacuation routes in the safety warning strategy with the real-time dispatch instructions of the subway platform control system to trigger the corresponding level of emergency response operation.

[0136] Dynamic matching involves comparing and adjusting emergency evacuation routes in real time with the real-time dispatch instructions of the subway platform control system to ensure the feasibility and effectiveness of these routes. The real-time dispatch instructions from the subway platform control system include information such as train operation plans and platform equipment status. For example, if a train is about to arrive at the station, the emergency evacuation route may need to avoid the train's stopping area. Through dynamic matching, the emergency evacuation route can be adjusted according to the real-time situation to adapt to the real-time dispatch instructions. Triggering corresponding emergency response actions involves initiating appropriate emergency measures based on the warning level indicator. For example, when the warning level indicator is high, the subway platform control system can immediately adjust the train operation plan, suspend train arrivals, activate emergency lighting and broadcast announcements, and guide passengers to evacuate according to the emergency evacuation routes. In this way, the subway platform can respond promptly and effectively when safety risks occur.

[0137] As one implementation method, step S540 involves dynamically matching the emergency evacuation path in the safety warning strategy with the real-time dispatch instructions of the subway station platform control system. Specifically, this may include the following steps S541 to S544: Step S541: Determine the emergency response level based on the warning level identifier and call the preset evacuation scheme library in the subway station platform control system.

[0138] The warning level is divided into three levels: low, medium, and high, each corresponding to a different emergency response level. A low warning level may require a mild response, such as increased platform patrols and reminding passengers to be careful. A medium warning level may require a moderate response, such as adjusting train intervals and activating some emergency equipment. A high warning level may require a severe response, such as suspending train operations and completely evacuating passengers. After determining the emergency response level based on the warning level, the system calls upon the preset evacuation plan library in the subway platform control system. This library stores evacuation plans for different emergency response levels, pre-designed based on factors such as subway platform layout and passenger flow. For example, when the emergency response level is severe, the corresponding plan for complete passenger evacuation is invoked.

[0139] Step S542: Match multiple candidate schemes that match the emergency evacuation path in the evacuation scheme library, and filter the schemes based on the resource availability parameters in the real-time dispatch instructions.

[0140] Matching multiple candidate solutions matching the emergency evacuation route in the evacuation plan database requires considering the route's starting point, ending point, and intermediate areas. Solutions that cover the emergency evacuation route are then selected as candidates. For example, if the emergency evacuation route needs to pass through a platform passageway, solutions that include that passageway are selected as candidates. Solution selection is also based on resource availability parameters in real-time dispatch instructions, including train operating status, platform equipment status, and staff distribution. For instance, if a candidate solution requires the use of emergency equipment that is currently malfunctioning, the solution is not suitable and must be excluded. In this way, feasible evacuation solutions that meet real-time conditions are selected.

[0141] Step S543: Perform dynamic simulation verification on the screened candidate schemes and density change trend information to generate an optimized diversion scheme that meets risk control requirements.

[0142] Dynamic simulation verification uses computer simulation technology to run the selected candidate evacuation plans, combining density change trend information to evaluate the effectiveness of each candidate plan. During the simulation, passengers are simulated to evacuate along the evacuation paths of the candidate plans, and the changes in passenger density during the evacuation are observed. Based on the density change trend information, potential risks during the evacuation process are predicted, such as congestion and stampedes. The performance of each candidate plan in risk control is evaluated, such as whether it can quickly evacuate passengers and avoid congestion. Through dynamic simulation verification, the candidate plan with the best performance is selected as the optimized evacuation plan. For example, a multi-agent simulation model can be used to simulate passenger behavior and the evacuation process, verifying the selected candidate plans and generating an optimized evacuation plan that meets risk control requirements.

[0143] Step S544: The optimized evacuation plan is converted into station broadcast instructions and electronic directional sign update instructions through the subway platform control system to realize multi-channel coordinated emergency response operations.

[0144] The optimized evacuation plan is translated into in-station broadcast instructions and electronic signage update instructions to effectively communicate this plan to passengers and staff. In-station broadcast instructions are played through the subway platform's public address system, informing passengers of the emergency situation and evacuation directions. Electronic signage update instructions update the electronic signage within the subway platform, guiding passengers to evacuate along the emergency evacuation routes outlined in the optimized plan. This multi-channel coordination ensures passengers receive timely and accurate information about the emergency situation and evacuation procedures. For example, an announcement on the subway platform might say, "Please follow the electronic signage and quickly evacuate to the safety exits," while simultaneously updating the electronic signage to indicate the emergency evacuation routes. This improves the efficiency of emergency response and safeguards passenger safety.

[0145] In summary, this subway platform screen door safety monitoring method based on multi-source data analysis acquires multi-source data sets from the target subway platform, performs spatiotemporal alignment processing, identifies foreign objects and predicts their density, and ultimately generates a safety early warning strategy and triggers emergency response operations. This method can comprehensively and accurately monitor the safety status of the subway platform screen door area, promptly detect foreign objects and abnormal situations, and provide strong support for the safe operation of subway platforms.

[0146] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention, such as Dijkstra's algorithm, image blur detection algorithm, principal component analysis algorithm, etc., can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solution of the present invention. For example, they can use normalization to eliminate dimensional conflicts before feature fusion, use interpolation to eliminate dimensional differences, reasonably set the threshold based on historical data, experience or business scenario requirements, train the model based on a general model training method, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes here.

[0147] This invention provides a monitoring system, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0148] Figure 2 This is a schematic diagram of the hardware entity of a monitoring system provided in an embodiment of the present invention, such as... Figure 2 As shown, the hardware entity of the monitoring system 1000 includes a processor 1001 and a memory 1002. The memory 1002 stores a computer program that can run on the processor 1001. When the processor 1001 executes the program, it implements the steps of the method in any of the above embodiments. The memory 1002 stores the computer program that can run on the processor. The memory 1002 is configured to store instructions and applications executable by the processor 1001, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1001 and various modules in the monitoring system 1000. This can be implemented using flash memory or random access memory (RAM). When the processor 1001 executes the program, it implements the steps of the subway platform screen door safety monitoring method based on multi-source data analysis described above. The processor 1001 typically controls the overall operation of the monitoring system 1000.

[0149] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for safety monitoring of subway platform screen doors based on multi-source data analysis, characterized in that, include: Acquire a multi-source data set of the target subway platform, the multi-source data set including a visual data sequence of the platform screen door area captured by an image acquisition device and an obstacle distance data sequence collected by an ultrasonic ranging device; The multi-source data set is subjected to spatiotemporal alignment processing to generate a multimodal monitoring data set that integrates spatiotemporal features; Foreign object identification processing is performed based on the multimodal monitoring data set to generate foreign object monitoring results and foreign object location distribution information in the shielded door area; Based on the foreign object monitoring results and the passenger flow data sequence in the multi-source dataset, density prediction processing is performed to generate passenger line density prediction results and density change trend information. A safety warning strategy is generated based on the location distribution information of the foreign object and the density change trend information, and the safety warning strategy is transmitted to the subway platform control system to trigger emergency response operations.

2. The method according to claim 1, characterized in that, The step of performing spatiotemporal alignment processing on the multi-source data set to generate a multimodal monitoring data set fused with spatiotemporal features includes: The visual data sequence of the shielding door area is subjected to blur compensation processing, the blur parameters of each frame image are extracted, and the motion trajectory between adjacent frame images is compensated and corrected based on the blur parameters to generate an enhanced visual data sequence with continuous motion trajectory. Multi-dimensional interference source analysis is performed on the obstacle distance data sequence to decompose the environmental vibration interference component and temperature drift interference component in the obstacle distance data sequence, and a calibration ranging data sequence after interference suppression is generated based on the distribution characteristics of the decomposed interference components. The timestamps of the enhanced visual data sequence and the acquisition timestamps of the calibrated ranging data sequence are bidirectionally synchronized and aligned to generate a spatiotemporal mapping relationship between visual data and ranging data in the multi-source data set; Based on the spatiotemporal mapping relationship, the spatial resolution of the enhanced visual data sequence and the spatial coordinates of the calibrated ranging data sequence are cross-validated to generate a validated visual data sequence and a validated ranging data sequence with spatiotemporal consistency constraints. Multimodal feature cross-fusion is performed on the edge contour features in the verified visual data sequence and the distance fluctuation features in the verified ranging data sequence to generate the multimodal monitoring data set containing the visual-ranging joint feature vector. The visual-ranging joint feature vector adjusts the fusion ratio of the edge contour features and the distance fluctuation features through dynamic weight parameters, and the dynamic weight parameters are adaptively adjusted according to the residual amount of environmental interference in the calibrated ranging data sequence.

3. The method according to claim 2, characterized in that, The foreign object identification processing based on the multimodal monitoring data set, generating foreign object monitoring results and foreign object location distribution information in the shielded door area, includes: The spatial distribution characteristics and temporal fluctuation characteristics of the foreign object movement trajectory of the shielded door area are extracted from the multimodal monitoring data set. A foreign object discrimination model is constructed based on the spatial distribution characteristics and the temporal fluctuation characteristics, and the parameters of the foreign object discrimination model are updated using a preset foreign object feature library. The updated foreign object discrimination model is used to perform multimodal feature fusion analysis on the multimodal monitoring data set to determine the presence status of foreign objects and their spatial coverage within the shielding door area; Based on the presence status of the foreign object and the spatial coverage area, a foreign object monitoring result containing a foreign object type identifier and a set of location coordinates is generated, and the foreign object location distribution information is generated based on the set of location coordinates.

4. The method according to claim 3, characterized in that, The step of performing density prediction processing based on the foreign object monitoring results and the passenger flow data sequence in the multi-source dataset to generate passenger line density prediction results and density change trend information includes: The flow rate characteristics and directional distribution characteristics of passenger flow data sequences are extracted from the multi-source data set, and a correlation mapping relationship is established between the flow rate characteristics and the foreign object location distribution information; Based on a preset density prediction model, the flow rate characteristics and the directional distribution characteristics are analyzed to generate an initial density prediction value for the passenger gathering area. The initial density prediction value is corrected based on the foreign object location distribution information to generate the passenger line density prediction result and its corresponding density change rate. Based on the density change rate, the passenger line density prediction result is extrapolated to generate density change trend information that includes a density fluctuation curve in the time dimension.

5. The method according to claim 4, characterized in that, The safety warning strategy generated based on the foreign object location distribution information and the density change trend information includes: Based on the foreign object location distribution information, the obstruction impact area of ​​the platform screen door area is determined, and the passenger number and delay time distribution parameters within the obstruction impact area are extracted. By combining the time-dimensional density fluctuation curve in the density change trend information, the density risk score and risk diffusion rate of the blockage-affected area are calculated. The safety warning strategy, which includes warning level identifiers and emergency evacuation routes, is generated based on the density risk score and the risk diffusion rate. The emergency evacuation routes in the aforementioned safety early warning strategy are dynamically matched with the real-time dispatch instructions of the subway platform control system to trigger emergency response operations of the corresponding level.

6. The method according to claim 1, characterized in that, The acquisition of the multi-source data set of the target subway station includes: By periodically acquiring visual data from different perspectives through multiple image acquisition devices deployed in the platform screen door area, and by time-synchronizing the visual data from each perspective, a visual data sequence of the platform screen door area is generated. The dynamic distance data between the shielded door and the obstacle is acquired in real time by an ultrasonic ranging device, and the dynamic distance data is encoded based on the spatial coordinates of the ranging sensor to generate the obstacle distance data sequence. Passenger entry and exit record data and real-time video monitoring data are obtained from the subway platform control system, and the flow trajectory is extracted from the entry and exit record data and the video monitoring data to generate the passenger flow data sequence. The visual data sequence of the platform screen door area, the obstacle distance data sequence, and the passenger flow data sequence are processed to standardize the data format, generating the multi-source data set containing unified time and spatial identifiers.

7. The method according to claim 6, characterized in that, The step of performing spatiotemporal alignment processing on the multi-source data set to generate a multimodal monitoring data set fused with spatiotemporal features further includes: Denoising and edge enhancement processing are performed on each frame of the visual data sequence of the shielding door area to eliminate interference from illumination changes and motion blur, thereby generating a high-definition standard visual data sequence. The obstacle distance data sequence is filtered and time-frequency converted to extract the spectral and spatial distribution features of the effective ranging signal, generating a standardized ranging data sequence with environmental noise eliminated. The spatial resolution of the standard visual data sequence is registered with the spatial coordinates of the standardized ranging data sequence to establish a spatial mapping relationship between the visual data and the ranging data. Based on the spatial mapping relationship, the standard visual data sequence and the standardized ranging data sequence are fused to generate the multimodal monitoring data set containing complementary features of multi-source data.

8. The method according to claim 3, characterized in that, The construction of the foreign object discrimination model based on the spatial distribution characteristics and the temporal fluctuation characteristics includes: Spatial texture features and foreign object shape features are extracted from the multimodal monitoring dataset based on convolutional neural networks, and an initial discrimination model containing a multi-layer feature fusion structure is constructed. The initial discrimination model is trained using a pre-defined foreign object sample library, and the model parameters are dynamically optimized using a transfer learning algorithm. The temporal fluctuation features are introduced as time-dimensional constraints during model training to enhance the model's ability to continuously identify the trajectory of foreign objects. Based on the error feedback between the trained model output and real-time monitoring data, the classification threshold of the foreign object discrimination model is dynamically adjusted to generate an optimized foreign object discrimination model.

9. The method according to claim 4, characterized in that, The method analyzes the flow rate characteristics and directional distribution characteristics based on a preset density prediction model to generate initial density prediction values ​​for passenger gathering areas, including: The flow rate characteristics are subjected to time series decomposition to extract the periodic trend component and random disturbance component of passenger flow, and the periodic trend component is spatiotemporally correlated with the directional distribution characteristics to generate an initial density distribution feature set. The multi-layer learning network of the density prediction model is configured based on the initial density distribution feature set. During the model training phase, the random perturbation component is introduced as an input noise parameter to enhance the robustness of the multi-layer learning network to abnormal flow patterns. The influence range characteristics of the obstruction area in the foreign object location distribution information are coupled with the directional distribution characteristics to generate passenger path offset parameters and path adjustment frequency parameters caused by foreign object obstruction. The output of the multilayer learning network is dynamically corrected based on the path offset parameter and the path adjustment frequency parameter to generate a density prediction correction value that incorporates the influence of foreign object blockage. The density prediction correction value is adjusted by the iterative optimization module of the multi-layer learning network, and the network weight parameters are updated in combination with the changing law of the periodic trend component to generate the initial density prediction value containing the foreign object influence factor. The initial density prediction value is compared with the validation set in the historical density data. Based on the comparison result, the feature fusion threshold of the multilayer learning network is calibrated to generate the optimized density prediction model output result.

10. A monitoring system comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 9.

Images