Intelligent rail area safety early warning method and device based on edge computing

Abstract

The application discloses an edge computing-based intelligent rail area safety early warning method and device, relates to the technical field of safety early warning, and comprises the following steps: collecting video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area in real time; constructing a multi-modal data cube with space-time correlation according to the video streams, three-dimensional vibration spectra and micro-environment parameter sets; extracting deep features of each mode in the multi-modal data cube; dynamically weighting the deep features; generating a risk feature tensor with space-time consistency; determining a neighborhood risk factor based on the risk feature tensor; determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor; post-quantum encrypting the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint; generating an alarm message based on the risk fingerprint; embedding the alarm message into a guarantee time window and uploading the alarm message to a central cloud platform so that the central cloud platform triggers an early warning response.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of safety early warning, in particular to a smart rail area safety early warning method and device based on edge computing. BACKGROUND

[0002] With the development of rail transit towards high speed and high density, the safe operation and maintenance of rail areas (including tracks, tunnels, stations, etc.) are facing unprecedented challenges. Existing systems mostly use isolated monitoring methods, but false positives or false negatives often occur, so the existing rail area safety early warning technology cannot meet the safety requirements.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a smart rail area safety early warning method and device based on edge computing, aiming to solve the technical problem of low early warning efficiency caused by data isolation in the safety monitoring of rail areas in the prior art.

[0005] To achieve the above purpose, the present application provides a smart rail area safety early warning method based on edge computing, the method comprising:

[0006] real-time collection of video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area, construction of a spatio-temporally correlated multi-modal data cube according to the video streams, the three-dimensional vibration spectra and the micro-environment parameter sets;

[0007] extraction of deep features of each modality in the multi-modal data cube, dynamic weighting of the deep features, and generation of a risk feature tensor with spatio-temporal consistency;

[0008] determination of a neighborhood risk factor based on the risk feature tensor, determination of a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor;

[0009] post-quantum encryption of the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, and generation of an alarm message based on the risk fingerprint;

[0010] embedding the alarm message into a protection time window and uploading it to a central cloud platform to make the central cloud platform trigger an early warning response.

[0011] In an embodiment, the step of real-time collection of video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area, and construction of a spatio-temporally correlated multi-modal data cube according to the video streams, the three-dimensional vibration spectra and the micro-environment parameter sets comprises:

[0012] aligning clock information of all sensors within the coverage area, and establishing a global timestamp according to the clock information, mapping position information of all sensors to a three-dimensional spatial coordinate system, and generating a spatio-temporal unified network according to the global timestamp and the three-dimensional spatial coordinate system;

[0013] performing dynamic background modeling and foreground segmentation on the video stream of the coverage area, determining a moving target within the coverage area, and determining a spatio-temporal dynamic feature of the moving target;

[0014] performing wavelet packet decomposition on a three-dimensional vibration spectrum of the coverage area, extracting an impact energy feature, and constructing an abnormal pattern recognition matrix based on the impact energy feature;

[0015] fusing a micro-environment parameter set through a Kalman filter to generate an environment risk index;

[0016] in the spatio-temporal unified network, constructing a spatio-temporally correlated multi-modal data cube according to the spatio-temporal dynamic feature, the abnormal pattern recognition matrix, and the environment risk index.

[0017] In an embodiment, the step of performing dynamic background modeling and foreground segmentation on the video stream of the coverage area, determining a moving target within the coverage area, and determining a spatio-temporal dynamic feature of the moving target comprises:

[0018] determining an illumination intensity in the video stream of the coverage area, generating a dynamic light intensity weight according to the illumination intensity, and creating a dynamic background based on the dynamic light intensity weight;

[0019] performing a morphological top-hat transformation on the video stream, determining a contrast of a small target in the video stream, determining a foreground mask according to the contrast, performing foreground segmentation based on the foreground mask, and determining a foreground target;

[0020] projecting the foreground target to the dynamic background to determine a moving target within the coverage area;

[0021] calculating a minimum circumscribed rectangle of the moving target based on the foreground mask, extracting a feature vector of the minimum circumscribed rectangle, and generating a spatio-temporal dynamic feature according to the feature vector.

[0022] In an embodiment, the step of performing wavelet packet decomposition on a three-dimensional vibration spectrum of the coverage area, extracting an impact energy feature, and constructing an abnormal pattern recognition matrix based on the impact energy feature comprises:

[0023] performing adaptive noise complete ensemble empirical mode decomposition on the three-dimensional vibration spectrum of the coverage area to generate an enhanced signal;

[0024] performing wavelet packet decomposition on the enhanced signal to obtain a plurality of sub-frequency bands;

[0025] An energy entropy of each of the sub-bands is calculated, and a sub-band with a preset frequency in the energy entropy is marked as a shock-sensitive frequency band;

[0026] An energy mutation rate of the shock-sensitive frequency band is determined, and when the energy mutation rate is greater than an abnormal mutation rate, the shock-sensitive frequency band is marked as an abnormal mode, and a shock energy feature of the shock-sensitive frequency band is extracted;

[0027] A time feature, a frequency domain feature, and a spatial feature of the shock energy feature are determined, and an abnormal mode recognition matrix is generated according to the time feature, the frequency domain feature, and the spatial feature.

[0028] In an embodiment, the step of fusing the micro-environment parameter set through the Kalman filter to generate the environment risk index comprises:

[0029] The micro-environment parameter set is interpolated and compensated to obtain a micro-environment parameter pre-processing set;

[0030] A state vector is determined according to the micro-environment parameter pre-processing set, and the state vector is fused through an observation equation of the Kalman filter to obtain a micro-environment filtering parameter;

[0031] An environment risk index is generated according to an environment risk weight factor and the micro-environment filtering parameter.

[0032] In an embodiment, the step of extracting a deep feature of each modality in the multi-modal data cube, dynamically weighting the deep feature, and generating a risk feature tensor with spatio-temporal consistency comprises:

[0033] The spatial feature of the video stream and the spatio-temporal feature of the vibration spectrum in the multi-modal data cube are extracted through a double-path convolution;

[0034] The spatial feature and the spatio-temporal feature are aligned to obtain a spatio-temporally aligned deep feature;

[0035] An environment risk index is determined, and an attention weight is generated according to the environment risk index;

[0036] The deep feature is dynamically weighted based on the attention weight to generate a risk feature tensor with spatio-temporal consistency.

[0037] In an embodiment, the step of determining a neighborhood risk factor based on the risk feature tensor, determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor comprises:

[0038] A physical connection relationship of each device in the coverage area is determined, and a device topology relationship is determined based on the physical connection relationship;

[0039] determine a neighborhood relationship of each device based on the device topology relationship;

[0040] spatiotemporal convolution is performed on the neighborhood relationship to aggregate neighborhood risk information of a target hop number;

[0041] a neighborhood risk factor and a risk gradient are determined according to the neighborhood risk information;

[0042] a risk positioning coordinate is determined according to the risk gradient, and a dynamic risk level is determined according to the neighborhood risk factor and the risk positioning coordinate.

[0043] In an embodiment, the post-quantum encryption of the risk positioning coordinate and the dynamic risk level to obtain risk fingerprints, and the generation of an alarm message based on the risk fingerprints comprise:

[0044] post-quantum encryption is performed on the risk positioning coordinate and the dynamic risk level to obtain encrypted information;

[0045] hash operation is performed on the encrypted information to obtain risk fingerprints;

[0046] the risk fingerprints and the encrypted information are encapsulated to obtain alarm information.

[0047] In an embodiment, the embedding of the alarm message into a guarantee time window and uploading to a central cloud platform comprises:

[0048] a priority queue is determined based on the dynamic risk level and a time sequence;

[0049] based on the priority queue, the alarm messages in the priority queue are sequentially embedded into a guarantee time window from the head to the tail to obtain early warning information;

[0050] the early warning information is uploaded to the central cloud platform.

[0051] In addition, to achieve the above-mentioned purpose, the application further provides a smart rail area safety early warning device based on edge computing, which comprises:

[0052] a data acquisition module, configured to acquire video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area in real time, and construct a spatiotemporally correlated multi-modal data cube according to the video streams, the three-dimensional vibration spectra and the micro-environment parameter sets;

[0053] a feature processing module, configured to extract deep features of each modality in the multi-modal data cube, dynamically weight the deep features, and generate a risk feature tensor with spatiotemporal consistency;

[0054]

[0054] a risk rating module configured to determine a neighborhood risk factor based on the risk feature tensor, determine a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor;

[0055] an information encryption module configured to post-quantum encrypt the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, and generate an alarm message based on the risk fingerprint;

[0056] a safety warning module configured to upload the alarm message embedded in a guarantee time window to a central cloud platform, so that the central cloud platform triggers a warning response.

[0057] In addition, to achieve the above-mentioned purpose, the present application also provides a safety warning device for a smart rail area based on edge computing, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the safety warning method for a smart rail area based on edge computing as described above.

[0058] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the safety warning method for a smart rail area based on edge computing as described above.

[0059] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the safety warning method for a smart rail area based on edge computing as described above.

[0060] The present application provides a safety warning method for a smart rail area based on edge computing, which comprises the following steps: collecting video stream, three-dimensional vibration spectrum and micro-environment parameter set in a covered area in real time, constructing a multi-modal data cube with space-time correlation according to the video stream, three-dimensional vibration spectrum and micro-environment parameter set, extracting deep features of each mode in the multi-modal data cube, dynamically weighting the deep features, generating a risk feature tensor with space-time consistency, determining a neighborhood risk factor based on the risk feature tensor, determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor, post-quantum encrypting the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, generating an alarm message based on the risk fingerprint, embedding the alarm message in a guarantee time window and uploading it to a central cloud platform, so that the central cloud platform triggers a warning response. The present application realizes accurate, real-time and safe perception and response to risks. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating an embodiment of the smart rail transit area safety early warning method based on edge computing in this application.

[0064] Figure 2 This is a schematic diagram of the data processing flow of an embodiment of the smart rail transit area safety early warning method based on edge computing in this application;

[0065] Figure 3 This is a schematic diagram of the module structure of the smart rail transit area safety early warning device based on edge computing, as described in an embodiment of this application.

[0066] Figure 4 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the smart rail transit area safety early warning method based on edge computing in the embodiments of this application.

[0067] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0069] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0070] The main solution of this application embodiment is: to collect video streams, three-dimensional vibration spectra and microenvironment parameter sets in real time within the coverage area, and to construct a spatiotemporally correlated multimodal data cube based on the video streams, the three-dimensional vibration spectra and the microenvironment parameter sets;

[0071] The depth features of each modality in the multimodal data cube are extracted, and the depth features are dynamically weighted to generate a risk feature tensor with spatiotemporal consistency.

[0072] Based on the risk feature tensor, a neighborhood risk factor is determined, and based on the neighborhood risk factor, the risk location coordinates and dynamic risk level are determined.

[0073] The risk location coordinates and the dynamic risk level are subjected to post-quantum encryption to obtain a risk fingerprint, and an alarm message is generated based on the risk fingerprint.

[0074] The alarm message is embedded in a guarantee time window and uploaded to the central cloud platform, so that the central cloud platform triggers an early warning response.

[0075] At present, with the development of rail transit towards high speed and high density, the safe operation and maintenance of the rail area (including tracks, tunnels, stations, etc.) is facing unprecedented challenges. The existing system mostly uses isolated monitoring methods, and false positives or false negatives often occur, so the existing rail area safety warning technology cannot meet the safety requirements.

[0076] The present application provides a solution, by collecting video streams, three-dimensional vibration spectra and micro-environment parameter sets in the covered area in real time, constructing a spatio-temporal correlated multi-modal data cube according to the video streams, three-dimensional vibration spectra and micro-environment parameter sets, extracting the deep features of each modality in the multi-modal data cube, dynamically weighting the deep features, generating a risk feature tensor with spatio-temporal consistency, determining a neighborhood risk factor based on the risk feature tensor, determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor, post-quantum encrypting the risk positioning coordinate and the dynamic risk level, obtaining a risk fingerprint, generating an alarm message based on the risk fingerprint, embedding the alarm message in a guarantee time window and uploading it to a central cloud platform, so that the central cloud platform triggers an early warning response. The precise, real-time and safe perception and response to risks are realized.

[0077] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a smart rail area safety warning device based on edge computing, etc., and the present embodiment does not make specific limitations thereon. The following takes the smart rail area safety warning device based on edge computing as an example to describe the present embodiment and the following embodiments.

[0078] All actions of obtaining signals, information or data in the present application are carried out in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

[0079] The present application provides a smart rail area safety warning method based on edge computing, which is described with reference to Figure 1 , Figure 1 The present application provides a smart rail area safety warning method based on edge computing, which is described with reference to

[0080] In the present embodiment, the smart rail area safety warning method based on edge computing includes steps S10-S50:

[0081] Step S10, real-time collection of video stream, three-dimensional vibration spectrum and micro-environment parameter set in the coverage area, construction of a multi-modal data cube with time and space correlation according to the video stream, the three-dimensional vibration spectrum and the micro-environment parameter set;

[0082] It should be noted that the physical area in which the sensor, camera and other monitoring equipment are deployed in the intelligent rail area usually includes the track, tunnel, platform, turnout and other key sections. The area that can be monitored by the monitoring equipment is the coverage area. The video stream, three-dimensional vibration spectrum and micro-environment parameter set are different forms of data collected by different monitoring equipment. The video stream refers to a continuous image sequence collected in real time by a high-definition camera, which is used to monitor visual information such as personnel activities, equipment status and foreign object intrusion in the rail area. The three-dimensional vibration spectrum refers to the frequency spectrum features obtained by Fourier transform of the vibration signals collected by a vibration sensor in three orthogonal directions, which is used to analyze the type, intensity and position of the vibration source. The micro-environment parameter set includes temperature, humidity, air pressure, light intensity, wind speed and other environmental monitoring data, which is used to evaluate the environmental status of the rail area and its impact on the safety of the equipment. The multi-modal data cube with time and space correlation is a data structure that aligns and fuses data from different sources and different modalities in the time and space dimensions.

[0083] It can be understood that the video stream is captured in real time by a high-definition camera deployed in the rail area, and is preliminarily compressed and denoised by an edge node; the three-dimensional vibration spectrum is collected by a vibration sensor, and is filtered, Fourier transformed and frequency domain features are extracted by an edge computing node; the micro-environment parameters are collected by various environmental sensors, and are calibrated and normalized by an edge node. After the data is collected and preprocessed, the multi-modal data can be fused to construct a multi-modal data cube. At this time, time synchronization, spatial alignment and data fusion need to be performed respectively to obtain the data cube. The data cube can be defined as: .

[0084] wherein, is the data cube, is the time dimension, representing the timestamp sequence of data collection, is the space dimension, representing the three-dimensional coordinates in the rail area, is the modal dimension, representing different data types, respectively represents the length of each dimension.

[0085] In a feasible implementation, the step of real-time collection of video stream, three-dimensional vibration spectrum and micro-environment parameter set in the coverage area, construction of a multi-modal data cube with time and space correlation according to the video stream, the three-dimensional vibration spectrum and the micro-environment parameter set includes:

[0086] aligning clock information of all sensors in the coverage area, and establishing a global timestamp according to the clock information, mapping position information of all sensors to a three-dimensional spatial coordinate system, and generating a space-time unified network according to the global timestamp and the three-dimensional spatial coordinate system;

[0087] performing dynamic background modeling and foreground segmentation on a video stream of the coverage area, determining a moving target in the area, and determining a space-time dynamic feature of the moving target;

[0088] performing wavelet packet decomposition on a three-dimensional vibration spectrum of the coverage area, extracting an impact energy feature, and constructing an abnormal pattern recognition matrix based on the impact energy feature;

[0089] fusing a micro-environment parameter set through a Kalman filter to generate an environment risk index;

[0090] in the space-time unified network, constructing a multi-modal data cube associated with space-time according to the space-time dynamic feature, the abnormal pattern recognition matrix, and the environment risk index.

[0091] It should be noted that the clock information refers to the local time information recorded by the internal clock of each sensor. Due to hardware differences or network delays, there is a problem of time asynchronization. The global timestamp is a unified time reference system generated by aligning the clocks of all sensors, which can ensure that all data are consistent in the time dimension. The three-dimensional spatial coordinate system is a coordinate system established according to the actual geographical space of the track area, which is used to uniformly represent the physical positions of sensors and targets. The space-time unified network is a structured data network that associates time and space, which is used to integrate multi-source sensor data.

[0092] It can be understood that the space-time dynamic feature is the behavior feature of the moving target extracted from the video in time and space, such as motion trajectory, speed, acceleration, direction, etc. The abnormal pattern recognition matrix is a matrix constructed by feature extraction on vibration signals, which is used to identify abnormal vibration patterns such as impact, fracture, collision, etc. The environment risk index is a quantitative index calculated by fusing micro-environment parameters, which is used to evaluate the degree of influence of the environment on the safety of the track.

[0093] In specific implementation, refer to Figure 2 , Figure 2 for a data processing flow diagram. When time synchronization is performed, NTP (Network Time Protocol) or PTP (Precision Time Protocol) is used to align the time of all sensors, and a global timestamp t gThen the positions of all sensors are mapped into a three-dimensional spatial coordinate system, and the position of each sensor can be mapped into a unified three-dimensional coordinate system through laser scanning or visual calibration, and the sensor position is recorded as Then a space-time unified network is generated according to the global timestamp and the three-dimensional spatial coordinate system, and the space-time unified network can be represented as a graph structure Wherein the nodes are sensors or spatial grids, and the edges represent the space-time correlation relationship. Then, dynamic background modeling and foreground segmentation are performed on the video stream in the coverage area. When performing dynamic background modeling, Gaussian mixture model (GMM) can be used to model the video background, and the model can be represented as:

[0094]

[0095] Wherein, I t is the current video frame, K is the number of Gaussian components, N(·) is the Gaussian distribution, ω k,t is the weight, μ k,t is the mean, and Σ k,t is the covariance.

[0096] When performing foreground segmentation, moving targets can be extracted by background subtraction to generate a binary mask to determine the moving targets in the region. Then the space-time dynamic characteristics of each moving target are determined, the trajectory of the moving target is calculated, and the velocity and acceleration are extracted.

[0097] When performing wavelet packet decomposition on the three-dimensional vibration spectrum of the coverage area, wavelet packet transform is performed on the three-dimensional vibration signal s(t)=[s x (t),s y (t),s z (t)] to extract the frequency band energy feature E j,k :

[0098]

[0099] Wherein, W j,k is the wavelet transform coefficient, j is the decomposition level, and k is the frequency band index.

[0100] The high frequency band feature is extracted in the frequency band energy feature to obtain an impact energy feature. Then, an abnormal pattern recognition matrix A is constructed according to the impact energy feature. Then, the micro-environment parameter set is interpolated and compensated to obtain a micro-environment parameter pre-processing set, a state vector is determined according to the micro-environment parameter pre-processing set, the state vector is fused by using an observation equation of the Kalman filter to obtain a micro-environment filtering parameter, and an environmental risk index is generated according to an environmental risk weight factor and the micro-environment filtering parameter.

[0101] The micro-environment parameter set is fused by using the Kalman filter, and the temperature T, humidity H and wind speed W in the micro-environment parameter set are state estimated to obtain filtered environmental parameters .

[0102]

[0103]

[0104] wherein, is a state vector, is an observation vector, and are state transition and observation matrices, and are noises.

[0105] After the environmental parameters are fused, the environmental risk index R is calculated env , and the calculation formula of the environmental risk index is: .

[0106] wherein, is a weight coefficient.

[0107] According to the spatio-temporal dynamic feature, the abnormal pattern recognition matrix and the environmental risk index, a spatio-temporal correlation multi-modal data cube is constructed.

[0108] In a feasible implementation, the step of dynamically modeling a background and segmenting a foreground of a video stream covering a region, determining a moving target in the region, and determining a spatio-temporal dynamic feature of the moving target comprises:

[0109] Determining an illumination intensity in the video stream covering the region, generating a dynamic light intensity weight according to the illumination intensity, and creating a dynamic background based on the dynamic light intensity weight;

[0110] Performing a morphological top-hat transformation on the video stream, determining a contrast of a small target in the video stream, determining a foreground mask according to the contrast, performing foreground segmentation based on the foreground mask, and determining a foreground target;

[0111] Projecting the foreground target to the dynamic background, and determining a moving target in the region.

[0112] Calculate the minimum bounding rectangle of the moving target based on the foreground mask, extract a feature vector of the minimum bounding rectangle, and generate a spatio-temporal dynamic feature based on the feature vector.

[0113] It should be noted that the dynamic light intensity weight is a weight coefficient calculated according to the real-time light intensity in the video stream, which is used to adjust the adaptability of the background model to light changes and ensure the stability of background modeling under different light conditions. The dynamic background is a background model that can adaptively update over time, effectively dealing with changes in light, shadow interference and other scene changes by fusing historical frame information and current light conditions. The foreground mask is a binary image, where the pixel points with a value of 1 represent the moving target, and the pixel points with a value of 0 represent the background. The foreground target is a moving object segmented from the video stream, such as personnel, vehicles, foreign objects, etc., which is the object that needs to be monitored and analyzed.

[0114] In a specific implementation, the average gray value or luminance component of the current video frame is calculated to obtain the light intensity The dynamic light intensity weight is generated according to the light intensity

[0115]

[0116] wherein is the sliding average value of the recent light intensity, is an adjustment parameter.

[0117] Then a dynamic background is created according to the dynamic light intensity weight , and the equation is constructed as: .

[0118] And the video stream is subjected to morphological top-hat transformation to enhance the contrast of small targets and background, determine the contrast of small targets in the video stream, determine the foreground mask according to the contrast, and the mask generation formula is:

[0119]

[0120] wherein T contrast is the contrast threshold.

[0121] The mask is subjected to connected region analysis, and the continuous foreground region is extracted as a candidate moving target. The foreground target is projected onto the dynamic background , and the coincidence degree is calculated. If the coincidence degree is low, it is confirmed as a real moving target. The minimum bounding rectangle of each moving target is calculated to obtain parameters such as position, width, height and angle. Then the feature vector F is extracted: . ​​

[0122] wherein, is a target center coordinate, is a rectangular width and height, is a rectangular rotation angle, is a target speed in x and y directions, is a target acceleration in x and y directions.

[0123] In an implementable embodiment, the step of performing wavelet packet decomposition on the three-dimensional vibration spectrum of the coverage area, extracting impact energy features, and constructing an anomaly pattern recognition matrix based on the impact energy features comprises:

[0124] performing adaptive noise complete ensemble empirical mode decomposition on the three-dimensional vibration spectrum of the coverage area to generate an enhanced signal;

[0125] performing wavelet packet decomposition on the enhanced signal to obtain a plurality of sub-bands;

[0126] calculating energy entropy of each of the sub-bands, and marking a sub-band with a preset frequency in the energy entropy as an impact sensitive band;

[0127] determining an energy mutation rate of the impact sensitive band, marking the impact sensitive band as an anomaly pattern when the energy mutation rate is greater than an anomaly mutation rate, and extracting impact energy features of the impact sensitive band;

[0128] determining time features, frequency domain features, and spatial features of the impact energy features, and generating an anomaly pattern recognition matrix according to the time features, the frequency domain features, and the spatial features.

[0129] It should be noted that the enhanced signal refers to a vibration signal processed by adaptive noise complete ensemble empirical mode decomposition, which effectively separates the noise components in the original vibration data. Wavelet packet decomposition divides the signal into a plurality of sub-bands at multiple scales, and each sub-band corresponds to the component of the signal in a specific frequency range. The impact sensitive band is a frequency band that contains main impact energy and has high energy entropy, and is a key frequency band for impact feature extraction. The energy mutation rate is an impact sensitive band whose energy mutation rate exceeds a threshold. The impact energy features refer to feature quantities extracted from the impact sensitive band that can represent impact events.

[0130] In a specific implementation, adaptive noise complete ensemble empirical mode decomposition is used to enhance the three-dimensional vibration spectrum signal:

[0131]

[0132] wherein, is an original vibration signal, IMF i (t) is an i-th intrinsic mode function, rn (t) is a residual component.

[0133] The enhanced signal is reconstructed from the first k main IMF components:

[0134]

[0135] where, x enhanced (t) is an enhanced signal.

[0136] Then the enhanced signal is decomposed by L-layer wavelet packet to obtain sub-bands: .

[0137] where, is a wavelet packet basis function, is a decomposition scale (j=1, 2, …, L), and k is a frequency band index (k=0, 1, …, -1).

[0138] The energy entropy of each sub-band is calculated:

[0139]

[0140]

[0141]

[0142] where, E k is the energy of the kth sub-band, P k is the energy proportion of the kth sub-band, H k is the energy entropy of the kth sub-band.

[0143] The energy entropy H k of each sub-band is compared with a threshold H th If the energy entropy is greater than the threshold, the sub-band is marked as an impact-sensitive frequency band.

[0144] The energy mutation rate of the impact-sensitive frequency band is calculated :

[0145]

[0146] When , the frequency band is marked as an abnormal pattern, is an abnormal mutation rate threshold determined according to historical data and experiments. Then three types of features of the impact energy features are extracted: time features , frequency domain features , and spatial features . An abnormal pattern recognition matrix B is constructed:

[0147]

[0148] extracting deep features of each modality in the multi-modal data cube, dynamically weighting the deep features, and generating a risk feature tensor with spatio-temporal consistency;

[0149] It should be noted that the deep features refer to high-level abstract feature representations extracted from the original multi-modal data by a deep learning network. The risk feature tensor is a multi-dimensional data structure generated by dynamically weighting and fusing the deep features of each modality, which is used to comprehensively represent the safety risk state of the track area.

[0150] It can be understood that the deep features of each modality in the multi-modal data cube are extracted, for the visual feature branch, 3DCNN is used to extract spatio-temporal features, for the vibration feature branch, time series CNN+LSTM network is used to process vibration data, and fully connected deep network is used to process environmental feature branch. Then, the attention mechanism is used for dynamic calculation to generate a risk feature tensor with spatio-temporal consistency.

[0151] In one possible implementation, the step of extracting deep features of each modality in the multi-modal data cube, dynamically weighting the deep features, and generating a risk feature tensor with spatio-temporal consistency includes:

[0152] extracting spatial features of the video stream and spatio-temporal features of the vibration spectrum in the multi-modal data cube by double-path convolution;

[0153] aligning the spatial features and the spatio-temporal features to obtain spatio-temporally aligned deep features;

[0154] determining an environmental risk index, and generating an attention weight according to the environmental risk index;

[0155] dynamically weighting the deep features based on the attention weight to generate a risk feature tensor with spatio-temporal consistency.

[0156] It should be noted that the spatial features of the video stream refer to features extracted from the video data representing the distribution of visual content in two-dimensional space, mainly describing the shape, texture, edge, and other static appearance attributes of the target. The spatio-temporal features of the vibration spectrum refer to features that simultaneously capture the time evolution law and frequency distribution characteristics from the vibration signal, which contain both the dynamic change information of the vibration over time and the distribution characteristics in the frequency domain. The attention weight is used to measure the importance and reliability of different modal features under the current environmental conditions.

[0157] In a specific implementation, the spatial features of the video stream and the spatio-temporal features of the vibration spectrum in the multi-modal data cube are extracted by double-path convolution. The video stream spatial feature extraction path uses 2D CNN to extract spatial features:

[0158]

[0159] wherein, denotes a two-dimensional convolution operation, , are weights and biases for spatial convolution, is video data in a data cube.

[0160] The vibration spectrum spatio-temporal feature extraction path uses a 3D CNN to extract spatio-temporal features: .

[0161] wherein, denotes a three-dimensional convolution operation, is vibration data in a data cube, and are weights and biases for spatio-temporal convolution.

[0162] The spatial features and the spatio-temporal features are then aligned in features to obtain spatio-temporally aligned deep features.

[0163] At the same time, an environmental risk index is determined, and an attention weight is calculated based on the environmental risk index: .

[0164] wherein, is the attention weight, is an activation function, , is a learnable parameter.

[0165] After obtaining the attention weight, the deep features can be dynamically weighted with the attention weight to generate a risk feature tensor with spatio-temporal consistency.

[0166] Step S30, determining a neighborhood risk factor based on the risk feature tensor, determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor;

[0167] It should be noted that the neighborhood risk factor refers to a risk propagation indicator calculated based on a feature statistic in a neighborhood around a target position in the risk feature tensor, and is used to quantify the diffusion degree and influence range of the risk in space. The accurate position coordinate of the risk source is determined by analyzing the spatial distribution of the neighborhood risk factor.

[0168] It can be understood that the neighborhood risk factor of each spatial position (x, y, z) is calculated based on the risk feature tensor , including a local risk density, a risk gradient feature, and a neighborhood risk factor. The calculation formula of the local risk density is:

[0169]

[0170] The formula for calculating the risk gradient characteristic is:

[0171]

[0172] The formula for calculating the neighborhood risk factor is:

[0173]

[0174] in, The neighborhood radius, The number of points in the neighborhood. To adjust the parameters.

[0175] Then, the location of the risk source is determined by finding the extreme points of neighboring risk factors. This requires maximum value detection and sub-pixel precise positioning to determine the final location coordinates. The specific implementation process is as follows:

[0176] The formula for calculating maximum detection is:

[0177]

[0178] The formula for calculating sub-pixel precise positioning is:

[0179]

[0180] Determine the final positioning coordinates The calculation formula is: .

[0181] When conducting dynamic risk level assessment, risk levels are classified based on the statistical characteristics of neighboring risk factors. First, the risk intensity index and the degree of risk diffusion can be calculated and determined. The dynamic risk level is then determined based on these indicators. The specific calculation formula is as follows:

[0182] Risk intensity index The calculation formula is:

[0183]

[0184] Risk diffusion level The calculation formula is:

[0185]

[0186] Dynamic risk level The calculation formula is:

[0187]

[0188] in, These are the weighting coefficients.

[0189] In an implementable embodiment, the step of determining the neighborhood risk factor based on the risk feature tensor, determining the risk positioning coordinate and the dynamic risk level based on the neighborhood risk factor comprises:

[0190] determining the physical connection relationship of each device in the coverage area, and determining the device topology relationship based on the physical connection relationship;

[0191] determining the neighborhood relationship of each device based on the device topology relationship;

[0192] spatiotemporal convolution is performed on the neighborhood relationship, and the neighborhood risk information of the target hop number is aggregated;

[0193] determining the neighborhood risk factor and the risk gradient according to the neighborhood risk information;

[0194] determining the risk positioning coordinate according to the risk gradient, and determining the dynamic risk level according to the neighborhood risk factor and the risk positioning coordinate.

[0195] It should be noted that the device topology relationship refers to the network structure formed by the physical connection relationship and the spatial position relationship between each monitoring device (sensor, camera, etc.) in the rail area, which is usually represented by a graph structure, wherein the node represents the device, and the edge represents the connection relationship between the devices. The neighborhood relationship is a set of adjacent devices of each device within a certain hop number range determined based on the device topology relationship. The neighborhood risk information is a comprehensive risk index obtained by aggregating the risk features of the target device and its neighborhood devices. The risk factor is a numerical index quantifying the risk degree of the device or the area. The risk gradient is a vector field representing the rate and direction of change of risk in space.

[0196] In a specific implementation, first, a device topology network is constructed according to the physical connection relationship of each sensor in the coverage area, the spatial correlation between devices is determined, and then the neighborhood range of each device is defined based on the topology relationship. The risk feature information of the neighborhood devices within a certain hop number is aggregated through a spatiotemporal convolution operation to form neighborhood risk data reflecting the risk propagation characteristics. The neighborhood risk factor representing the local risk intensity and the risk gradient vector indicating the risk change direction are calculated according to these aggregated data. Finally, the precise spatial coordinates of the risk source are determined by reverse tracking using the risk gradient field, and the risk level index quantitatively evaluating the risk severity is dynamically generated by comprehensively considering the neighborhood risk factor value and the risk source positioning result.

[0197] In step S40, the risk positioning coordinate and the dynamic risk level are post-quantum encrypted to obtain a risk fingerprint, and an alarm message is generated based on the risk fingerprint.

[0198] It should be noted that the risk fingerprint refers to a unique digital identifier generated by performing post-quantum encryption and hash operation on the risk positioning coordinates and the dynamic risk level. The alarm message contains a structured data packet of risk fingerprint and encrypted risk information, which is used to transmit risk warning information in the system. The encrypted information refers to the ciphertext data obtained by encrypting the original risk data using a post-quantum cryptographic algorithm.

[0199] It can be understood that the specific process of generating an alarm message based on the risk fingerprint generated by performing post-quantum encryption on the risk positioning coordinates and the dynamic risk level can be described as follows: performing post-quantum encryption on the risk positioning coordinates and the dynamic risk level to obtain encrypted information; performing hash operation on the encrypted information to obtain a risk fingerprint; and encapsulating the risk fingerprint and the encrypted information to obtain alarm information.

[0200] In a specific implementation, the risk positioning coordinates and the dynamic risk level are encrypted using a post-quantum encryption algorithm to generate ciphertext data with anti-quantum computing attack characteristics. Then, the digital fingerprint of the ciphertext data is extracted through hash operation to form a unique risk identifier. Finally, the encrypted data and the risk fingerprint are encapsulated with metadata such as a timestamp to generate a standardized alarm message, ensuring the confidentiality, integrity, and verifiability of the risk information during transmission.

[0201] Step S50, embedding the alarm message in a protection time window and uploading it to a central cloud platform, so that the central cloud platform triggers a pre-warning response.

[0202] It should be noted that the protection time window refers to the maximum delay time threshold allowed for data transmission. The central cloud platform refers to a remote server cluster with large-scale data storage and processing capabilities.

[0203] It can be understood that the generated alarm message is first bound to a preset protection time window to ensure that the alarm information is transmitted within a specified time limit; then the message with the bound time window is uploaded to the central cloud platform through a secure communication protocol; the central cloud platform verifies the validity of the time window after receiving the message, analyzes the encrypted risk information, and finally automatically triggers the corresponding pre-warning response mechanism according to the analysis result, realizing the cloud-based collaborative disposal of risk events.

[0204] In a feasible implementation, the step of embedding the alarm message in a protection time window and uploading it to a central cloud platform includes:

[0205] determining a priority queue based on the dynamic risk level and the time sequence;

[0206] based on the priority queue, sequentially embedding the alarm messages in the priority queue into the protection time window from the head to the tail to obtain pre-warning information.

[0207] uploading the early warning information to a central cloud platform.

[0208] In a specific implementation, first, according to the dynamic risk level contained in the alarm message and the time sequence of its generation, a plurality of alarms are sorted to form a priority queue; then, according to the queue order, a corresponding guarantee time window is embedded for each alarm message in turn to form early warning information with time constraint; finally, the packaged early warning information is uploaded to the central cloud platform through the network, and the platform receives and analyzes the information within the time window and triggers the corresponding early warning response mechanism.

[0209] The embodiment provides a smart rail area safety early warning method based on edge computing, video stream, three-dimensional vibration spectrum and micro-environment parameter set in a covered area are collected in real time, a multi-modal data cube with space-time correlation is constructed according to the video stream, three-dimensional vibration spectrum and micro-environment parameter set, deep features of each mode in the multi-modal data cube are extracted, the deep features are dynamically weighted, a risk feature tensor with space-time consistency is generated, a neighborhood risk factor is determined based on the risk feature tensor, a risk positioning coordinate and a dynamic risk level are determined based on the neighborhood risk factor, the risk positioning coordinate and the dynamic risk level are post-quantum encrypted to obtain a risk fingerprint, an alarm message is generated based on the risk fingerprint, the alarm message is embedded in a guarantee time window and uploaded to a central cloud platform, so that the central cloud platform triggers an early warning response. The precise, real-time and safe perception and response of the risk are realized.

[0210] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the smart rail area safety early warning method based on edge computing of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0211] The present application also provides a smart rail area safety early warning device based on edge computing, please refer to Figure 3 The smart rail area safety early warning device based on edge computing comprises:

[0212] A data acquisition module 10 is configured to collect video stream, three-dimensional vibration spectrum and micro-environment parameter set in a covered area in real time, and construct a multi-modal data cube with space-time correlation according to the video stream, the three-dimensional vibration spectrum and the micro-environment parameter set;

[0213] A feature processing module 20 is configured to extract deep features of each mode in the multi-modal data cube, dynamically weight the deep features, and generate a risk feature tensor with space-time consistency.

[0214] The risk rating module 30 is configured to determine a neighborhood risk factor based on the risk feature tensor, determine a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor;

[0215] The information encryption module 40 is configured to perform post-quantum encryption on the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, and generate an alarm message based on the risk fingerprint;

[0216] The security warning module 50 is configured to upload the alarm message embedded in a protection time window to a central cloud platform, so that the central cloud platform triggers a warning response.

[0217] In an embodiment, the data acquisition module 10 is further configured to align clock information of all sensors in the coverage area, establish a global timestamp based on the clock information, map position information of all sensors to a three-dimensional coordinate system, and generate a space-time unified network based on the global timestamp and the three-dimensional coordinate system;

[0218] The video stream of the coverage area is dynamically background modeled and foreground segmented to determine a moving target in the area and determine a space-time dynamic feature of the moving target;

[0219] The three-dimensional vibration spectrum of the coverage area is wavelet packet decomposed to extract an impact energy feature, and an abnormal pattern recognition matrix is constructed based on the impact energy feature;

[0220] The micro-environment parameter set is fused by a Kalman filter to generate an environmental risk index;

[0221] In the space-time unified network, a space-time correlated multi-modal data cube is constructed based on the space-time dynamic feature, the abnormal pattern recognition matrix, and the environmental risk index.

[0222] In an embodiment, the data acquisition module 10 is further configured to determine an illumination intensity in the video stream of the coverage area, generate a dynamic light intensity weight based on the illumination intensity, and create a dynamic background based on the dynamic light intensity weight;

[0223] The video stream is morphologically top-hat transformed to determine a contrast of small targets in the video stream, a foreground mask is determined based on the contrast, foreground segmentation is performed based on the foreground mask, and a foreground target is determined;

[0224] The foreground target is projected to the dynamic background to determine a moving target in the area;

[0225] A minimum circumscribed rectangle of the moving target is calculated based on the foreground mask, a feature vector of the minimum circumscribed rectangle is extracted, and a space-time dynamic feature is generated based on the feature vector.

[0226] In an implementable embodiment, the data acquisition module 10 is further configured to perform adaptive noise complete ensemble empirical mode decomposition on the three-dimensional vibration spectrum of the coverage area to generate an enhanced signal;

[0227] perform wavelet packet decomposition on the enhanced signal to obtain a plurality of sub-bands;

[0228] calculate the energy entropy of each of the sub-bands, and mark the sub-band with a preset frequency in the energy entropy as an impact-sensitive frequency band;

[0229] determine the energy mutation rate of the impact-sensitive frequency band, mark the impact-sensitive frequency band as an abnormal pattern when the energy mutation rate is greater than an abnormal mutation rate, and extract an impact energy feature of the impact-sensitive frequency band;

[0230] determine the time feature, frequency domain feature and spatial feature of the impact energy feature, and generate an abnormal pattern recognition matrix according to the time feature, frequency domain feature and spatial feature.

[0231] In an implementable embodiment, the data acquisition module 10 is further configured to perform interpolation compensation on the micro-environment parameter set to obtain a micro-environment parameter pre-processing set;

[0232] determine a state vector according to the micro-environment parameter pre-processing set, fuse the state vector according to an observation equation of the Kalman filter, and obtain a micro-environment filtering parameter;

[0233] generate an environmental risk index according to an environmental risk weight factor and the micro-environment filtering parameter.

[0234] In an implementable embodiment, the feature processing module 20 is further configured to extract the spatial feature of the video stream and the space-time feature of the vibration spectrum in the multi-modal data cube through double-path convolution;

[0235] align the spatial feature and the space-time feature to obtain a space-time aligned deep feature;

[0236] determine an environmental risk index, and generate an attention weight according to the environmental risk index;

[0237] dynamically weight the deep feature based on the attention weight to generate a risk feature tensor with space-time consistency.

[0238] In an implementable embodiment, the risk rating module 30 is further configured to determine the physical connection relationship of each device in the coverage area, and determine a device topology relationship based on the physical connection relationship;

[0239] determine the neighborhood relationship of each device based on the device topology relationship;

[0240] spatially and temporally convolve the neighborhood relation, aggregate neighborhood risk information of the target hop number;

[0241] determine a neighborhood risk factor and a risk gradient according to the neighborhood risk information;

[0242] determine a risk positioning coordinate according to the risk gradient, and determine a dynamic risk level according to the neighborhood risk factor and the risk positioning coordinate.

[0243] In a feasible implementation, the information encryption module 40 is further configured to perform post-quantum encryption on the risk positioning coordinate and the dynamic risk level to obtain encrypted information.

[0244] perform a hash operation on the encrypted information to obtain a risk fingerprint.

[0245] package the risk fingerprint and the encrypted information to obtain an alarm information.

[0246] In a feasible implementation, the security warning module 50 is further configured to determine a priority queue based on the dynamic risk level and a time sequence.

[0247] embed the alarm information in the priority queue into a guarantee time window from the head to the tail of the priority queue to obtain a warning information.

[0248] upload the warning information to a central cloud platform.

[0249] The edge computing-based intelligent rail area security warning device provided in the present application adopts the edge computing-based intelligent rail area security warning method in the above embodiments, and can solve the technical problem of low warning efficiency caused by data isolation in rail area security monitoring. Compared with the prior art, the edge computing-based intelligent rail area security warning device provided in the present application has the same beneficial effects as the edge computing-based intelligent rail area security warning method provided in the above embodiments, and other technical features in the edge computing-based intelligent rail area security warning device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0250] The present application provides an edge computing-based intelligent rail area security warning device, which comprises at least one processor and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the edge computing-based intelligent rail area security warning method in the above embodiment one.

[0251] The following will be described with reference toFigure 4 This document illustrates a structural schematic diagram of a smart rail transit safety warning device based on edge computing, suitable for implementing embodiments of this application. The smart rail transit safety warning device based on edge computing in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The edge computing-based smart rail zone safety early warning device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0252] like Figure 4 As shown, the edge computing-based smart rail transit safety warning device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in ROM (Read Only Memory) 1002 or programs loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the edge computing-based smart rail transit safety warning device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the edge computing-based smart rail safety warning device to exchange data with other devices wirelessly or via wired communication. Although the figure shows an edge computing-based smart rail safety warning device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0253] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0254] The edge computing-based intelligent rail area safety warning device provided by the present application adopts the edge computing-based intelligent rail area safety warning method in the above-mentioned embodiments, and can solve the technical problem of edge computing-based intelligent rail area safety warning. Compared with the prior art, the edge computing-based intelligent rail area safety warning device provided by the present application has the same beneficial effects as the edge computing-based intelligent rail area safety warning method provided by the above-mentioned embodiments, and other technical features in the edge computing-based intelligent rail area safety warning device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0255] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0256] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0257] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the edge computing-based intelligent rail area safety warning method in the above-mentioned embodiments.

[0258] The computer readable storage medium provided in the application may be, for example, a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory or flash memory), an optical fiber, a CD-ROM (CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination thereof.

[0259] The computer readable storage medium described above may be contained in the intelligent rail area safety warning device based on edge computing; or may exist independently without being assembled into the intelligent rail area safety warning device based on edge computing.

[0260] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the intelligent rail area safety warning device based on edge computing, the intelligent rail area safety warning device based on edge computing: collects video streams, three-dimensional vibration spectra, and micro-environment parameter sets in a coverage area in real time, constructs a multi-modal data cube with space-time correlation according to the video streams, the three-dimensional vibration spectra, and the micro-environment parameter sets; extracts deep features of each modality in the multi-modal data cube, dynamically weights the deep features, and generates a risk feature tensor with space-time consistency; determines a neighborhood risk factor based on the risk feature tensor, determines a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor; performs post-quantum encryption on the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, generates an alarm message based on the risk fingerprint; embeds the alarm message into a guarantee time window and uploads it to a central cloud platform, so that the central cloud platform triggers a warning response.

[0261] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0262] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0263] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0264] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned edge computing-based intelligent rail area safety warning method, and can solve the technical problem of edge computing-based intelligent rail area safety warning. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the edge computing-based intelligent rail area safety warning method provided by the above-mentioned embodiments, and will not be described here.

[0265] The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the edge computing-based safety early warning method for a smart rail area as described above.

[0266] The computer program product provided by the application can solve the technical problem of the edge computing-based safety early warning for a smart rail area. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the edge computing-based safety early warning method for a smart rail area provided by the above-mentioned embodiments, and are not described here.

[0267] The above is only some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A method for safety warning in a smart rail area based on edge computing, characterized in that, The edge computing-based intelligent rail area safety warning method comprises: real-time collection of video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area, construction of a multi-modal data cube with space-time correlation according to the video streams, the three-dimensional vibration spectra and the micro-environment parameter sets; extraction of deep features of each modality in the multi-modal data cube, dynamic weighting of the deep features, and generation of a risk feature tensor with space-time consistency; determination of a neighborhood risk factor based on the risk feature tensor, determination of a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor; post-quantum encryption of the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, and generation of an alarm message based on the risk fingerprint; embedding of the alarm message into a guarantee time window and uploading to a central cloud platform to enable the central cloud platform to trigger a pre-warning response; wherein the step of real-time collection of video streams, three-dimensional vibration spectra and micro-environment parameter sets in a covered area, and construction of a multi-modal data cube with space-time correlation according to the video streams, the three-dimensional vibration spectra and the micro-environment parameter sets comprises: alignment of clock information of all sensors in the covered area, establishment of a global timestamp according to the clock information, mapping of position information of all sensors to a three-dimensional space coordinate system, and generation of a space-time unified network according to the global timestamp and the three-dimensional space coordinate system; dynamic background modeling and foreground segmentation of video streams of the covered area, determination of moving targets in the area, and determination of space-time dynamic features of the moving targets; wavelet packet decomposition of three-dimensional vibration spectra of the covered area, extraction of impact energy features, and construction of an abnormal pattern recognition matrix based on the impact energy features; environmental risk index generation through a Kalman filter for micro-environment parameter set fusion; in the space-time unified network, construction of a multi-modal data cube with space-time correlation according to the space-time dynamic features, the abnormal pattern recognition matrix and the environmental risk index; the steps of determining a neighborhood risk factor based on the risk feature tensor, and determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor comprise: determination of physical connection relationships of devices in the covered area, and determination of device topology relationships based on the physical connection relationships; determination of neighborhood relationships of each device based on the device topology relationships; spatial-temporal convolution of the neighborhood relationships, and aggregation of neighborhood risk information of a target hop count; determination of a neighborhood risk factor and a risk gradient according to the neighborhood risk information; determination of a risk positioning coordinate according to the risk gradient, and determination of a dynamic risk level according to the neighborhood risk factor and the risk positioning coordinate.

2. The method of claim 1, wherein, the steps of dynamic background modeling and foreground segmentation of video streams of the covered area, determination of moving targets in the area, and determination of space-time dynamic features of the moving targets comprise: determination of illumination intensity in video streams of the covered area, generation of a dynamic light intensity weight according to the illumination intensity, and creation of a dynamic background based on the dynamic light intensity weight; Performing a morphological top-hat transform on the video stream to determine a contrast of small targets in the video stream, determining a foreground mask according to the contrast, performing foreground segmentation based on the foreground mask, and determining a foreground target; Projecting the foreground target to the dynamic background to determine a moving target in a region; Calculating a minimum circumscribed rectangle of the moving target based on the foreground mask, extracting a feature vector of the minimum circumscribed rectangle, and generating a spatio-temporal dynamic feature according to the feature vector.

3. The method of claim 1, wherein, The steps of performing wavelet packet decomposition on the three-dimensional vibration spectrum of the coverage area, extracting impact energy features, and constructing an anomaly pattern recognition matrix based on the impact energy features include: Performing adaptive noise complete ensemble empirical mode decomposition on the three-dimensional vibration spectrum of the coverage area to generate an enhanced signal; Performing wavelet packet decomposition on the enhanced signal to obtain a plurality of sub-bands; Calculating energy entropy of each of the sub-bands, and marking a sub-band with a preset frequency in the energy entropy as an impact-sensitive frequency band; Determining an energy mutation rate of the impact-sensitive frequency band, marking the impact-sensitive frequency band as an anomaly pattern when the energy mutation rate is greater than an abnormal mutation rate, and extracting impact energy features of the impact-sensitive frequency band; Determining time features, frequency domain features, and spatial features of the impact energy features, and generating an anomaly pattern recognition matrix according to the time features, the frequency domain features, and the spatial features.

4. The method of claim 1, wherein, The steps of fusing the micro-environment parameter set through the Kalman filter to generate an environmental risk index include: Performing interpolation compensation on the micro-environment parameter set to obtain a micro-environment parameter pre-processing set; Determining a state vector according to the micro-environment parameter pre-processing set, fusing the state vector through an observation equation of the Kalman filter to obtain a micro-environment filtering parameter, and generating an environmental risk index according to an environmental risk weight factor and the micro-environment filtering parameter. The steps of extracting depth features of each modality in the multi-modal data cube, dynamically weighting the depth features, and generating a risk feature tensor with spatio-temporal consistency include:

5. The method of claim 1, wherein, Extracting spatial features of a video stream and spatio-temporal features of a vibration spectrum in the multi-modal data cube through a double-path convolution; Performing feature alignment on the spatial features and the spatio-temporal features to obtain spatio-temporally aligned depth features; Determining an environmental risk index, and generating an attention weight according to the environmental risk index; Dynamically weighting the depth features based on the attention weight to generate a risk feature tensor with spatio-temporal consistency. The steps of post-quantum encrypting the risk positioning coordinates and the dynamic risk level to obtain a risk fingerprint, and generating an alarm message based on the risk fingerprint include:

6. The method of claim 1, wherein, Post-quantum encrypting the risk positioning coordinates and the dynamic risk level to obtain encrypted information; Performing a hash operation on the encrypted information to obtain a risk fingerprint; Packaging the risk fingerprint and the encrypted information to obtain an alarm message. The steps of embedding the alarm message in a guarantee time window and uploading it to a central cloud platform include:

7. The method of claim 1, wherein, Determining a priority queue for the alarm message based on the dynamic risk level and a time sequence; ​ Embedding the alarm messages in the priority queue into a guarantee time window from the head to the tail of the priority queue to obtain early warning information based on the priority queue; Upload the early warning information to a central cloud platform.

8. An edge computing-based intelligent rail area safety warning device, characterized in that, The safety early warning device based on edge computing in the intelligent rail area comprises: A data acquisition module is configured to acquire video streams, three-dimensional vibration spectra, and micro-environment parameter sets in a covered area in real time, and construct a multi-modal data cube with spatiotemporal correlation based on the video streams, the three-dimensional vibration spectra, and the micro-environment parameter sets; A feature processing module is configured to extract deep features of each modality in the multi-modal data cube, dynamically weight the deep features, and generate a risk feature tensor with spatiotemporal consistency; A risk rating module is configured to determine a neighborhood risk factor based on the risk feature tensor, determine a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor; An information encryption module is configured to post-quantum encrypt the risk positioning coordinate and the dynamic risk level to obtain a risk fingerprint, and generate an alarm message based on the risk fingerprint; A safety early warning module is configured to upload the alarm message embedded in the guarantee time window to a central cloud platform, so that the central cloud platform triggers an early warning response; The step of acquiring the video streams, the three-dimensional vibration spectra, and the micro-environment parameter sets in the covered area in real time, and constructing the multi-modal data cube with spatiotemporal correlation based on the video streams, the three-dimensional vibration spectra, and the micro-environment parameter sets comprises: Aligning clock information of all sensors in the covered area, establishing a global timestamp based on the clock information, mapping position information of all sensors to a three-dimensional space coordinate system, and generating a spatiotemporal unified network based on the global timestamp and the three-dimensional space coordinate system; Performing dynamic background modeling and foreground segmentation on the video streams of the covered area, determining moving targets in the area, and determining spatiotemporal dynamic features of the moving targets; Performing wavelet packet decomposition on the three-dimensional vibration spectra of the covered area, extracting impact energy features, and constructing an abnormal pattern recognition matrix based on the impact energy features; Fusing the micro-environment parameter set through a Kalman filter to generate an environmental risk index; In the spatiotemporal unified network, constructing the multi-modal data cube with spatiotemporal correlation based on the spatiotemporal dynamic features, the abnormal pattern recognition matrix, and the environmental risk index; The steps of determining a neighborhood risk factor based on the risk feature tensor, and determining a risk positioning coordinate and a dynamic risk level based on the neighborhood risk factor comprise: Determining a physical connection relationship of devices in the covered area, and determining a device topology relationship based on the physical connection relationship; Determining a neighborhood relationship of each device based on the device topology relationship; Performing spatiotemporal convolution on the neighborhood relationship, and aggregating neighborhood risk information of a target hop number; Determining a neighborhood risk factor and a risk gradient based on the neighborhood risk information; Determining a risk positioning coordinate based on the risk gradient, and determining a dynamic risk level based on the neighborhood risk factor and the risk positioning coordinate.

Images