Rail transit monitoring method, device, equipment, storage medium and program product

Abstract

This invention discloses a method, apparatus, device, storage medium, and program product for monitoring rail transit. The method includes the following steps: S1: collecting brightness changes within the monitoring scene to generate a raw event stream, and simultaneously collecting sensor data within the monitoring scene; S2: removing spurious events from the raw event stream and outputting a formatted valid event stream; S3: inputting the valid event stream into a pre-trained neural network model, which outputs visual anomaly events after inference; S4: fusing the visual anomaly events with the sensor data and outputting the monitoring results. This disclosure solves the problems of low environmental adaptability, poor real-time performance, poor accuracy, and high power consumption in rail transit monitoring.

Description

Technical Field

[0001] This disclosure relates to the field of safety monitoring technology, and in particular to methods, devices, equipment, storage media, and program products for monitoring rail transit. This disclosure employs neuromorphic intelligent hardware and computer vision technology to monitor rail transit safety, and is particularly suitable for safety production scenarios such as monitoring rail transit electrical systems, dynamic interactive monitoring of pantograph-catenary systems, and monitoring bridge and tunnel structural faults. Background Technology

[0002] During the operation of rail transit vehicles, safety hazards such as pantograph-catenary arc discharge, track cracks, bridge cracks, and excessive exposure to strong light at tunnel entrances and exits are prone to occur, necessitating safety monitoring. Rail transit vehicles operate at high speeds in challenging environments, including tunnels and bridges, with complex lighting conditions. Furthermore, most monitored scenarios are unattended, making safety monitoring conditions complex. The long distances of rail transit lines also place high demands on the power consumption of monitoring equipment.

[0003] Traditional safety monitoring relies on visual sensing devices (such as CMOS / CCD cameras). These devices use a frame sampling mechanism and rely on a fixed frame rate (e.g., 30-60fps) for image acquisition. The relative motion of objects in the image acquisition scene greatly affects the acquisition effect, and motion blur is prone to occur in high-speed motion scenes. Furthermore, these devices have a small dynamic range under extreme lighting conditions (often only 64-70dB), frequently resulting in overexposure or underexposure, leading to overall failure of safety monitoring. In addition, traditional visual sensing devices have high data processing latency (often 40-50ms), poor real-time warning performance in the event of sudden failures, high power consumption (often 2-4W) requiring fan cooling, and large device size (often greater than 60mm×40mm×25mm), making them difficult to adapt to the compact spaces of rail transit. Regarding data transmission, traditional visual sensing devices use a full-frame storage and transmission method, resulting in a massive amount of data transmitted, increasing the backend processing pressure and maintenance costs.

[0004] An event-based camera (EB) is a measurement device derived from biological vision systems. Its design philosophy is to capture only changes within a scene, rather than static images. Each pixel of an event camera independently monitors changes in light intensity, and when a pixel's brightness change exceeds a preset threshold, it immediately outputs an event. This mechanism ensures that the camera outputs data only when a change occurs, rather than periodically acquiring complete images like traditional cameras. Event cameras exhibit a logarithmic dynamic range in their response characteristics, enabling them to operate effectively in extremely dark or bright environments. Their asynchronous data acquisition mechanism gives them significant advantages in special environments such as high-speed movement and low light conditions in rail transit monitoring. Furthermore, event cameras feature a special pixel circuit design, with each pixel operating independently and activating only when a change occurs, significantly reducing system power consumption.

[0005] While existing event cameras outperform traditional visual sensing devices with features such as asynchronous event-driven operation, high dynamic range, and low power consumption, they still have shortcomings. When monitoring and collecting data on rail transit faults, existing event cameras suffer from a high false event rate. They rely on backend industrial control computers for processing, but these computers have simple processing programs that lack intelligent adjustments for complex events, resulting in low accuracy in event determination and high link latency. Furthermore, existing event cameras cannot collaborate with multiple sensors in the rail transit monitoring environment, making it difficult to achieve multi-source data fusion analysis.

[0006] Therefore, there is an urgent need to develop a safety monitoring method and device adapted to rail transit scenarios, addressing the aforementioned technical pain points through collaborative innovation in hardware architecture and methodology. This will improve the accuracy, real-time performance, environmental adaptability, and low-power consumption of rail transit safety monitoring, meeting the demands of complex dynamic scenarios and long-term unattended operation and maintenance requirements. Furthermore, collaborative innovation in hardware architecture and methodology can also solve the problem of multi-sensor collaboration, further enhancing the accuracy of safety monitoring in rail transit scenarios.

[0007] The above information is presented as background information only to aid in understanding this disclosure. No confirmation or other representation is made regarding whether any of the above constitutes an application of prior art to this disclosure. Summary of the Invention

[0008] This disclosure addresses the problems of low environmental adaptability, poor real-time performance, poor accuracy, and high power consumption in rail transit monitoring, and provides a rail transit monitoring method. It also provides a rail transit monitoring device, system, non-transitory storage medium, and computer program product to solve the problems in executing the rail transit monitoring method.

[0009] The first aspect of this disclosure provides a method for monitoring rail transit, comprising the following steps: S1: collecting brightness changes within the monitoring scene to generate an original event stream, and simultaneously collecting sensor data within the monitoring scene; S2: removing pseudo-events from the original event stream and outputting a formatted valid event stream; S3: inputting the valid event stream into a pre-trained neural network model, wherein the neural network model outputs visual anomaly events after inference; S4: fusing the visual anomaly events with the sensor data and outputting monitoring results.

[0010] For example, in at least one embodiment, it further includes S5: transmitting the monitoring results to a security management platform in real time, wherein the security management platform responds in real time and archives the monitoring results.

[0011] For example, in at least one embodiment, step S2 includes: S21: receiving the original event stream and formatting the event data therein; S22: constructing a spatial neighborhood window and a temporal neighborhood window; S23: traversing the original event stream to obtain associated events within the spatial neighborhood window and the temporal neighborhood window; S24: performing a consistency judgment on the brightness and polarity of the associated events, removing the pseudo-events, and retaining the valid events; S25: arranging the valid events in ascending order of timestamps and outputting the formatted valid event stream.

[0012] For example, in at least one embodiment, in step S22, the spatial neighborhood window and / or the temporal neighborhood window are dynamically and adaptively adjusted in real time.

[0013] For example, in at least one embodiment, step S3 includes: S31: dividing the effective event stream into continuous event blocks according to a time window, statistically analyzing the spatial distribution of the event blocks in the pixel array, and dividing the pixel array into n spatial grids; S32: calculating the number of events in each spatial grid, normalizing it, and using it as the initial input of the n neurons in the input layer of the neural network model; S33: the hidden layer of the neural network model includes m neurons (m>n), each input layer neuron connects to the neurons in the hidden layer, and uses a leak integral-fire model for pulse transmission; S34: the output layer of the neural network model includes 2 neurons, the 2 neurons integrate the pulse signals within the time window respectively, and output the visual abnormality event based on the integration result.

[0014] For example, in at least one embodiment, step S4 includes: S41: receiving the visual anomaly event and the sensor data; S42: sending a calibration pulse to the sensor based on the clock that generated the original event stream, and the sensor correcting its timestamp error based on the calibration pulse; S43: searching the sensor data within the adjacent time window of the visual anomaly event and establishing a data association pair; S44: fusing the data in the data association pair and outputting the monitoring result.

[0015] For example, in at least one embodiment, prior to step S3, the neural network model is pre-trained as follows: Step 01: Collect multiple sets of rail transit event samples in different scenarios, including multiple sets of normal event samples and multiple sets of fault event samples; Step 02: Input the multiple sets of rail transit event samples into the neural network model in batches, and perform multiple iterations using the impulse temporal dependence plasticity learning rule until the model accuracy converges to a preset threshold; Step 03: Prune redundant connections in the neural network model, retain the links of the core connections, and compress the total number of model parameters.

[0016] A second aspect of this disclosure provides a rail transit monitoring device, comprising: an event camera configured to collect brightness changes within a monitoring scene and generate an original event stream; a sensor configured to simultaneously collect sensor data within the monitoring scene along with the event camera; a neuromorphic processing module configured to remove pseudo-events from the original event stream, output a formatted valid event stream, input the valid event stream into a pre-trained neural network model, and output visual anomaly events after inference from the neural network model; and a data fusion module configured to perform data fusion on the visual anomaly events and the sensor data, and output monitoring results.

[0017] For example, in at least one embodiment, the event camera includes a cone-rod heterogeneous pixel array, which includes multiple cone pixel units and multiple rod pixel units; the neuromorphic processing module includes an event filtering unit, a sparse computation unit, and a temporary storage unit; the event filtering unit is configured to remove pseudo-events from the original event stream and output a formatted valid event stream; the sparse computation unit constructs a 3-layer lightweight model based on a spiking neural network, including an input layer, a hidden layer, and an output layer, wherein the input layer is fully connected to the hidden layer, the hidden layer is sparsely connected to the input layer and the output layer, and the output layer is fully connected to the hidden layer, configured to input the valid event stream into a pre-trained neural network model, and the neural network model outputs visual abnormal events after inference; the temporary storage unit is configured to store key event features without requiring an independent frame buffer.

[0018] For example, in at least one embodiment, it further includes a rail transit interface module and a safety management platform. The rail transit interface module includes a synchronization interface and a data interface, configured to realize signal transmission and time alignment. The safety management platform is configured to receive the monitoring results in real time, and to perform real-time response and archiving based on the monitoring results.

[0019] A third aspect of this disclosure provides a computer device, comprising: a memory for non-temporarily storing computer-executable instructions; and a processor for running the computer-executable instructions, wherein the computer-executable instructions, when run by the processor, perform the rail transit monitoring method described in any of the preceding claims.

[0020] The fourth aspect of this disclosure provides a non-transitory storage medium for non-transitory storage of computer-executable instructions, characterized in that, when the computer-executable instructions are executed by a computer, the rail transit monitoring method described in any of the preceding claims is executed.

[0021] The fifth aspect of this disclosure provides a computer program product, including a computer program, characterized in that, when executed by a processor, the computer program implements the rail transit monitoring method described in any of the preceding claims. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure, and are not intended to limit this disclosure.

[0023] Figure 1 This is a flowchart of a rail transit monitoring method according to an embodiment of the present disclosure;

[0024] Figure 2 This is a flowchart of the steps for removing pseudo-events according to an embodiment of the present disclosure;

[0025] Figure 3 This is a flowchart of the neural network model inference steps according to an embodiment of the present disclosure;

[0026] Figure 4 This is a flowchart of the data fusion steps according to an embodiment of the present disclosure;

[0027] Figure 5 This is a flowchart of the neural network model pre-training process according to an embodiment of the present disclosure;

[0028] Figure 6 This is a schematic diagram of a rail transit monitoring device according to an embodiment of the present disclosure;

[0029] Figure 7A These are conventional CMOS camera images according to embodiments of this disclosure;

[0030] Figure 7B This is a rail transit monitoring imaging image according to an embodiment of the present disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes. In this disclosure, “multiple” means two or more.

[0033] According to embodiments of this disclosure, Figure 1 A rail transit monitoring method is disclosed, comprising the following steps: Step S1: Acquire brightness changes within the monitoring scene to generate a raw event stream, and simultaneously acquire sensor data within the monitoring scene. The raw signal acquisition in Step S1 is based on an event camera and sensors. The installation location of the event camera is not limited; it can be installed on rail transit vehicles (e.g., high-speed rail, subway), or on the track or at a suitable location near the track (e.g., in tunnels, on bridges). Unlike traditional visual monitoring devices that acquire images at a static time, the event camera acquires brightness changes within the monitoring scene and generates a raw event stream based on these changes. In rail transit safety monitoring, monitoring in extremely dark environments such as nighttime and tunnels, as well as in extremely bright environments under special circumstances, is often a key focus. The event camera's response characteristics have a logarithmic dynamic range, enabling it to work effectively in extremely dark or bright environments, as well as during high-speed movement. Simultaneously, the event camera features a special pixel circuit design, with each pixel working independently and activating only when changes occur, significantly reducing system power consumption.

[0034] Preferably, the sensor can be one or more of the following: vibration sensor, strain sensor, acceleration sensor, lidar, and infrared sensor, used to collect sensor data changes near the measured location. The sensor can be installed on rail transit vehicles (e.g., high-speed rail, subway), or at a suitable location on or near the track (e.g., in tunnels, on bridges, on overhead wires), adjacent to key monitoring locations, to collect sensor data changes associated with the event. Event cameras combined with large models offer advantages in real-time monitoring and accuracy, while sensor data reflects fault locations more precisely and provides a more quantitative assessment of fault severity. Integrating visual data collected by event cameras with sensor data can improve fault location accuracy and the reliability of fault level determination.

[0035] Preferably, the event camera employs a cone-shaped + rod-shaped heterogeneous pixel array. This array includes staggered cone-shaped and rod-shaped heterogeneous pixels, with the number of each type potentially equal or unequal. Signals acquired by different types of cone-shaped heterogeneous pixels are classified and processed, while signals from multiple cone-shaped pixels are processed centrally for high-resolution color perception, primarily used to capture the color features of faults. Signals from multiple rod-shaped heterogeneous pixels are also processed centrally, with the rod pixels supporting spatiotemporal difference calculations, primarily used for locating structural faults. This cone-shaped + rod-shaped heterogeneous pixel array simulates the function of cone / rod cells in the human eye, drawing on the characteristics of the human visual system to form two complementary and comprehensive visual perception pathways. Such an event camera not only possesses high dynamic range and high resolution but also maintains excellent perception under high-speed motion and extreme lighting conditions, making it far superior to existing event cameras for monitoring extreme environments in rail transit.

[0036] Preferably, the cone-shaped + rod-shaped heterogeneous pixel array of the event camera includes multiple cone-shaped heterogeneous pixels and multiple rod-shaped heterogeneous pixels. The multiple cone-shaped heterogeneous pixels include multiple red light cone-shaped heterogeneous pixels, multiple green light cone-shaped heterogeneous pixels, and multiple blue light cone-shaped heterogeneous pixels. Both the cone-shaped and rod-shaped heterogeneous pixels consist of a photodetector, a dual-threshold comparator, and a microsecond-level timestamp circuit. The photodetector is used for high-sensitivity photoelectric conversion; the dual-threshold comparator is used to dynamically adjust the brightness change threshold to distinguish between normal light changes and fault events; and the microsecond-level timestamp circuit is used to accurately mark the time of event occurrence for timing alignment in data fusion.

[0037] Preferably, the cone-shaped heterogeneous pixels have a receptive field of 4 micrometers, sensing red, green, and blue light information. The photodetectors of the cone-shaped heterogeneous pixels include color filters: red cone-shaped heterogeneous pixels include a red light filter, resolving to red light information; green cone-shaped heterogeneous pixels include a green light filter, resolving to green light information; and blue cone-shaped heterogeneous pixels include a blue light filter, resolving to blue light information. The rod-shaped heterogeneous pixels have a receptive field of 8 micrometers, sensing white light information. The photodetectors of the rod-shaped heterogeneous pixels include a white light filter, resolving to white light information. The rod-shaped heterogeneous pixels have embedded memory, supporting spatiotemporal difference calculations. The cone + rod-shaped heterogeneous pixel array has a resolution of 640×320, a dynamic range of 90-120dB, an event rate of 12-450MEPS, and a dual-threshold comparator threshold that can be dynamically adjusted within a 5%-20% brightness variation range. This configuration of the event camera demonstrates superior performance in terms of quantum efficiency, dynamic range, response speed, power consumption, and bandwidth.

[0038] like Figure 1 As shown, the rail transit monitoring method includes the following step S2: removing spurious events from the original event stream and outputting a formatted valid event stream. In some harsh environments (e.g., low-light environments in tunnels with illumination intensity ≤1 lux), rail transit safety monitoring is prone to generating isolated spurious events due to abnormal factors such as temperature noise and circuit noise. Since real events typically have spatiotemporal continuity, while spurious events are randomly and isolated, constructing a spatiotemporal neighborhood window and filtering valid events that meet spatiotemporal correlation can eliminate isolated noise events, reduce misjudgments in rail transit safety monitoring, and improve the accuracy of rail transit safety monitoring.

[0039] Preferred, such as Figure 2 As shown, step S2 includes step S21: receiving the raw event stream and formatting the event data within it. The filtering unit of the neuromorphic processing module receives the raw event stream output by the event camera and defines each event data point as an event format: E = (x, y, t, p). Where: x and y are pixel coordinates, t is the timestamp, and p is the brightness polarity. The events are temporarily buffered in timestamp order, with a buffer depth ≤ 1000 events. The formatted and valid event stream provides a good foundation for subsequent processing.

[0040] Preferably, the unit for pixel coordinates x and y is pixels, and the value range for a 640×320 pixel array is 0≤x≤639, 0≤y≤319. The unit for the timestamp t is μs, with a precision of 1μs. The value of the brightness polarity p is +1 to indicate brightening and -1 to indicate darkening.

[0041] Preferred, such as Figure 2As shown, step S2 includes step S22: constructing a spatial neighborhood window and a temporal neighborhood window. For each decision event, a spatial neighborhood window is constructed, which includes multiple events that are spatially adjacent pixels to the decision event. For each decision event, a temporal neighborhood window is constructed, which includes multiple events that are temporally adjacent to the decision event.

[0042] Preferably, for each event to be determined, E0=(x0, y0, t0, p0), a 3×3 spatial neighborhood window is constructed. The pixel range of the spatial neighborhood window is: S = {(x, y) | x0-1 ≤ x ≤ x0+1, y0-1 ≤ y ≤ y0+1}, where x and y are pixel coordinates. This spatial neighborhood window includes events at 9 pixel positions. Simultaneously, the temporal neighborhood window is set to [t0-T]. t , t0+T t ], where T t The temporal neighborhood threshold is set to 8 μs. By judging the temporal difference and spatial distance between adjacent pixel events, isolated pseudo-events in low-light environments are eliminated. Using this method, the pseudo-event elimination rate can reach over 92%.

[0043] Preferably, the comparison of time neighborhood thresholds is achieved through a high-speed timer and a comparator, supporting real-time dynamic adaptive adjustment of the dynamic threshold.

[0044] Preferably, the spatial neighborhood window and / or temporal neighborhood window are dynamically and adaptively adjusted in real time. The event density D (unit: MEPS, where MEPS represents events per second) of the current environment is calculated in real time. When D is less than the preset minimum value, the system may be in an environment with extremely low light, where events are often dominated by noise. Enlarging the temporal neighborhood threshold and / or spatial neighborhood window can lower the threshold for valid event judgment and avoid erroneous deletion of valid events. When D is greater than the preset maximum value, events may be dense due to target movement, etc. Reducing the temporal neighborhood threshold and / or spatial neighborhood window helps to strictly filter related events and reduce redundancy.

[0045] Preferably, the event density D of the current environment is statistically analyzed in real time. When D < 5 MEPS, the time neighborhood threshold T is set. t Adjust to 10 μs, maintain a spatial neighborhood of 3×3; when 5 MEPS ≤ D ≤ 50 MEPS, maintain T. t =8μs; when D>50MEPS (target motion in low light causes dense events), T t Adjusted to 5μs. The time neighborhood threshold is dynamically adjusted based on event density, which can balance noise suppression and preservation of moving target events.

[0046] Preferred, such as Figure 2As shown, step S2 includes step S23: traversing the original event stream to obtain associated events within the spatial neighborhood window and the temporal neighborhood window. For the formatted original event stream, traverse all events, perform conditional judgments on all events, and count all associated events within the spatial neighborhood window and the temporal neighborhood window.

[0047] Preferably, for the event to be determined, E0 = (x0, y0, t0, p0), events that meet the conditions within the formatted original event stream are obtained. The conditions are: the space is within S = {(x, y) | x0-1 ≤ x ≤ x0+1, y0-1 ≤ y ≤ y0+1}, where x and y are pixel coordinates; the time is within [t0-T]. t , t0+T t Within ], of which T t This is the time neighborhood threshold.

[0048] Preferably, the event acquisition is implemented through a dedicated logic circuit, which traverses the 3×3 pixel coordinates in parallel and searches for events with a latency less than a preset threshold (e.g., ≤10ns / event).

[0049] Preferred, such as Figure 2 As shown, step S2 includes step S24: Performing a consistency judgment on the brightness and polarity of associated events, removing false events, and retaining valid events. Preset thresholds are set for the number of associated events and the proportion of events with the same polarity. If the number of associated events within the spatial and temporal neighborhood windows is greater than or equal to the preset thresholds, and the proportion of events with the same polarity is greater than or equal to the preset thresholds, then the event is determined to be a valid event. If the number of associated events within the spatial and temporal neighborhood windows is less than the preset thresholds, or the proportion of events with the same polarity is less than the preset thresholds, then the event is determined to be a false event.

[0050] Preferably, for the event to be judged, E0 = (x0, y0, t0, p0), the number of related events N that meet the conditions is counted. If N ≥ 2 (there are at least 2 spatiotemporally related events), and the proportion of related events with the same polarity as p0 is ≥ 60%, then E0 is judged as a valid event; if N < 2, or the proportion of events with the same polarity is < 60%, then E0 is judged as a pseudo-event and marked as to be removed.

[0051] Preferably, event acquisition and determination are implemented through an embedded microcontroller (MCU), with a single determination time of less than or equal to 50ns, ensuring real-time processing of the event stream (supporting a maximum event rate of 450MEPS).

[0052] Preferred, such as Figure 2As shown, step S2 includes step S25: Sort valid events in ascending order of timestamp and output a formatted valid event stream. Remove all pseudo-events marked as to be removed, sort valid events in ascending order of timestamp, and output a formatted valid event stream E_valid = {(x, y, t, p) | set of valid events}. Even in complex scenes such as low light, the pseudo-event rate can be less than 7%.

[0053] like Figure 1 As shown, the rail transit monitoring method includes the following step S3: The effective event stream is input into a pre-trained neural network model, and the neural network model outputs visual anomaly events after inference. In the measurement of event cameras, the actual fault characteristics are spatiotemporally continuous event clusters with rapidly changing brightness. Faults and sporadic visual anomalies differ in event density, frequency of polarity changes, and spatial diffusion speed. For example, pantograph-catenary arc faults are continuous discharge processes that generate high temperature, strong light, and high energy; their pseudo-events, such as occasional electric sparks, are not continuous in time and space compared to arc faults. By constructing a model and identifying the distinguishing features, the brightness polarity change frequency and spatial distribution pattern of the event stream can be accurately distinguished, eliminating the interference of sporadic visual anomalies and enabling timely and effective fault identification.

[0054] Preferably, the neural network model is a lightweight spiking neural network (SNN), which encodes the spatiotemporal features of the event stream into pulse signals and achieves fault classification and identification through the sparse connections of neurons and pulse transmission.

[0055] Preferably, the lightweight spiking neural network (SNN) includes a 3-layer lightweight model, wherein the input layer is fully connected to the hidden layer, the hidden layer is sparsely connected to the input layer and the output layer, and the output layer is fully connected to the hidden layer. It is configured to input a valid event stream into the pre-trained neural network model, and the neural network model outputs visual abnormal events after inference.

[0056] Preferably, the lightweight spiking neural network (SNN) comprises a 3-layer lightweight model, wherein the input layer contains 64 neurons, the hidden layer contains 128 neurons, and the output layer contains 2 neurons. After pre-training, this model can achieve a fault detection accuracy of over 95%.

[0057] Preferred, such as Figure 3 As shown, step 3 includes step S31: dividing the effective event stream into continuous event blocks according to the time window, statistically analyzing the spatial distribution of the event blocks in the pixel array, and dividing the pixel array into n spatial grids.

[0058] Preferably, the valid event stream E_valid is divided into consecutive event blocks Block = {E_valid | t ∈ [t_k, t_k+T_win]} according to a time window, where E_valid is the valid event stream, t is the time, t_k is the start time of the time stream, and T_win is the time window. The length of the time window matches the camera inference delay time.

[0059] Preferably, T_win=1ms. The spatial distribution of the statistical event block in the 640×320 pixel array is divided into 8×8=64 spatial grids (each grid is 80×40 pixels).

[0060] Preferred, such as Figure 3 As shown, step 3 includes step S32: calculating the number of events for each spatial grid, normalizing it, and using it as the initial input for the n neurons in the input layer of the neural network model.

[0061] Preferably, in 8×8=64 spatial grids (each grid is 80×40 pixels), the number of events C_i (i=1~64) in each grid is calculated, normalized to [0, 1] and used as the initial input I_i of the 64 neurons in the input layer.

[0062] Preferably, step S32 further includes enhancing the initial input I_i through event feature encoding: for each spatial grid, the temporal distribution variance σ_ti of the events is calculated, where σ_ti reflects the concentration of events. If σ_ti < 0.2 ms, it indicates that the events are highly concentrated, corresponding to faults such as arc bursts and obvious cracks. In this case, I_i is multiplied by an enhancement coefficient of 1.5 to enhance the temporal characteristics of the faults.

[0063] Preferred, such as Figure 3 As shown, step 3 includes step S33: the hidden layer of the neural network model includes m neurons (m>n), each input layer neuron connects to the neurons in the hidden layer, and the leak integral-fire (LIF) model is used for pulse transmission.

[0064] Preferably, each hidden layer neuron adopts the leakage integral-fire (LIF) model, and the membrane potential decays exponentially with time. The membrane potential formula is: V_m(t) = V_m(t-1)·e^(-Δt / τ_m) + I_j, where V_m(t) is the membrane potential at time t, V_m(t-1) is the membrane potential at time t-1, Δt is the time difference between time t and time t-1, τ_m is the membrane potential decay coefficient, and I_j is the initial input.

[0065] Preferably, the core parameters of the model are: membrane potential threshold V_th = 1.2 (after normalization), membrane potential decay coefficient τ_m = 0.3ms, and membrane potential reset to V_reset = 0 after pulse firing.

[0066] Preferably, there is sparse connection between neurons in the hidden layer, with each neuron in the input layer connecting only 30% of the neurons in the hidden layer. The connection structure is fixed after random initialization. This connection method can reduce the amount of computation.

[0067] Preferred, such as Figure 3 As shown, step 3 includes step S34: the output layer of the neural network model includes two neurons. The two neurons integrate the pulse signals within the time window respectively, and output the visual abnormality event based on the integration result.

[0068] Preferably, the two neurons in the output layer correspond to "normal events (neuron A)" and "fault events (neuron B)" respectively. The pulse signal within the time window T_win is integrated as follows: S_A = Σ pulse firing count (neuron A), S_B = Σ pulse firing count (neuron B). If S_B / (S_A + S_B) > 0.6, it is determined to be a fault, and an alert signal Alert=1 is output; otherwise, it is determined to be a normal situation, and Alert=0 is output.

[0069] Preferably, based on the different event stream characteristics of different fault types, the neural network model can identify fault types and monitor multiple fault types simultaneously. When multiple fault types are monitored simultaneously, the format of the output visual anomaly event includes parameters representing the fault type. When monitoring a single fault type, the format of the output visual anomaly event may not include parameters representing the fault type. For monitoring a single fault type, the format of the visual anomaly event output is V = (x_v, y_v, t_v, Grade_v), where x_v and y_v are the target pixel coordinates (accuracy ±1 pixel), t_v is the visual timestamp, and Grade_v is the fault level determined by vision (e.g., setting the fault level to 0-3). When Grade_v is greater than or equal to a preset threshold (e.g., Grade_v≥1), an early warning signal is actively output; when Grade_v is less than the preset threshold, no early warning signal is actively output, and only event blocks are buffered, which helps reduce the transmission volume. Multiple fault types are monitored. The format of visual abnormal events output by visual monitoring is V = (x_v, y_v, t_v, Grade_v, type_v), where x_v and y_v are the target pixel coordinates, t_v is the visual timestamp, Grade_v is the fault level determined by vision, and type_v is the fault type code.

[0070] Preferably, step S3 is implemented using a spiking neural network chip. The chip adopts a RISC-V architecture, integrates no less than 128 spiking neuron cores, has a weight storage capacity of ≥512KB, a computing power of ≥0.5TOPS, a power consumption of ≤20mW, and an inference latency of ≤1ms.

[0071] Preferably, the input layer feature encoding is implemented through hardware logic circuits, with event block segmentation and grid statistics taking ≤0.2ms. The hidden layer is implemented through a dedicated pulse computing unit, with 128 neurons operating in parallel, and a single forward inference taking ≤0.6ms. The output layer integration and decision-making are implemented through an embedded MCU, with decision-making taking ≤0.1ms and end-to-end inference latency ≤0.8ms.

[0072] like Figure 1 As shown, the rail transit monitoring method includes the following step S4: fusing visual anomaly data and sensor data to output monitoring results. In rail transit safety monitoring, visual monitoring of numerous scenarios is very challenging, and oversights and misjudgments can lead to serious consequences. Visual data is based on the optical characteristics of visible events, while sensor data is based on measurements of fault locations; both can extract quantitative features representing fault type, location, and severity. Collecting both visual and sensor data and extracting features from these two types of data is highly complementary for rail transit safety monitoring. Employing a weighted fusion strategy to fuse the two types of features can further improve the accuracy of fault location and fault severity determination.

[0073] Preferred, such as Figure 4 As shown, step 4 includes step S41: receiving visual anomaly events and sensor data, and extracting their respective feature parameters. The extracted feature parameter formats of the visual anomaly events and sensor data can be aligned in time. Based on a preset feature weighted fusion algorithm, the accuracy of fault location and the reliability of fault level determination can be improved.

[0074] Preferably, the format of the visual abnormality event output by the visual monitoring is the same as the format output in step S3, for example: V = (x_v, y_v, t_v, Grade_v), where x_v and y_v are the target pixel coordinates (accuracy ±1 pixel), t_v is the visual timestamp, and Grade_v is the fault level determined by the vision (for example, setting the fault level to 0-3).

[0075] Preferably, the sensor is an accelerometer. When the visual monitoring output is V = (x_v, y_v, t_v, Grade_v), the data format collected by the accelerometer is converted to M = (a_x, a_y, a_z, t_m, Grade_m) according to a preset algorithm, where a_x, a_y, and a_z are the three-dimensional acceleration values ​​collected by the accelerometer, t_m is the sensor sampling timestamp, and Grade_m is the fault level determined by the sensor. The sensor determines the fault level Grade_m based on a preset acceleration threshold. For example, the fault level is set to 0-3, where a≥2g is level 3, 1g≤a<2g is level 2, 0.5g≤a<1g is level 1, and a<0.5g is level 0.

[0076] Preferably, the sensor is an infrared sensor. When the visual monitoring output is V = (x_v, y_v, t_v, Grade_v), the data format collected by the infrared sensor is converted to M = (i_x, i_y, t_m, Grade_m) according to a preset algorithm, where i_x and i_y are the fault location coordinates collected by the infrared sensor, t_m is the sensor sampling timestamp, and Grade_m is the fault level determined by the sensor. The sensor determines the fault level Grade_m based on a preset acceleration threshold. For example, the fault level is set to 0-3, where a≥2g is level 3, 1g≤a<2g is level 2, 0.5g≤a<1g is level 1, and a<0.5g is level 0.

[0077] Preferred, such as Figure 4 As shown, step 4 includes step S42: using the clock that generated the original event stream as a reference, a calibration pulse is sent to the sensor, and the sensor corrects its timestamp error based on the calibration pulse. Correcting the timestamp error can ensure that the time windows of visually detected abnormal events and sensor data are aligned, thus ensuring the accuracy of subsequent data fusion.

[0078] Preferably, using the system clock of the event camera as a reference (with an accuracy of 1 μs), a calibration pulse is sent to the sensor via the MIPI-CSI2 synchronization interface. The timestamp of the calibration pulse sent by the event camera is t_v0, and the sensor records the local timestamp of the calibration pulse, t_m0. The time offset Δt = t_v0 - t_m0 is calculated. For each piece of sensor data M, its timestamp is corrected to the time t_m' = t_m + Δt under the reference clock, ensuring that the timestamp error between the visually detected abnormal events and the sensor data is less than or equal to 3 μs.

[0079] Preferred, such as Figure 4As shown, step 4 includes step S43: searching sensor data within the nearest time window of the visual anomaly event and establishing data association pairs. These data association pairs serve as the basis for data fusion.

[0080] Preferably, visual anomalies are determined based on a preset fault level, and the associated sensor data is also determined based on a preset fault level. After a visual anomaly is detected, a search is performed within its nearest time window. If the fault level of the sensor data is greater than a preset threshold, a data association pair is established, and the fault level and location are fused and reinforced. If no sensor data with a fault level greater than the preset threshold is found, the visual anomaly event information is output.

[0081] Preferably, when the neural network model outputs a visual anomaly event V after inference, it searches for sensor data M within the range [t_v-5ms, t_v+5ms]. If the fault level of the sensor data is greater than a preset threshold, an association pair (V, M) is established.

[0082] Preferred, such as Figure 4 As shown, step 4 includes step S44: fusing the data in the data association pair and outputting the monitoring results. Visually monitored abnormal events easily identify the occurrence of faults and distinguish fault types, allowing for more intuitive fault detection; sensor data has advantages in fault level determination and quantitative determination of location accuracy. The two are highly complementary, and fusing data can obtain more complete and accurate fault information. The fusion is based on a preset algorithm, which can be optimized through multiple on-site calibrations to improve the accuracy of its output after fusion.

[0083] Preferably, the fused fault level is calculated by weighting visually detected anomalies and sensor data, with each weight calculated based on feature confidence. The formula for calculating the fused fault level is: Grade_fusion = ω_v·Grade_v + ω_m·Grade_m, where Grade_fusion is the fused fault level, ω_v is the visual weight, ω_m is the sensor weight, Grade_v is the fault level determined by vision, and Grade_m is the fault level determined by the sensor. Based on the field calibration model, the fusion algorithm is calibrated at the monitoring site, and the positioning deviation after fusion is corrected by adjusting the parameters.

[0084] Preferably, for any fault type, the output fused monitoring result is: F = (X_fusion, Y_fusion, t_fusion, Grade_fusion), where X_fusion and Y_fusion are physical coordinates (fusion accuracy can reach ±4mm), t_fusion is the fusion timestamp, and Grade_fusion is the final fault level. For example, setting the final fault level to 0-3, triggering an early warning signal when Grade_fusion ≥ 2, and only outputting archived data when Grade_fusion < 2. When monitoring multiple fault types, the output also includes the fault type output by visual monitoring.

[0085] In the embodiments of this disclosure, after data fusion, the positioning accuracy of a single visual monitoring point is improved by 73% compared to before fusion, and the accuracy rate of fault level determination can reach over 96%.

[0086] Preferably, time calibration is achieved through the synchronization circuit of the dedicated rail transit interface module, with a calibration pulse transmission period of 1 second to ensure the stability of the time offset Δt. Feature extraction and weighted fusion are implemented through the sparse computing unit of the neuromorphic processing module, with the hardware-based weighted calculation logic taking ≤0.3ms. Position mapping and error correction are implemented through an embedded MCU, taking ≤0.2ms, with a total end-to-end fusion latency of ≤1ms, meeting real-time monitoring requirements.

[0087] like Figure 1 As shown, the rail transit monitoring method also includes step S5: transmitting the monitoring results to the safety management platform in real time, and the safety management platform responding and archiving based on the monitoring results. Through a dedicated rail transit interface module, the fused results are transmitted to the safety management platform, enabling real-time early warning, data archiving, and remote interaction, completing the monitoring closed loop. For sudden rail transit safety faults, real-time feedback control of equipment and early warning output to personnel can be achieved.

[0088] Preferred, such as Figure 5 As shown, before step S3, the neural network model is pre-trained, including step 01: collecting multiple sets of rail transit event samples in different scenarios, including multiple sets of normal event samples and multiple sets of fault event samples. Step 02: inputting the multiple sets of rail transit event samples into the neural network model in batches, using the impulse temporal dependence plasticity learning rule, and performing multiple rounds of iteration until the model accuracy converges to a preset threshold. Step 03: pruning redundant connections in the neural network model, retaining the links of core connections, and compressing the total number of model parameters. After the pre-training reaches the convergence threshold, the accuracy of the neural network model reaches a high level, and compared with traditional safety monitoring methods, the accuracy and real-time performance of fault determination are significantly improved.

[0089] Preferably, in step 01, the multiple sets of normal event samples include events that are not judged as faults, such as normal sparks, contamination of the track or equipment surface, and debris on the bridge. The multiple sets of visually abnormal event samples include events that are judged as faults, such as faulty electric arcs, track cracks, and bridge cracks.

[0090] Preferably, pre-training is performed for multiple fault types, collecting data on various fault types for pre-training. Taking pantograph-catenary current faults as an example: under different vehicle speeds (e.g., between 200-350 km / h) and different lighting conditions (e.g., strong tunnel light, low nighttime light, etc.), 100,000 sets of rail transit pantograph-catenary arc event samples are collected, including 50,000 sets of normal sparks and 50,000 sets of fault arcs. Taking track crack faults as an example, 100,000 sets of track crack event samples are collected, including 50,000 sets of pseudo-events that visually resemble cracks and 50,000 sets of track cracks, collected in random scenarios where track cracks occur.

[0091] Preferably, the sample collection required for pre-training can be completed on a rail transit simulation platform.

[0092] Preferably, in step 02, the impulse temporal dependent plasticity (STDP) learning rule is adopted, and training is stopped when the accuracy of the neural network model converges to 96% after 100 iterations.

[0093] Preferably, in step 03, redundant connections are pruned, retaining only links with a core connection weight ≥ 0.5. The total number of model parameters is compressed to 512KB, and the compressed model can be adapted to the weight storage unit of the neuromorphic processing module. In rail transit safety monitoring, the real-time performance of early warning and feedback is crucial. The low total number of compressed model parameters ensures real-time early warning and feedback while reducing processor hardware requirements.

[0094] According to embodiments of this disclosure, Figure 6A rail transit monitoring device is disclosed, comprising: an event camera 1, configured to collect brightness changes within the monitoring scene and generate a raw event stream; a sensor 2, configured to simultaneously collect sensor data within the monitoring scene with the event camera 1; a neuromorphic processing module 4, configured to remove pseudo-events from the raw event stream, output a formatted valid event stream, input the valid event stream into a pre-trained neural network model, and output visual anomaly events after inference; and a data fusion module 5, configured to fuse visual anomaly events with sensor data and output monitoring results. The installation location of the event camera 1 is not limited; it can be installed on rail transit vehicles (e.g., high-speed rail, subway), or on the track or in a suitable location near the track (e.g., in tunnels, on bridges). Unlike traditional visual monitoring devices that collect images at a static time, the event camera 1 collects brightness changes within the monitoring scene and generates a raw event stream based on these changes. Monitoring in extremely dark environments such as nighttime and tunnels, as well as in extremely bright environments under special circumstances, is often a key focus in rail transit safety monitoring. Event Camera 1 features a logarithmic dynamic range in its response characteristics, enabling it to operate effectively in extremely dark or bright environments, as well as during high-speed motion. Furthermore, Event Camera 1 employs a unique pixel circuit design, where each pixel operates independently and is activated only when changes occur, significantly reducing system power consumption.

[0095] Preferably, the light intensity range acquired by event camera 1 is 0.1 lux to 10000 lux, which can cover different light signal scenarios such as low light in tunnels and overexposure in strong light. The trigger condition for event camera 1 is a pixel brightness change ΔL ≥ 10%. Each pixel independently monitors the light signal, and when ΔL reaches the threshold, it triggers the timestamp circuit to generate single event data. Event camera 1 generates events only based on the pixels with brightness changes, without a synchronous frame clock, thus avoiding redundant data. The pixel brightness change corresponds to the threshold of the dual threshold comparator in event camera 1, which supports dynamic adjustment. The event encoding module arranges all single events in ascending order of timestamp, forming a standardized raw event stream. Event camera 1 and the neuromorphic processing module 4 use a built-in bus for point-to-point asynchronous transmission without frame buffering. At the maximum event rate, the data transmission volume is ≤450MB / s, which is 2-3 orders of magnitude lower than that of traditional frame cameras.

[0096] Preferably, the pixel array of the event camera 1 is in sleep mode when no event is triggered. This design can save power consumption of ≤5mW, which accounts for 25% of the total power consumption.

[0097] Preferably, sensor 2 can be one or more of a vibration sensor, strain sensor, acceleration sensor, lidar, and infrared sensor, used to collect sensor data changes near the measured location. Sensor 2 can be installed on rail transit vehicles (such as high-speed trains, subways, etc.), or on the track or in a suitable location near the track (such as in tunnels, bridges, or pantographs), adjacent to key monitoring locations, to collect sensor data changes associated with the event. Event camera 1, combined with a large model, has advantages in real-time monitoring and accuracy, while the fault location reflected by sensor data is more precise, and the determination of fault severity is more quantitative. Fusing the visual data collected by event camera 1 with the sensor data collected by sensor 2 can improve the accuracy of fault location and the reliability of fault level determination.

[0098] Preferably, the event camera 1 includes a cone-rod heterogeneous pixel array, which comprises multiple cone-shaped pixel units and multiple rod-shaped pixel units. The heterogeneous pixel array includes staggered cone-shaped and rod-shaped heterogeneous pixels, the number of which can be equal or unequal. Specifically, signals acquired by different types of cone-shaped heterogeneous pixels are classified and processed, and signals acquired by multiple cone-shaped heterogeneous pixels are processed centrally, responsible for high-resolution color perception, primarily used to capture the color features of faults; signals acquired by multiple rod-shaped heterogeneous pixels are processed centrally, with the rod pixels supporting spatiotemporal difference calculation, primarily used to locate structural faults. The cone + rod heterogeneous pixel array simulates the function of cone / rod cells in the human eye, drawing on the characteristics of the human visual system to form two complementary and comprehensive visual perception pathways.

[0099] Preferably, the cone-shaped + rod-shaped heterogeneous pixel array of the event camera 1 includes multiple cone-shaped heterogeneous pixels and multiple rod-shaped heterogeneous pixels. The multiple cone-shaped heterogeneous pixels include multiple red light cone-shaped heterogeneous pixels, multiple green light cone-shaped heterogeneous pixels, and multiple blue light cone-shaped heterogeneous pixels. Both the cone-shaped and rod-shaped heterogeneous pixels are composed of a photodetector, a dual-threshold comparator, and a microsecond-level timestamp circuit. The photodetector is used for high-sensitivity photoelectric conversion; the dual-threshold comparator is used to dynamically adjust the brightness change threshold to distinguish between normal light changes and fault events; the microsecond-level timestamp circuit is used to accurately mark the time of event occurrence for time sequence alignment in data fusion. The event camera 1 not only has high dynamic range and high resolution, but also maintains excellent perception under high-speed motion and extreme lighting conditions, making its monitoring performance in extreme environments of rail transit far superior to existing event cameras.

[0100] Preferably, the cone-shaped heterogeneous pixels have a receptive field of 4 micrometers, sensing red, green, and blue light information. The photodetectors of the cone-shaped heterogeneous pixels include color filters: red cone-shaped heterogeneous pixels include a red light filter, resolving to red light information; green cone-shaped heterogeneous pixels include a green light filter, resolving to green light information; and blue cone-shaped heterogeneous pixels include a blue light filter, resolving to blue light information. The rod-shaped heterogeneous pixels have a receptive field of 8 micrometers, sensing white light information. The photodetectors of the rod-shaped heterogeneous pixels include a white light filter, resolving to white light information. The rod-shaped heterogeneous pixels have embedded memory, supporting spatiotemporal difference calculations. The cone + rod-shaped heterogeneous pixel array has a resolution of 640×320, a dynamic range of 90-120dB, an event rate of 12-450MEPS, and a dual-threshold comparator threshold that can be dynamically adjusted within a 5%-20% brightness variation range. This configuration of the event camera 1 demonstrates superior performance in terms of quantum efficiency, dynamic range, response speed, power consumption, and bandwidth.

[0101] Preferably, the neuromorphic processing module 4 includes an event filtering unit, a sparse computation unit, and a temporary storage unit. The event filtering unit is configured to remove spurious events from the original event stream and output a formatted, valid event stream. The sparse computation unit constructs a three-layer lightweight model based on a spiking neural network, where the input layer is fully connected to the hidden layers, the hidden layers are sparsely connected to the input and output layers, and the output layer is fully connected to the hidden layers. It is configured to input the valid event stream into the pre-trained neural network model, and the neural network model outputs visual anomaly events after inference. The temporary storage unit is configured to store key event features without requiring a separate frame buffer.

[0102] Preferably, the event filtering unit removes pseudo-events from the original event stream based on a preset algorithm. The configuration is as follows: receiving the original event stream and formatting the event data within it. The filtering unit of the neuromorphic processing module 4 receives the original event stream output by the event camera 1 and defines each event data point as an event format: E = (x, y, t, p). Where: x and y are pixel coordinates, t is a timestamp, and p is the brightness polarity. The formatted, valid event stream provides a good foundation for subsequent processing. A spatial neighborhood window and a temporal neighborhood window are constructed. For each decision event, a spatial neighborhood window is constructed, containing multiple events that are spatially adjacent pixels to the decision event. For each decision event, a temporal neighborhood window is constructed, containing multiple events that are temporally adjacent to the decision event. The original event stream is traversed to obtain associated events within the spatial and temporal neighborhood windows. For the formatted original event stream, all events are traversed, conditional judgments are performed on all events, and all associated events within the spatial and temporal neighborhood windows are counted. Perform brightness polarity consistency checks on associated events to remove spurious events.

[0103] Preferably, the event filtering unit is implemented using an embedded microcontroller (MCU), configured to define each event data point as an event format: E = (x, y, t, p). Where: x and y are pixel coordinates, t is a timestamp, and p is the brightness polarity. The unit for pixel coordinates x and y is pixels, corresponding to a range of 0 ≤ x ≤ 639 and 0 ≤ y ≤ 319 for a 640×320 pixel array. The unit for the timestamp t is μs, with a precision of 1 μs. The value of the brightness polarity p is +1 indicating brighter light and -1 indicating darker light. For each event to be decided, E0=(x0, y0, t0, p0), a 3×3 spatial neighborhood window is constructed. The pixel range of the spatial neighborhood window is: S = {(x, y) |x0-1 ≤ x ≤ x0+1, y0-1 ≤ y ≤ y0+1}, where x and y are pixel coordinates. This spatial neighborhood window includes events at 9 pixel positions. Simultaneously, the temporal neighborhood window is set to [t0-T]. t , t0+T t ], where T t The temporal neighborhood threshold is set to 8 μs. For the event to be determined, E0 = (x0, y0, t0, p0), events that meet the conditions within the formatted original event stream are obtained. The conditions are: spatially within S = {(x, y) | x0-1 ≤ x ≤ x0+1, y0-1 ≤ y ≤ y0+1}, where x and y are pixel coordinates; and temporally within [t0-T]. t , t0+T t Within ], of which T t The temporal neighborhood threshold is used. Event acquisition is implemented through dedicated logic circuits, traversing 3×3 pixel coordinates in parallel, with a search latency ≤10ns / event. For an event to be judged, E0=(x0, y0, t0, p0), the number of related events N that meet the conditions is counted. If N≥2 (there are at least 2 spatiotemporally related events), and the proportion of events with the same polarity as p0 among the related events is ≥60%, then E0 is judged as a valid event; if N<2, or the proportion of events with the same polarity is <60%, then E0 is judged as a pseudo-event and marked as to be removed. The time taken for a single event judgment is ≤50ns, ensuring real-time processing of the event stream (supporting a maximum event rate of 450MEPS). Valid events are sorted in ascending order of timestamp, and a formatted valid event stream E_valid = {(x, y, t, p) | set of valid events} is output.

[0104] Preferably, the comparison of time neighborhood thresholds is achieved through a high-speed timer and a comparator, supporting real-time dynamic adaptive adjustment of the dynamic threshold.

[0105] Preferably, the spatial neighborhood window and / or temporal neighborhood window are dynamically and adaptively adjusted in real time. The event density D (unit: MEPS, where MEPS represents events per second) of the current environment is calculated in real time. When D is less than the preset minimum value, the system may be in an environment with extremely low light, where events are often dominated by noise. Enlarging the temporal neighborhood threshold and / or spatial neighborhood window can lower the threshold for valid event judgment and avoid erroneous deletion of valid events. When D is greater than the preset maximum value, events may be dense due to target movement, etc. Reducing the temporal neighborhood threshold and / or spatial neighborhood window helps to strictly filter related events and reduce redundancy.

[0106] Preferably, the event density D of the current environment is statistically analyzed in real time. When D < 5 MEPS, the time neighborhood threshold T is set. t Adjust to 10 μs, maintain a spatial neighborhood of 3×3; when 5 MEPS ≤ D ≤ 50 MEPS, maintain T. t =8μs; when D>50MEPS (target motion in low light causes dense events), T t Adjust to 5μs.

[0107] Preferably, the sparse computing unit is implemented using a spiking neural network chip. The spiking neural network chip adopts a RISC-V architecture, integrating no fewer than 128 spiking neuron cores, with a weight storage capacity of ≥512KB, achieving lightweight deployment. It boasts a computing power of ≥0.5 TOPS and a power consumption of ≤20mW, achieving high computing power and low power consumption. Inference latency is ≤1ms, and the total process time is ≤0.8ms, accounting for 60% of the total latency, meeting real-time early warning requirements. Based on pulse signal processing, it exhibits strong anti-interference capabilities, adapts to event flow fluctuations in high-speed motion scenarios, and demonstrates high robustness.

[0108] Preferably, the input layer feature encoding of the sparse computing unit is implemented through hardware logic circuits, with event block segmentation and grid statistics taking ≤0.2ms. The output layer integration and decision-making of the sparse computing unit are implemented through an embedded MCU, with decision-making taking ≤0.1ms. This setup allows for low time consumption at each step, keeping the overall monitoring process time low and ensuring the real-time performance of the rail transit monitoring device.

[0109] Preferred, Figure 6 The publicly disclosed rail transit monitoring device also includes a rail transit interface module 3 and a safety management platform 6. The rail transit interface module 3 includes a MIPI-CSI2 synchronization interface and a USB-C3.0 data interface, configured to realize signal transmission and time alignment; the safety management platform 6 is configured to receive monitoring results in real time, and to perform real-time response and archiving based on the monitoring results.

[0110] Preferably, the board size of the rail transit interface module 3 is ≤35mm×35mm×15mm, which is suitable for compact installation scenarios such as train roofs and tunnel sidewalls. The rail transit interface module 3 supports an "always on" wake-up mode, with a minimum standby power consumption of ≤0.1mW, enabling long-term unattended operation for more than one year.

[0111] Preferably, the safety management platform 6 encapsulates monitoring results and early warning signals into a standardized data frame Frame = (Header, X_fusion, Y_fusion, t_fusion, Grade_fusion, Alert, Checksum). Real-time early warnings are sent to the train ECU via the MIPI-CSI2 synchronization interface, and archived data is transmitted to the platform via the USB-C3.0 interface. The real-time early warning level signals and audible / visual alarm trigger commands are compatible with the existing train alarm system. The rail transit interface module 3 interfaces with the existing rail transit ECU and safety management platform 6, avoiding modifications to existing equipment. The archived data transmitted to the platform is a standardized data frame, stored in timestamp order, and supports backtracking for more than one year. The safety management platform 6 has the function of interactive fault location maps, grade identification, occurrence time, historical trends, and other visualized information. When there is no early warning, the transmission frequency is 1Hz, retaining only the heartbeat signal; when an early warning occurs, it switches to a 1kHz high-frequency transmission, dynamically adjusting the transmission frequency. Combined with an event-driven mechanism, the long-term power consumption of the entire system is reduced to less than or equal to 20mW.

[0112] The embodiments of this disclosure were tested on a high-speed rail pantograph-catenary simulation platform, simulating a speed of 350 km / h, strong tunnel light, and low-light nighttime environments. The results compared with a traditional CMOS camera solution are shown in the table below:

[0113] Test metrics CMOS camera solution This disclosure embodiment Improvement effect Arc detection delay of pantograph-catenary circuit 45ms 0.8ms Reduced by 98.2% Low light environment spurious event rate 25% 7% 72% reduction Device operating power consumption 3.5W 18mW Reduced by 99.5% accuracy of pantograph-catenary fault warning 78% 96% An increase of 23.1% Contact wire wear positioning error 15mm 4mm Reduced by 73.3% Plate volume 60mm×40mm×25mm 32mm×32mm×14mm Reduced by 68.5%

[0114] The test scenario and result comparison chart is as follows: Figure 7A and Figure 7B As shown, 7A is an overexposed image of a traditional CMOS camera under strong tunnel light. Figure 7B This is an example of imaging according to an embodiment of the present disclosure. In addition to the aforementioned advantages in environmental adaptability, real-time performance, accuracy, and low power consumption, this embodiment also has advantages in image clarity and fault arc identification marking.

[0115] According to an embodiment of this disclosure, a computer device is also provided, comprising: a memory for non-temporarily storing computer-executable instructions; and a processor for running the computer-executable instructions, wherein the computer-executable instructions, when run by the processor, execute the rail transit monitoring method of any of the preceding embodiments.

[0116] According to embodiments of this disclosure, a non-temporary storage medium is also provided for storing computer-executable instructions non-temporarily, characterized in that when the computer-executable instructions are executed by a computer, the rail transit monitoring method described in any of the preceding claims is executed.

[0117] According to an embodiment of this disclosure, a computer program product is also provided, including a computer program, characterized in that the computer program, when executed by a processor, implements the rail transit monitoring method described in any of the preceding claims.

[0118] The following points also need to be explained:

[0119] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure, and other structures can be referred to the general design.

[0120] (2) For clarity, the thickness of devices, layers, or regions is enlarged or reduced in the drawings used to describe embodiments of the present disclosure, i.e., these drawings are not drawn to scale. It will be understood that when an element such as a layer, film, region, or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0121] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0122] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure shall be determined by the scope of the claims.

Claims

1. A method for monitoring rail transit, comprising the following steps: S1: Collect brightness changes within the monitored scene, generate a raw event stream, and simultaneously collect sensor data within the monitored scene; S2: Remove pseudo-events from the original event stream and output a formatted valid event stream; S3: Input the effective event stream into a pre-trained neural network model, and the neural network model outputs visual abnormal events after inference; S4: Perform data fusion on the visual anomaly event and the sensor data, and output the monitoring results.

2. The rail transit monitoring method according to claim 1, characterized in that, It also includes S5: transmitting the monitoring results to the security management platform in real time, and the security management platform responding and archiving in real time based on the monitoring results.

3. The rail transit monitoring method according to claim 1, characterized in that, Step S2 includes: S21: Receive the original event stream and format the event data therein; S22: Construct spatial and temporal neighborhood windows; S23: Traverse the original event stream to obtain the associated events within the spatial neighborhood window and the temporal neighborhood window; S24: Perform a consistency judgment on the brightness and polarity of the associated events, remove the pseudo-events, and retain the valid events; S25: Sort the valid events in ascending order of timestamps and output a formatted stream of the valid events.

4. The rail transit monitoring method according to claim 3, characterized in that, In step S22, the spatial neighborhood window and / or temporal neighborhood window are dynamically and adaptively adjusted in real time.

5. The rail transit monitoring method according to claim 1, characterized in that, Step S3 includes: S31: Divide the effective event stream into continuous event blocks according to a time window, count the spatial distribution of the event blocks in the pixel array, and divide the pixel array into n spatial grids; S32: Calculate the number of events for each spatial grid, normalize it, and use it as the initial input for the n neurons in the input layer of the neural network model; S33: The hidden layer of the neural network model includes m neurons (m>n), and each input layer neuron is connected to the neurons in the hidden layer. The leakage integral-fire model is used for pulse transmission. S34: The output layer of the neural network model includes two neurons. The two neurons integrate the pulse signals within the time window and output the visual abnormality event based on the integration result.

6. The rail transit monitoring method according to claim 1, characterized in that, Step S4 includes: S41: Receive the visual anomaly event and the sensor data; S42: Using the clock that generated the original event stream as a reference, a calibration pulse is sent to the sensor, and the sensor corrects its timestamp error based on the calibration pulse; S43: Search the sensor data within the time window adjacent to the visual anomaly event and establish data association pairs; S44: Merge the data in the data association pair and output the monitoring result.

7. The rail transit monitoring method according to claim 1, characterized in that, Prior to step S3, the neural network model is pre-trained: Step 01: Collect multiple sets of rail transit event samples in different scenarios, including multiple sets of normal event samples and multiple sets of fault event samples; Step 02: Input the multiple sets of rail transit event samples into the neural network model in batches, and use the impulse temporal dependence plasticity learning rule to perform multiple rounds of iteration until the model accuracy converges to a preset threshold; Step 03: Perform redundant connection pruning on the neural network model, retain the links of the core connections, and compress the total number of model parameters.

8. A rail transit monitoring device, comprising: An event camera is configured to capture and monitor brightness changes within a scene and generate a raw event stream. The sensor is configured to simultaneously acquire sensor data within the monitored scene along with the event camera; The brain-like processing module is configured to remove pseudo-events from the original event stream, output a formatted valid event stream, input the valid event stream into a pre-trained neural network model, and output visual abnormal events after inference. The data fusion module is configured to fuse the visual anomaly event with the sensor data and output the monitoring results.

9. The rail transit monitoring device according to claim 8, characterized in that, The event camera includes a cone-rod heterogeneous pixel array, which comprises multiple cone-shaped pixel units and multiple rod-shaped pixel units; The neuromorphic processing module includes an event filtering unit, a sparse computing unit, and a temporary storage unit; The event filtering unit is configured to remove pseudo-events from the original event stream and output a formatted valid event stream. The sparse computing unit constructs a three-layer lightweight model based on a spiking neural network, including an input layer, a hidden layer, and an output layer. The input layer is fully connected to the hidden layer, the hidden layer is sparsely connected to the input layer and the output layer, and the output layer is fully connected to the hidden layer. It is configured to input the effective event stream into the pre-trained neural network model, and the neural network model outputs visual abnormal events after inference. The temporary storage unit is configured to store key event features and does not require a separate frame buffer.

10. The rail transit monitoring device according to claim 8, characterized in that, It also includes a rail transit interface module and a safety management platform. The rail transit interface module includes a synchronization interface and a data interface, configured to realize signal transmission and time alignment. The security management platform is configured to receive the monitoring results in real time and to perform real-time response and archiving based on the monitoring results.

11. A computer device, comprising: Memory is used to store non-temporary executable instructions for a computer. And a processor for running the computer-executable instructions, wherein the computer-executable instructions are executed by the processor during execution of the rail transit monitoring method according to any one of claims 1 to 7.

12. A non-transitory storage medium for non-transitory storage of computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a computer, the rail transit monitoring method according to any one of claims 1 to 7 is performed.

13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the rail transit monitoring method according to any one of claims 1 to 7.

Images