Fault diagnosis method and system for airport terminal pedestrian path and electronic equipment

Abstract

The invention relates to a fault diagnosis method and system for a pedestrian path of an airport terminal and electronic equipment, and the method comprises the following steps: arranging collectors at all key parts of the pedestrian path, and obtaining the data of the collectors; preprocessing the data of the collector, integrating the multi-modal features of the data of the collector through a feature level fusion algorithm, and carrying out time sequence modeling according to the multi-modal features; based on time sequence modeling, obtaining a health degree index of the pedestrian path, and performing fault type and severity level evaluation; the health degree index and the evaluation result are uploaded to an upper computer system, result display and fault alarm are carried out, and a diagnosis model is constructed; and carrying out collaborative optimization by adopting a federated learning network, and carrying out cross-scene optimization upgrading on the diagnosis model. According to the diagnosis method, real-time and accurate monitoring and fault diagnosis can be carried out on the running state of the pedestrian path; and meanwhile, a federated learning network of cross-terminal equipment is constructed, and the operation data security of an airline company and an airport is ensured through a node parameter interaction optimization model.

Description

Technical Field

[0001] This invention belongs to the field of terminal building pedestrian walkway technology, and relates to intelligent operation and maintenance of pedestrian walkways, especially a fault diagnosis method, system and electronic equipment for terminal building pedestrian walkways. Background Technology

[0002] In modern air transport, airport terminals serve as crucial hubs for passenger travel, and their internal pedestrian walkways are key components ensuring efficient and convenient passenger access. However, due to the long-term high-intensity, continuous operation of these walkways, coupled with complex and ever-changing passenger flow and operating conditions, equipment malfunctions occur frequently. This not only affects the passenger travel experience but can also lead to congestion and delays within the terminal.

[0003] Currently, traditional methods for diagnosing faults in pedestrian walkways typically rely on manual inspections and simple monitoring of equipment operating parameters, which have the following problems:

[0004] 1. Manual inspection has disadvantages such as low efficiency, strong subjectivity, and inability to monitor equipment operating status in real time, making it difficult to detect and deal with problems in the early stages of failure.

[0005] 2. Existing equipment operation parameter monitoring systems can only collect a few key data points and lack a complete multimodal fusion and time-series analysis process. They cannot fully perceive the equipment's operating status and have obvious defects in adaptability to the working conditions of the monitoring scenario. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a comprehensive, safe and reliable fault diagnosis method, system and electronic equipment for monitoring pedestrian walkways in airport terminals.

[0007] To solve the above problems, the technical solution of the present invention is as follows:

[0008] A fault diagnosis method for pedestrian walkways in an airport terminal includes the following steps:

[0009] Data collectors were installed at key locations along the pedestrian walkway to acquire data.

[0010] The data from the data collector is preprocessed, and the multimodal features of the data collector are integrated through a feature-level fusion algorithm. Time series modeling is then performed based on the multimodal features.

[0011] Based on time series modeling, the health index of pedestrian walkways is obtained, and the fault type and severity level are evaluated.

[0012] The health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a diagnostic model is built.

[0013] Federated learning networks are used for collaborative optimization to upgrade the diagnostic model across different scenarios.

[0014] In a further embodiment, the data acquisition device includes a vibration sensor, a temperature sensor, a microphone array, and an image acquisition device.

[0015] In a further embodiment, the vibration sensor is fixed to the motor housing of the pedestrian walkway and the drive chain of the pedestrian walkway.

[0016] The temperature sensor is fixedly installed on the motor windings of the pedestrian walkway and on the bearing housing of the pedestrian walkway;

[0017] The microphone array is fixedly installed under the handrail of the pedestrian walkway, which is suitable for acquiring noise signals;

[0018] The image acquisition device is fixedly installed at the entrance, exit, and middle of the pedestrian walkway, and is suitable for acquiring video data.

[0019] In a further embodiment, the collector data is preprocessed, and the multimodal features of the collector data are integrated through a feature-level fusion algorithm. Temporal modeling is then performed based on these multimodal features, specifically including:

[0020] Preprocessing of the data from the data acquisition unit includes:

[0021] Remove low-frequency interference from the vibration signal data obtained by the data acquisition device:

[0022]

[0023] In the formula, H a (s) is the transfer function of the filter; n is the filter order; s is the complex frequency variable; s k is the filter pole; k is the k-th node of the filter;

[0024] The output signal x after Butterworth high-pass filtering fil (t)(after removing low-frequency interference) satisfies:

[0025] x fil (t)=L -1 [H a [(s)·X(s)];

[0026] In the formula, x fil (t) represents the filtered output signal; L -1 [·] represents the inverse Laplace transform; H a X(s) is the transfer function of the filter; X(s) is the Laplace transform of the original vibration signal.

[0027] The temperature data obtained by the data acquisition unit is smoothed by a sliding window mean filter. The calculation formula is as follows:

[0028]

[0029] In the formula, n represents a discrete time point; y s (n) represents the output signal at discrete time point n after sliding window mean filtering; L is the length of the sliding window; i is the index variable for summation; x s (ni) represents the temperature signal sample value at discrete time point n after subtracting the summed index variable;

[0030] The sound signal obtained by the acquisition device is denoised using wavelet basis decomposition, specifically including:

[0031] Wavelet decomposition was performed on the collected raw sound signal:

[0032]

[0033]

[0034] In the formula, A k (n) represents the approximation coefficients of the k-th layer; h(m) represents the low-pass filter coefficients of the wavelet decomposition; n represents the sample index of the k-th layer; m represents the coefficient index; D j (n) represents the detail coefficients of the j-th layer; g(m) represents the high-pass filter coefficients of the wavelet decomposition; This represents the convolution operation;

[0035] For each layer of detail coefficient D j (n) Soft thresholding is applied, resulting in a higher detail factor. The following conditions must be met:

[0036]

[0037] In the formula, D represents the detail coefficients after thresholding. j (n) represents the detail coefficients of the j-th layer; λ j The threshold value for the j-th layer is denoted by 'j'; sign(·) is the sign function.

[0038] The processed coefficients are reconstructed using wavelet reconstruction, and the calculation formula is as follows:

[0039]

[0040] In the formula, x * (n) represents the denoised audio signal; A k (n) represents the approximation coefficients for the k-th layer; represents the detail coefficients of each layer after thresholding; j represents the layer number of the detail coefficients; k represents the layer number of the approximation coefficients.

[0041] The video data is sequentially converted to grayscale and histogram equalized. The U-Net model is used to segment the region of interest, and the YOLOv5 algorithm is used to identify passenger flow and abnormal behavior.

[0042] In a further embodiment, the data collector data is preprocessed, and the multimodal features of the data collector data are integrated through a feature-level fusion algorithm. Temporal modeling is then performed based on the multimodal features. The method further includes:

[0043] An attention mechanism is used to dynamically adjust the modal weights of the data collector. The fusion strategy for each modal weight is as follows:

[0044] F = α·F vib +β·F tem +γ·F nio +δ·F vid ;

[0045] α+β+γ+δ=1;

[0046] In the formula, F is the fused multimodal feature vector; F vib F is the feature vector of the vibration signal data; tem F is the feature vector of the temperature data. noi F is the feature vector of the noisy data; vid denoted as the feature vector of the video data; α represents the attention weight for the vibration mode; β represents the attention weight for the temperature mode; γ represents the attention weight for the noise mode; and δ represents the attention weight for the video mode.

[0047] The fused feature vectors of each modality are input into a bidirectional LSTM network for time series modeling.

[0048] In a further embodiment, based on time-series modeling, the health index of the pedestrian walkway is obtained, and the fault type and severity level are evaluated, specifically including:

[0049] Outputting a health index for pedestrian walkways based on a CNN model:

[0050] H=σ(W3·σ(W2·σ(W1·F+b1)+b2)+b3);

[0051] In the formula, H is the health index; σ is the ReLU activation function; W1, W2, and W3 are the convolution weights; and b1, b2, and b3 are the bias terms.

[0052] An improved SVM algorithm is used to classify fault types. For any health feature vector H of the test sample, its decision score belonging to fault category j is calculated by the following formula:

[0053]

[0054] In the formula, fj (H) is the decision function for fault category j; H is the health index feature vector of the input sample; α i For Lagrange multipliers; y i Let be the true class label corresponding to the i-th support vector; N is the total number of support vectors. b is the RBF kernel function; b is the bias term; j is the fault type;

[0055] The fault types are divided into three categories: mechanical faults, electrical faults, and control faults. The severity levels are divided into three levels: minor, moderate, and severe.

[0056] In a further embodiment, the health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a diagnostic model is constructed, specifically including:

[0057] Fault alarms should be responded to in a graded manner according to their severity level:

[0058] Minor faults: recorded in the system log;

[0059] Medium-level fault: An alarm icon will be displayed in the results presentation;

[0060] Critical fault level: Triggers audible and visual alarms around the fault location and sends push notification messages.

[0061] In a further embodiment, a federated learning network is used for collaborative optimization to upgrade the diagnostic model across scenarios, specifically including:

[0062] The local model parameters for the pedestrian walkway have been updated as follows:

[0063]

[0064] In the formula, The local model parameters are updated for the k-th scene; θ k Here are the parameters of the local diagnostic model for the k-th scene; η is the learning rate; |D k | represents the number of local dataset samples for the k-th scene; The gradient of the loss function L with respect to the model parameters; f(x; θ) k ) represents the model's predicted value; y represents the true label;

[0065] Federation server global model aggregation:

[0066]

[0067] In the formula, θ g For global model parameters of the federated server; The sample size weights for the k-th scene; These are the local model parameters updated for the k-th scene;

[0068] Cross-scene model optimization iteration:

[0069] Each scenario receives global model parameters and uses them as initial parameters for a new round of local training, achieving cross-scenario optimization through T iterations:

[0070]

[0071] In the formula, These are the global model parameters for the (t+1)th round; t represents the sample size weight for the k-th scene; t is the iteration round. Let be the global model parameters for round t; η be the learning rate; |D k | represents the number of local dataset samples for the k-th scene; Let L be the gradient of the loss function L with respect to the model parameters; y represents the model's predicted value; y represents the true label.

[0072] A fault diagnosis system for a terminal building pedestrian walkway includes:

[0073] Data acquisition module: Collects data through vibration sensors, temperature sensors, microphone arrays and image acquisition devices to obtain real-time data on mechanical vibration, temperature changes, acoustic characteristics and video images;

[0074] Preprocessing and fusion module: used to preprocess the acquired real-time data, realize multimodal feature weighted fusion through attention mechanism, and perform temporal modeling in combination with bidirectional LSTM algorithm;

[0075] Fault diagnosis and evaluation module: used for data transmission and interaction, transmitting health indicators and fault evaluation results to the host computer system, and performing fault result visualization and graded alarm processing;

[0076] Federated Learning Optimization Module: Combines federated learning networks for collaborative optimization, enabling cross-scenario upgrades to the diagnostic model.

[0077] An electronic device, comprising:

[0078] At least one processor;

[0079] Memory that is communicatively connected to at least one processor;

[0080] The memory stores instructions that can be executed by at least one processor, and the instructions, when executed by at least one processor, enable at least one processor to:

[0081] Data collectors were installed at key locations along the pedestrian walkway to acquire data.

[0082] The data from the data collector is preprocessed, and the multimodal features of the data collector are integrated through a feature-level fusion algorithm. Time series modeling is then performed based on the multimodal features.

[0083] Based on time series modeling, the health index of pedestrian walkways is obtained, and the fault type and severity level are evaluated.

[0084] The health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a diagnostic model is built.

[0085] Federated learning networks are used for collaborative optimization to upgrade the diagnostic model across different scenarios.

[0086] Compared with the prior art, the beneficial effects of the present invention are:

[0087] 1. This diagnostic method includes an edge computing device, a federated learning network, and a host computer system. The edge computing device collects multimodal data such as vibration, temperature, noise, and video through distributed sensors. After preprocessing, multimodal fusion and time-series analysis techniques are used, combined with a deep learning model, to diagnose faults and output health evaluation results, achieving real-time and accurate monitoring and fault diagnosis of the pedestrian walkway's operational status. Simultaneously, a federated learning network is constructed across terminal equipment, optimizing the model through node parameter interaction to ensure the security of operational data for airlines and the airport. This system enables real-time equipment status monitoring, accurate fault diagnosis, and tiered alarms, improving the efficiency and reliability of terminal pedestrian walkway operation and maintenance.

[0088] 2. This diagnostic method adopts multimodal fault diagnosis technology. By collecting multimodal information such as vibration signals, temperature signals, noise signals, passenger flow, and abnormal passenger behavior, and using multimodal fusion and time series analysis methods, it can more comprehensively and accurately reflect the operating status of the pedestrian walkway, effectively improve the accuracy and reliability of fault diagnosis, and reduce misjudgment and missed judgment.

[0089] 3. The edge computing device used in this diagnostic method collects and processes data locally in real time, greatly reducing data processing time delay and enabling real-time monitoring of the pedestrian walkway's operational status. Once a fault is detected, an alarm mechanism can be quickly triggered to notify staff for timely handling, preventing further escalation of the fault and ensuring the normal operation of the equipment and the safe travel of passengers.

[0090] 4. This diagnostic method constructs a federated learning network across terminal equipment, where data from each terminal is processed and trained locally without uploading raw data, effectively ensuring the security and privacy of terminal data information and meeting the stringent data security requirements of airports and other similar venues.

[0091] 5. The host computer system of this diagnostic method includes a web-based host computer access system, which allows administrators to view equipment status and fault information anytime and anywhere. The fault alarm and display module classifies and processes faults according to their severity, which helps maintenance personnel to quickly locate and resolve problems, improve the efficiency of maintenance work, and reduce maintenance costs. Attached Figure Description

[0092] Figure 1 A flowchart of a fault diagnosis method for a pedestrian walkway in an airport terminal.

[0093] Figure 2 A schematic diagram of the data acquisition device layout for a fault diagnosis method for a pedestrian walkway in an airport terminal.

[0094] Figure 3 A schematic diagram of the network architecture of a fault diagnosis system for a pedestrian walkway in an airport terminal.

[0095] Figure 4 This is a schematic diagram of the workflow of a fault diagnosis system for a pedestrian walkway in an airport terminal.

[0096] Figure 5 A flowchart of a federated learning network for a fault diagnosis method for pedestrian walkways in an airport terminal. Detailed Implementation

[0097] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0098] Example 1:

[0099] A fault diagnosis method for pedestrian walkways in airport terminals, such as... Figures 1 to 5 As shown, it includes the following steps:

[0100] S101. Install data collectors at key locations along the pedestrian walkway to acquire data from the collectors:

[0101] As the front-end sensing link in the fault diagnosis process, sensors are deployed and data is collected. By deploying several vibration sensors, several temperature sensors, several microphone arrays, and several high-definition image acquisition devices at key locations along the pedestrian walkway, the synchronous acquisition of multi-source physical signals is achieved.

[0102] like Figure 2 As shown, a distributed sensor network enables real-time monitoring of multiple physical quantities. This network includes several vibration sensors, temperature sensors, microphone arrays, and high-definition image acquisition devices. Several triaxial vibration sensors are installed at intervals on key locations such as the axial end face of the motor housing and the connection point of the drive chain's drive wheel on the pedestrian walkway. These sensors are connected to the analog input module of an edge computing device via shielded twisted-pair cables to analyze the mechanical state of the equipment and detect potential faults such as wear and loosening of components. Infrared temperature sensors are used, with several sensors fixed at intervals on the motor windings and the outer ring of the motor bearing housing. The temperature data acquired by the sensors is transmitted to the edge computing device via an RS485 bus to collect temperature signals from areas prone to overheating on the pedestrian walkway. By analyzing temperature change trends, overheating faults can be identified, preventing equipment damage caused by overheating. Multi-channel microphone arrays are evenly spaced below the handrails on both sides of the pedestrian walkway, and audio acquisition cards are used to synchronously acquire acoustic signals. By analyzing the frequency, intensity, and other characteristics of the noise, abnormal friction and collisions that may occur during equipment operation can be identified. High-definition image acquisition devices are fixedly installed directly in front of the entrance, exit, and midway point of the pedestrian walkway. These devices collect pedestrian flow and abnormal passenger behavior information, including falls, fighting, jumping, leaning out, walking against traffic, and carrying oversized luggage. The data is connected to an edge computing device via Ethernet and an industrial bridge. Through image recognition and analysis technology, the device counts the number of passengers passing through the walkway per unit time to determine if the current passenger flow is within the device's normal capacity. It also identifies any abnormal passenger behavior. This information assists in assessing the pedestrian walkway's operational status. All sensors are synchronized using fiber optic timing. The raw data acquired by each sensor is temporarily stored locally in the edge computing device in the format of "Device ID-Timestamp-Physical Quantity."

[0103] S103. Preprocess the data from the data collector, integrate the multimodal features of the data collector through a feature-level fusion algorithm, and perform time series modeling based on the multimodal features:

[0104] As a core data processing step in the fault diagnosis process, data preprocessing and multimodal fusion are performed. Vibration signal data, temperature data, noise data, and video data collected by several vibration sensors, temperature sensors, microphone arrays, and high-definition image acquisition devices are preprocessed in a targeted manner. Then, feature-level fusion algorithms are used to integrate multimodal features and perform time-series modeling.

[0105] The data preprocessing includes the collected vibration signal data, temperature data, noise data, and video data. For vibration, temperature, and noise signals, filtering is performed to remove noise interference and improve signal quality. For video data, image enhancement and deblurring are performed to improve image clarity for subsequent image recognition and analysis.

[0106] The vibration signal data is preprocessed by removing low-frequency interference using a Butterworth high-pass filter, as shown in formula (1):

[0107]

[0108] In the formula, H a (s) is the transfer function of the filter, n is the filter order; s is a complex frequency variable, and s = jω, where j is the imaginary unit, ω is the angular frequency, ω = 2πf, and f is the signal frequency; k is the k-th node of the filter; s k For the filter poles, satisfying Ensure that the filter has a flat amplitude-frequency response within the passband.

[0109] The output signal x after Butterworth high-pass filtering fil (t) is shown in formula (2):

[0110] x fil (t)=L -1 [H a (s)·X(s)] (2)

[0111] In the formula, x fil (t) represents the filtered output signal; L -1 [·] represents the inverse Laplace transform; H a X(s) is the transfer function of the filter; X(s) is the Laplace transform of the original vibration signal x(t).

[0112] For the temperature data, a sliding window mean filter is used to smooth the noise, and the filtered output signal is calculated as shown in formula (3):

[0113]

[0114] In the formula, n represents a discrete time point; y s(n) represents the output signal after the discrete-time point n is processed by the sliding window mean filter; L is the length of the sliding window, i.e., the number of consecutive sampling points used to calculate the average; i is the index variable for summation; x s (n) represents the noisy raw temperature signal sample value acquired at discrete time point n; x s (ni) represents the temperature signal sample value at discrete time point n after subtracting the summed index variable.

[0115] The original sound signal is absorbed and then decomposed using wavelet basis functions to achieve noise reduction, as shown in formulas (4) and (5):

[0116]

[0117] In the formula, A k (n) represents the approximation coefficients of the k-th layer; h(m) represents the low-pass filter coefficients of the wavelet decomposition; n represents the sample index of the k-th layer; m represents the coefficient index; A0(n) = x(n) represents the original sound signal; D j (n) represents the detail coefficients of the j-th level, j = 1, 2, ..., k; g(m) represents the high-pass filter coefficients of the wavelet decomposition; This represents the convolution operation.

[0118] For each layer of detail coefficient D j (n) Soft thresholding is applied, resulting in a higher detail factor. As shown in formula (6):

[0119]

[0120] In the formula, D represents the detail coefficients after thresholding. j (n) represents the detail coefficients of the j-th layer; λ j is the threshold of the j-th layer; sign(·) is the sign function.

[0121] The processed coefficients are reconstructed using wavelet reconstruction, as shown in formula (7):

[0122]

[0123] In the formula, x * (n) represents the denoised audio signal; A k (n) represents the approximation coefficients for the k-th layer; represents the detail coefficients of each layer after thresholding; j represents the layer number of the detail coefficients; and k represents the layer number of the approximation coefficients.

[0124] The video data is sequentially converted to grayscale and histogram equalized. The U-Net model is used to segment the region of interest, and the YOLOv5 algorithm is used to identify passenger flow and abnormal behavior. The unit of passenger flow is people / minute.

[0125] After data preprocessing, the weights of each modality are dynamically adjusted using an attention mechanism. The multimodal features of the data collector are integrated through a feature-level fusion algorithm. The fusion strategy is defined as shown in formulas (8) and (9):

[0126] F = α·F vib +β·F tem +γ·F nio +δ·F vid (8)

[0127] α+β+γ+δ=1 (9)

[0128] In the formula, F is the fused multimodal feature vector; F vib F is the feature vector of the vibration signal data; tem F is the feature vector of the temperature data. noi F is the feature vector of the noisy data; vid is the feature vector of the video data; α, β, γ, and δ are the attention weights for vibration, temperature, noise, and video modality, respectively; δ is set to 0.15 by default, and when the passenger flow is greater than 50 people / minute, δ is set to 0.3, and α is reduced accordingly.

[0129] The fused feature vectors are input into a bidirectional LSTM network for time series modeling. The bidirectional LSTM network learns the feature dependencies over the past ten minutes and outputs fused features that include the time dimension. Combined with time series analysis methods, the time series characteristics of the data are considered to explore the patterns and trends of data changes over time, providing more comprehensive and accurate data support for health assessment.

[0130] S105. Based on time-series modeling, obtain the health indicators of pedestrian walkways and evaluate the fault type and severity level:

[0131] As a core decision-making step in the fault diagnosis process, fault diagnosis and health assessment are performed. The health assessment model is built based on a convolutional neural network, and the fault diagnosis uses a support vector machine classification algorithm. Based on the fused time-series feature data, the intelligent model outputs the health index of the pedestrian walkway and completes the identification of fault type and severity level.

[0132] The health index of the pedestrian walkway is output based on the CNN model, as shown in formula (10):

[0133] H=σ(W3·σ(W2·σ(W1·F+b1)+b2)+b3) (10)

[0134] In the formula, H is the health index, with a value range of [0,1]. The closer the value is to 1, the healthier the device is; σ is the ReLU activation function; W1, W2, and W3 are convolution weights; and b1, b2, and b3 are bias terms.

[0135] S107. Upload the health indicators and evaluation results to the host computer system for result display and fault alarm, and construct the pedestrian walkway model:

[0136] As a result feedback link in the fault diagnosis process, it transmits diagnostic results and displays alarms. The health assessment results and fault information are transmitted to the host computer system for visualization and tiered alarm response.

[0137] The improved SVM algorithm is used to classify the fault types. For any health feature vector H of the test sample, the decision score of its belonging to fault category j is shown in formula (11):

[0138]

[0139] In the formula, f j (H) is the decision function for fault category j; H is the health index feature vector of the input sample; α i For Lagrange multipliers; y i Let be the true class label corresponding to the i-th support vector; N is the total number of support vectors. is the RBF kernel function; b is the bias term; j is an element in the fault category set (mechanical fault, electrical fault, control fault).

[0140] Faults are categorized into three types: mechanical faults, electrical faults, and control faults. Severity levels are classified as minor, moderate, and severe. Minor faults do not affect the operation of the pedestrian walkway; moderate faults cause partial functional limitations; and severe faults require immediate shutdown. The edge computing device transmits health indicators, fault types, and severity levels to a host computer system via the MQTT protocol. The host computer system includes a web-based access system, providing an authorized user interface for viewing device operating status and diagnostic results. It displays real-time health curves, fault history charts, and other information visually, allowing users to quickly understand the device's condition.

[0141] Fault alarms execute tiered responses:

[0142] Critical fault level: Triggers an audible and visual alarm near the equipment and simultaneously sends a text message to maintenance personnel to alert them and prompt them to take timely action;

[0143] Moderate level fault: A yellow alarm icon is displayed on the digital screen in the duty room to facilitate daily monitoring and recording by the duty personnel;

[0144] Minor faults: only recorded in the system log, without triggering audible or visual alarms.

[0145] S109. Employ federated learning networks for collaborative optimization to upgrade the pedestrian walkway model across different scenarios:

[0146] like Figure 5 As shown, as a model iteration step in the fault diagnosis process, the diagnostic model is optimized and upgraded across scenarios and data privacy is ensured through parameter interaction between multi-terminal edge nodes and the federated server.

[0147] The federated learning network consists of edge computing devices and federated servers in multiple terminals; the edge computing devices in each terminal act as federated learning nodes, performing data processing and model training locally, and only uploading the model parameters to the federated server, as shown in formula (12):

[0148]

[0149] In the formula, The local model parameters are updated for the k-th scene; θ k Here are the parameters of the local diagnostic model for the k-th scene; η is the learning rate; |D k | represents the number of local dataset samples for the k-th scene; The gradient of the loss function L with respect to the model parameters; f(x; θ) k ) represents the model's predicted value; y represents the true label.

[0150] After updating the local model, the federated learning node uploads the model to the federated server for model aggregation and updates the global model, as shown in formula (13):

[0151]

[0152] In the formula, θ g For global model parameters of the federated server; The sample size weights for the k-th scene; These are the updated local model parameters for the k-th scene.

[0153] The updated global model is distributed to each edge computing device. Each edge node uses local data to perform multiple iterations to improve the accuracy of cross-scenario fault diagnosis while ensuring data privacy, as shown in formula (14):

[0154]

[0155] In the formula, These are the global model parameters for the (t+1)th round; t represents the sample size weight for the k-th scene; t is the iteration round, t = 1, 2, ..., T; Let be the global model parameters for round t; η be the learning rate; |D k | represents the number of local dataset samples for the k-th scene; ▽L(·) represents the gradient of the loss function L with respect to the model parameters; y represents the model's predicted value; y represents the true label.

[0156] Through the above federated learning optimization formula, the diagnostic model can be upgraded collaboratively in different scenarios without sharing the original data, thereby improving the model's generalization ability and diagnostic accuracy in different scenarios.

[0157] Example 2:

[0158] A fault diagnosis system for pedestrian walkways in airport terminals, such as Figure 3 As shown, it includes:

[0159] Data acquisition module: used for synchronous acquisition of multi-source signals, through vibration sensors, temperature sensors, microphone arrays and high-definition image acquisition devices, to realize real-time perception of mechanical vibration, temperature changes, acoustic characteristics and video images;

[0160] Preprocessing and fusion module: used for data purification and feature integration. It uses algorithms such as bandpass filtering and wavelet denoising to process the raw data obtained from vibration sensors, temperature sensors, microphone arrays and high-definition image acquisition devices. It achieves multimodal feature weighted fusion through attention mechanism and combines bidirectional LSTM for time series modeling.

[0161] Fault diagnosis and evaluation module: used for health status assessment and fault location, outputs health index based on CNN model, and achieves accurate identification of fault type and level through improved SVM algorithm;

[0162] Data transmission and alarm module: used for diagnostic information interaction and response, transmitting health evaluation results and fault information to the host computer system to realize data visualization and hierarchical alarm handling;

[0163] Federated Learning Optimization Module: Used for cross-scenario model upgrades. Through parameter interaction between edge nodes and the federated server, it achieves collaborative optimization of the global model and improves the system's diagnostic adaptability.

[0164] like Figure 4 As shown, the terminal pedestrian walkway fault diagnosis system based on multimodal fusion and health assessment of the present invention includes the following main components in its structure:

[0165] The edge computing device, serving as the core processing unit of the system, integrates a data acquisition card, a GPU acceleration module, and an industrial communication interface to achieve real-time processing of multi-source data. The vibration sensor array is fixed to the motor housing and transmission chain connection via a magnetic base, while the temperature sensor is bolted to the truss and near the motor. Both are connected to the edge computing device via industrial cables.

[0166] Microphone arrays are arranged below the handrails on both sides of the pedestrian walkway and are connected to the audio acquisition module of the edge computing device via audio cables. High-definition image acquisition devices are installed at the entrance, exit, and middle sections via brackets and communicate with the edge computing device via Ethernet.

[0167] The web-based host computer is deployed in the airport operations and maintenance center. It receives data transmitted from the edge computing devices via the local area network and displays the device's operating status through a visual interface. Audible and visual alarm devices are installed on pillars near the pedestrian walkways, triggering differentiated alarms based on the fault level.

[0168] The federated server is deployed on the airport's private cloud platform and connects to edge computing devices in each terminal via encrypted communication links to achieve global model aggregation and distribution. The power supply module uses a UPS uninterruptible power supply to ensure that the system maintains normal operation for 1 hour in the event of a power outage.

[0169] The present invention also provides a computer device, including a processor component comprising at least one processor; a storage component comprising a memory; program instructions comprising a computer program stored in the memory; the processor component being configured to read and execute the program instructions in the storage unit to implement steps S101 to S109.

[0170] The present invention provides a computer program product including computer-executable instructions, which, when executed on a processor of an electronic device, enable the electronic device to perform steps S101 to S109.

[0171] This invention also provides a computer-readable storage medium storing computer program instructions. The storage medium includes non-volatile memory (e.g., solid-state drives, flash memory) or temporary storage media (e.g., memory cache). The computer-readable storage medium stores an independent product, which implements steps S101 to S109 in the form of software functional units. Essentially, the part of the technical solution of this invention that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute steps S101 to S109. The aforementioned storage media include, but are not limited to, semiconductor memories (e.g., ROM, RAM, Flash), magnetic storage media (e.g., hard disks, magnetic tapes), optical storage media (e.g., CDs, DVDs), and other physical storage devices capable of carrying program code (e.g., USB flash drives, portable hard drives, etc.).

[0172] The present invention also provides an information data processing terminal, which is used to provide a user input interface to implement steps S101 to S109 as described in the above method embodiments when executed on an electronic device. The information data processing terminal is not limited to mobile phones, computers, and switches.

[0173] The present invention also provides a fault diagnosis method for a pedestrian walkway in an airport terminal, comprising: a computing cluster containing multiple processor nodes, each node being configured with at least one multi-core processor; a distributed storage system for storing a 3D point cloud map and training dataset, as well as the required computer program; a network communication interface configured to receive real-time image streams transmitted by a terminal; and an in-memory database for caching intermediate data during the implementation of the invention; wherein the computing cluster executes the computer program to implement steps S101 to S109.

[0174] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A fault diagnosis method for pedestrian walkways in an airport terminal, characterized in that, Includes the following steps: Data collectors are installed at key locations along the pedestrian walkway to acquire data from the data collectors. The data from the data collector is preprocessed, and the multimodal features of the data collector are integrated through a feature-level fusion algorithm. Temporal modeling is then performed based on the multimodal features. Based on the time-series modeling, the health index of the pedestrian walkway is obtained, and the fault type and severity level are evaluated. The health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a pedestrian walkway model is constructed. A federated learning network is used for collaborative optimization to upgrade the pedestrian walkway model across different scenarios.

2. The fault diagnosis method for terminal building pedestrian walkways according to claim 1, characterized in that, The data acquisition device includes a vibration sensor, a temperature sensor, a microphone array, and an image acquisition device.

3. The fault diagnosis method for terminal building pedestrian walkways according to claim 2, characterized in that, The vibration sensor is fixedly mounted on the motor housing of the pedestrian walkway and on the transmission chain of the pedestrian walkway; The temperature sensor is fixedly installed on the motor winding of the pedestrian walkway and on the bearing seat of the pedestrian walkway; The microphone array is fixedly installed below the handrail of the pedestrian walkway and is suitable for acquiring noise signals; The image acquisition device is fixedly installed at the entrance, exit, and middle of the pedestrian walkway, and is suitable for acquiring video data.

4. The fault diagnosis method for terminal building pedestrian walkways according to claim 3, characterized in that, The data from the data collector is preprocessed, and its multimodal features are integrated using a feature-level fusion algorithm. Temporal modeling is then performed based on these multimodal features, specifically including: Preprocessing the data from the data collector includes: Low-frequency interference is removed from the vibration signal data obtained by the acquisition device: In the formula, H a (s) is the transfer function of the filter; n is the filter order; s is the complex frequency variable; s k is the filter pole; k is the k-th node of the filter; The output signal after Butterworth high-pass filtering is: x fil (t)=L -1 [H a (s)·X(s)]; In the formula, x fil (t) represents the filtered output signal; L -1 [·] represents the inverse Laplace transform; H a X(s) is the transfer function of the filter; X(s) is the Laplace transform of the original vibration signal. The temperature data obtained by the data acquisition device is smoothed by a sliding window mean filter. The calculation formula is as follows: In the formula, n represents a discrete time point; y s (n) represents the output signal at discrete time point n after sliding window mean filtering; L is the length of the sliding window; i is the index variable for summation; x s (ni) represents the temperature signal sample value at discrete time point n after subtracting the summed index variable; The sound signal obtained by the acquisition device is denoised by wavelet basis decomposition, specifically including: Wavelet decomposition was performed on the collected raw sound signal: In the formula, A k (n) represents the approximation coefficients of the k-th layer; h(m) represents the low-pass filter coefficients of the wavelet decomposition; n represents the sample index of the k-th layer; m represents the coefficient index; D j (n) represents the detail coefficients of the j-th layer; g(m) represents the high-pass filter coefficients of the wavelet decomposition; This represents the convolution operation; Soft thresholding denoising is applied to the detail coefficients of each layer, resulting in the following detail coefficients. The following conditions must be met: In the formula, D represents the detail coefficients after thresholding. j (n) represents the detail coefficients of the j-th layer; λ j The threshold value for the j-th layer is denoted by 'j'; sign(·) is the sign function. The processed coefficients are reconstructed using wavelet reconstruction, and the calculation formula is as follows: In the formula, x * (n) represents the denoised audio signal; A k (n) represents the approximation coefficients for the k-th layer; represents the detail coefficients of each layer after thresholding; j represents the layer number of the detail coefficients; k represents the layer number of the approximation coefficients. The video data is sequentially converted to grayscale and histogram equalized. The U-Net model is used to segment the region of interest, and the YOLOv5 algorithm is used to identify passenger flow and abnormal behavior.

5. The fault diagnosis method for terminal building pedestrian walkways according to claim 4, characterized in that, The data from the data collector is preprocessed, and the multimodal features of the data collector are integrated using a feature-level fusion algorithm. Temporal modeling is then performed based on these multimodal features. The process also includes: An attention mechanism is used to dynamically adjust the modal weights of the data collector. The fusion strategy for each modal weight is as follows: F=α·F vib +β·F tem +γ·F nio +δ·F vid ; α+β+γ+δ=1; In the formula, F is the fused multimodal feature vector; F vib F is the feature vector of the vibration signal data; tem F is the feature vector of the temperature data. noi F is the feature vector of the noisy data; vid Let be the feature vector of the video data; α and are the attention weights for the vibration mode; β is the attention weight for the temperature mode; γ and are the attention weights for the noise mode; and δ is the attention weight for the video mode. The fused multimodal feature vectors are input into a bidirectional LSTM network for time series modeling.

6. The fault diagnosis method for terminal building pedestrian walkways according to claim 5, characterized in that, Based on the aforementioned time-series modeling, the health index of the pedestrian walkway is obtained, and the fault type and severity level are evaluated, specifically including: The health index of the pedestrian walkway is output based on the CNN model: H=σ(W3·σ(W2·σ(W1·F+b1)+b2)+b3); In the formula, H is the health index; σ is the ReLU activation function; W1, W2, and W3 are the convolution weights; and b1, b2, and b3 are the bias terms. An improved SVM algorithm is used to classify the aforementioned fault types. For each health index of the test sample, a decision score for the fault category is calculated: In the formula, f j (H) is the decision function for fault category j; H is the health index feature vector of the input sample; α i For Lagrange multipliers; y i Let be the true class label corresponding to the i-th support vector; N is the total number of support vectors. b is the RBF kernel function; b is the bias term; j is the fault type; The fault types are divided into three categories: mechanical faults, electrical faults, and control faults. The severity levels are divided into three levels: minor, moderate, and severe.

7. The fault diagnosis method for terminal pedestrian walkways according to claim 6, characterized in that, The health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a pedestrian walkway model is constructed, specifically including: The fault alarms are responded to in a graded manner according to their severity level: Minor faults: recorded in the system log; Moderate fault: An alarm icon is displayed in the results presentation; Critical fault level: Triggers audible and visual alarms around the fault location and sends a push notification.

8. The fault diagnosis method for terminal pedestrian walkways according to claim 7, characterized in that, The pedestrian walkway model is optimized and upgraded across different scenarios using a federated learning network for collaborative optimization. Specifically, this includes: The local model parameters of the pedestrian walkway are updated as follows: In the formula, The local model parameters are updated for the k-th scene; θ k Here are the parameters of the local diagnostic model for the k-th scene; η is the learning rate; |D k | represents the number of local dataset samples for the k-th scene; The gradient of the loss function L with respect to the model parameters; f(x; θ) k ) represents the model's predicted value; y represents the true label; Federation server global model aggregation: In the formula, θ g For global model parameters of the federated server; The sample size weights for the k-th scene; These are the local model parameters updated for the k-th scene; Cross-scene model optimization iteration: Each scenario receives global model parameters and uses them as initial parameters for a new round of local training, achieving cross-scenario optimization through several iterations: In the formula, These are the global model parameters for the (t+1)th round; t represents the sample size weight for the k-th scene; t is the iteration round. Let be the global model parameters for round t; η be the learning rate; |D k | represents the number of local dataset samples for the k-th scene; Let L be the gradient of the loss function L with respect to the model parameters; y represents the model's predicted value; y represents the true label.

9. A fault diagnosis system for pedestrian walkways in an airport terminal, characterized in that, include: Data acquisition module: Collects data through vibration sensors, temperature sensors, microphone arrays and image acquisition devices to obtain real-time data on mechanical vibration, temperature changes, acoustic characteristics and video images; Preprocessing and fusion module: used to preprocess the acquired real-time data, realize multimodal feature weighted fusion through attention mechanism, and perform temporal modeling in combination with bidirectional LSTM algorithm; Fault diagnosis and evaluation module: used for data transmission and interaction, transmitting health indicators and fault evaluation results to the host computer system, and performing fault result visualization and graded alarm processing; Federated Learning Optimization Module: Combines federated learning networks for collaborative optimization, enabling cross-scenario upgrades to the diagnostic model.

10. An electronic device, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Data collectors are installed at key locations along the pedestrian walkway to acquire data from the data collectors. The data from the data collector is preprocessed, and the multimodal features of the data collector are integrated through a feature-level fusion algorithm. Temporal modeling is then performed based on the multimodal features. Based on the time-series modeling, the health index of the pedestrian walkway is obtained, and the fault type and severity level are evaluated. The health indicators and evaluation results are uploaded to the host computer system for result display and fault alarm, and a diagnostic model is constructed. A federated learning network is used for collaborative optimization to upgrade the diagnostic model across different scenarios.

Images