A method and system for testing a vehicle simulation failure of a fao system

Abstract

The application discloses a kind of FAO system vehicle simulation fault test method and system, it is related to rail transit technical field.The present application simulates fire through vehicle simulation model, fuses physical data, passenger behavior data and sound data, carries out effectiveness detection through gradient boosting tree, ensures that decision is based on credible data;YOLOv8 model is improved, the corresponding inference speed is improved, the demand of rapid trajectory extraction is met;Combined with Kalman filtering and cosine distance, the continuity of trajectory is improved under the scene of shielding and similar appearance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rail transit, in particular to a FAO system vehicle simulation fault test method and system. BACKGROUND

[0002] With the continuous development of urban rail transit technology, automation and intelligence have become the core direction of modern rail transit. In this context, the fully automated operation (FAO) system as a senior form of urban rail operation has gradually become a hot spot in the field of rail transit research and application. The FAO system detection laboratory is an important technical platform for ensuring the safe and efficient operation of the system. In the field of rail transit, the safety and reliability of the FAO system is crucial, and the vehicle fire emergency handling is one of the core test scenarios. In the prior art, the test of the FAO system vehicle fire related faults mainly depends on actual scene simulation or single data detection. For example, data processing mainly depends on a single sensor (such as a smoke sensor) or a single algorithm (such as a simple threshold judgment), which is prone to false positives due to sensor failure or environmental interference (such as dust causing smoke sensor false positives); or image recognition and multi-target tracking algorithms (such as traditional YOLO models) are insufficient in real-time and lightweight, and are difficult to adapt to the power limit of the vehicle-mounted edge device, resulting in delayed trajectory extraction and affecting the accuracy of the test.

[0003] The prior art relies on real experiments or simple simulations, which cannot flexibly simulate different fire intensities (such as smoldering and open fire) and fire scenes in different locations, and the cost of repeated testing is high, making it difficult to cover a variety of risk scenarios; in addition, a multi-dimensional fusion model of "physical data + behavior data (trajectory, aggregation) + sound data" has not been established, and only a single threshold (such as temperature > 80℃ to determine a fire) is used for judgment, ignoring the correlation between data, and the decision reliability is low. SUMMARY

[0004] The purpose of the present application is to provide a FAO system vehicle simulation fault test method and system to improve the above technical problems.

[0005] In order to achieve the above application purpose, the embodiments of the present application provide the following technical solutions:

[0006] A FAO system vehicle simulation fault test method and system, comprising:

[0007] Simulating a vehicle fire scene of the FAO system through a vehicle simulation model to obtain corresponding fire multi-source data;

[0008] Transmitting the fire multi-source data to a TIAS system and calling monitoring data of a CCTV system;

[0009] Separate monitoring data, obtain monitoring image and monitoring sound; based on monitoring image and monitoring sound, obtain simulation passenger motion trajectory graph, gathering degree and sound risk value;

[0010] Fire multi-source data is processed by using machine learning algorithm to obtain corresponding fire evaluation value;

[0011] Based on fire evaluation value, gathering degree and sound risk value, the fault detection of FAO system is carried out, and the fault detection result is obtained, the corresponding control instruction is determined and sent to FAS system;

[0012] The simulation passenger motion trajectory graph, the gathering degree and the sound risk value are obtained, comprising:

[0013] The monitoring data is separated by using audio and video separation tool or demultiplexing tool, and monitoring video and monitoring sound are extracted;

[0014] The monitoring image is processed by using image recognition algorithm and multi-target tracking algorithm to obtain passenger recognition result of different frames; based on passenger recognition result of different frames, combined with multi-target tracking algorithm, simulation passenger motion trajectory graph is drawn; the passenger training recognition result is the bounding box coordinates, confidence and appearance feature vector of each simulation passenger under different frames;

[0015] Based on simulation passenger motion trajectory graph and last frame monitoring image, gathering coefficient, average distance, average consistency and synchronization coefficient are calculated;

[0016] Based on gathering coefficient, average distance, average consistency and synchronization coefficient, gathering degree is calculated by weighted average;

[0017] The mel spectrum of monitoring sound is calculated; the mel spectrum is input into the sound risk assessment model to obtain the sound risk value.

[0018] Further, the motion trajectory graph of each simulation passenger is drawn, comprising:

[0019] The monitoring image is subjected to Gaussian filtering to eliminate common snowflakes in monitoring video and improve the effective identification of subsequent image recognition to obtain monitoring filtered image;

[0020] The monitoring filtered image is identified by using image recognition algorithm to detect the position of all passengers to obtain passenger recognition result of different frames;

[0021] The passenger recognition result of different frames is processed by using multi-target tracking algorithm to match the same simulation passenger under different frames to obtain position information of different simulation passengers under different frames; the position information comprises position and moving direction;

[0022] Based on the position information of different simulation passengers under different frames, simulation passenger motion trajectory graph is drawn.

[0023] Furthermore, the image recognition algorithm employs an improved YOLOv8 neural network model, including a Backbone network, a Neck network, and a Head network. The Backbone network comprises a first module, a second module, a third module, and an SPPF layer connected in series. The first to third modules each include a concatenated convolutional layer and a ShuffleNetV2 module. The Neck network comprises a fourth module, a fifth module, a sixth module, and a seventh module connected in series. The fourth and sixth modules each include an upsampling layer, a stitching layer, and a Ghost-C2f layer. The sixth and seventh modules each include a Ghost-C2f layer, a stitching layer, and a Ghost-CBS layer. Each Ghost-C2f layer comprises a Ghost-Conv layer, a segmentation layer, N Bottleneck modules, a stitching layer, and a Ghost-Conv layer connected in series. Each Bottleneck module comprises two Ghost-Conv layers connected in series. Each Ghost-CBS layer comprises a Ghost-Conv layer, a batch normalization layer, and a linear layer connected in series. The linear layer uses the Swish activation function.

[0024] Furthermore, the process of acquiring the location information of different simulated passengers in different frames:

[0025] Using the bounding box coordinates in consecutive frames, the moving speed of different simulated passengers is calculated; based on each moving speed, the bounding box coordinates in the current frame t are processed using the Kalman filter algorithm to obtain the predicted coordinates for the next frame t+1.

[0026] Calculate the cosine distance between any two distinct appearance feature vectors and use it as the appearance similarity.

[0027] Based on appearance similarity and the predicted coordinates of the next frame t+1, the Hungarian algorithm is used to match the simulated passengers in the current frame t and the next frame t+1 to obtain the tracking ID and location information of each simulated passenger.

[0028] Furthermore, the clustering coefficient The corresponding formula is:

[0029] ;

[0030] in, Indicates the first Line number The number of simulated passengers in the column, This represents the summation function. Indicates the number of grid cells;

[0031] Average Consistency The corresponding formula is:

[0032] ;

[0033] ;

[0034] wherein, denotes the vector norm, denotes the cosine similarity matrix between the th simulated passenger and the th simulated passenger, , denote the displacement vector of the th simulated passenger and the th simulated passenger, respectively, denotes the number of simulated passengers;

[0035] the synchronization coefficient The corresponding formula is:

[0036] ;

[0037] wherein, denotes the average speed, denotes the standard deviation.

[0038] Further, the sound risk value acquisition process is:

[0039] Performing short-time Fourier transform on the monitored sound to obtain the corresponding time-frequency graph;

[0040] Processing the time-frequency graph using a Mel filter bank to obtain an initial Mel spectrum; performing logarithmic calculation on the initial Mel spectrum to obtain a Mel spectrum;

[0041] Calculating the energy of the Mel spectrum as an energy feature, and calculating the zero-crossing rate of the Mel spectrum, and extracting the fundamental frequency feature of the Mel spectrum using an autocorrelation algorithm;

[0042] Fusing the Mel spectrum, the energy feature, the zero-crossing rate, and the fundamental frequency feature to obtain global sound information features and inputting the global sound information features into a sound risk assessment model for risk assessment to obtain a sound risk value; the sound risk assessment model adopts a CNN model.

[0043] A vehicle simulated fault test system of an FAO system, comprising:

[0044] A fire multi-source data acquisition module for simulating a vehicle fire scenario of the FAO system through a vehicle simulation model and acquiring corresponding fire multi-source data;

[0045] A monitoring calling module for transmitting the fire multi-source data to a TIAS system and calling monitoring data of a CCTV system;

[0046] A monitoring data separation module is configured to separate monitoring data, acquire monitoring images and monitoring sounds, and filter the monitoring images to obtain monitoring filtered images.

[0047] A passenger simulation module is configured to identify the monitoring filtered images by using an image recognition algorithm to obtain passenger identification results of different frames.

[0048] A passenger motion trajectory acquisition module is configured to process the passenger identification results of different frames by using a multi-target tracking algorithm to obtain position information of different simulated passengers in different frames, and draw a simulated passenger motion trajectory map based on the position information of the different simulated passengers in different frames.

[0049] An aggregation degree calculation module is configured to calculate an aggregation coefficient, an average distance, an average consistency and a synchronization coefficient based on the simulated passenger motion trajectory map and the monitoring image of the last frame, and calculate the aggregation degree by weighted average based on the aggregation coefficient, the average distance, the average consistency and the synchronization coefficient.

[0050] A sound risk value calculation module is configured to calculate a mel spectrum of the monitoring sound, input the mel spectrum into a sound risk assessment model, and obtain a sound risk value.

[0051] A fire assessment value calculation module is configured to process fire multi-source data by using a machine learning algorithm to obtain a corresponding fire assessment value.

[0052] A fault detection result determination module is configured to perform fault detection on the FAO system based on the fire assessment value, the aggregation degree and the sound risk value, obtain a fault detection result, and send an alarm instruction.

[0053] A fire alarm and train operation module is configured to receive the alarm instruction of the fault detection result determination module and perform alarm, control the train vehicle to stop at the next platform and keep the train door open.

[0054] The present application has the following advantages:

[0055] The present application simulates a fire by using a vehicle simulation model, fuses physical data, passenger behavior data and sound data, determines the correlation between multiple data, performs effectiveness detection by using a gradient boosting tree, ensures that the decision is based on reliable data, improves the YOLOv8 model, improves the corresponding inference speed, meets the demand of quickly extracting a trajectory, and improves the trajectory continuity in the scene of occlusion and similar appearance by combining Kalman filtering and cosine distance. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0057] Figure 1 The method flowchart in the embodiments of the present application;

[0058] Figure 2 The improved YOLOv8 neural network model structure diagram in the embodiments of the present application;

[0059] Figure 3 The structure diagram of Ghost-C2f layer, Bottleneck module and Ghost-CBS layer in the embodiments of the present application;

[0060] Figure 4 The system structure diagram in the embodiments of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0062] Please refer to Figure 1 The present embodiment provides a FAO system vehicle simulation fault test method. Figure 1 The execution subject of the method shown can be a software and / or hardware device. The execution subject of the present application can include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, the user equipment can include but is not limited to computers, smart phones, personal digital assistants (Personal Digital Assistant, PDA) and the above-mentioned electronic devices, etc. The network equipment can include but is not limited to a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Cloud computing is a kind of distributed computing, which is a super virtual computer composed of a loose coupled computer group. The present embodiment does not limit this.

[0063] A vehicle simulation fault test method of an FAO system, comprising:

[0064] S1, simulating a vehicle fire scenario of the FAO system through a vehicle simulation model, and obtaining corresponding fire multi-source data; the fire multi-source data comprises temperature, concentration, concentration, smoke concentration, and fire location; the FAO system comprises a TIAS system (a driving integrated automation system), a CCTV system (a closed-circuit television monitoring system), and a FAS system (a fire alarm system).

[0065] S2, transmitting the fire multi-source data to the TIAS system and calling monitoring data of the CCTV system;

[0066] The fire multi-source data is transmitted to the TIAS system, the TIAS system sends a calling instruction to the CCTV system, and the CCTV system returns corresponding monitoring video (monitoring data) to the TIAS system after receiving the calling instruction.

[0067] S3, separating the monitoring data to obtain monitoring images and monitoring sounds, and obtaining simulated passenger motion trajectories, aggregation degrees, and sound risk values based on the monitoring images and the monitoring sounds;

[0068] The S3 comprises:

[0069] S3-1, separating the monitoring data by using an audio-video separation tool or a demultiplexing tool to extract monitoring video and monitoring sounds, setting a time interval point, extracting the monitoring video based on the time interval point to obtain monitoring images; in this embodiment, the time interval point is 1S, and one frame of the monitoring video is extracted as a monitoring image every 1S;

[0070] S3-2, processing the monitoring images by using an image recognition algorithm and a multi-target tracking algorithm to obtain passenger recognition results of different frames; based on the passenger recognition results of different frames, an simulated passenger motion trajectory diagram is drawn by combining the multi-target tracking algorithm; the passenger training recognition result is a bounding box coordinate, a confidence degree, and an appearance feature vector of each simulated passenger under different frames;

[0071] The S3-2 comprises:

[0072] S3-2-1, performing Gaussian filtering on the monitoring images to eliminate common snowflakes in the monitoring video, improve the effective recognition of subsequent image recognition, and obtain a monitoring filtered image;

[0073] S3-2-2, recognizing the monitoring filtered image by using an image recognition algorithm to detect the positions of all passengers, and obtaining passenger recognition results of different frames; the image recognition algorithm adopts a neural network model of a YOLO series.

[0074] In the embodiment, the improved YOLOv8 neural network model is used to identify the monitoring filtered image. When identifying the image, in order to quickly identify the passenger, the YOLOv8 neural network model is simplified, only the accurate identification of the passenger is required, and the demand for quickly positioning the passenger is still met, so that the lightweight ShuffleNetV2 module (Shuffle Network Version 2, Shuffle Network V2 edition) is used to replace the cross-stage part connection darknet (CSPDarknet) module of the Backbone network, the channel shuffle (ChannelShuffle) and the depth separable convolution are used, the parameter amount is significantly reduced while the feature expression ability is maintained; the GhostPAN module (Ghost Path Aggregation Network module) is used to replace the PANet (Path Aggregation Network) module of the Neck network, the Ghost module (a “ghost” feature is generated through a cheap operation) is used to replace the standard convolution, the number of feature pyramid layers is reduced to 2 layers (the main scale is retained), the feature fusion efficiency is improved; the share head (Share-head) is used to replace the decoupled head, the calculation speed is improved and the redundant calculation is reduced. Therefore, as shown in Figure 2 、 Figure 3 the improved YOLOv8 neural network model includes a Backbone network, a Neck network and a Head network; the Backbone network includes a first module, a second module, a third module and an SPPF (Fast Spatial Pyramid Pooling) layer connected in series; the first to third modules each include a convolution layer (Conv), a ShuffleNetV2 module connected in series; the Neck network includes a fourth module, a fifth module, a sixth module and a seventh module connected in series; the fourth module and the sixth module each include an up-sampling layer (Upsample), a concatenation layer (Concat) and a Ghost-C2f layer; the sixth module and the seventh module each include a Ghost-C2f layer, a concatenation layer and a Ghost-CBS layer; each Ghost-C2f layer includes a Ghost-Conv layer (Ghost Convolution layer), a split layer (Split), N Bottleneck modules, a concatenation layer and a Ghost-Conv layer connected in series; each Bottleneck module includes two Ghost-Conv layers connected in series; each Ghost-CBS layer includes a Ghost-Conv layer, a batch normalization (Batch Normalization) layer and a linear layer (SiLu) connected in series; the linear layer adopts a Swish activation function.

[0075] Therefore, the training process of the improved YOLOv8 neural network model includes:

[0076] T1, image pre-processing (size scaling and normalization) is performed on the monitoring filtered image to obtain a monitoring training image; wherein the monitoring training image has a channel number of 3 and a dimension of ;

[0077] T2, the monitoring training image is input into a Backbone network to respectively output a first scale contour feature, a second scale contour feature and a third scale contour feature;

[0078] The monitoring training image is input into a first module to extract a first convolutional feature of the monitoring training image through a convolutional layer; a depth separable convolution and a channel shuffle in a ShuffleNetV2 module are used to output the first scale contour feature; since the ShuffleNetV2 module is a publicly disclosed technology, the corresponding process is not described in detail.

[0079] The first scale contour feature is input into a second module to obtain a second scale contour feature; the second scale contour feature is input into a third module to obtain an initial third scale contour feature; the processing processes of the second module and the third module are the same as that of the first module.

[0080] The initial third scale contour feature is input into an SPPF layer to obtain a third scale contour feature.

[0081] T3, the first scale contour feature to the third scale contour feature is input into a Neck network to respectively output a first scale contour enhanced feature, a second scale contour enhanced feature;

[0082] The T3 includes:

[0083] T3-1, the second scale contour feature and the third scale contour feature are input into a fourth module to obtain a first scale contour fusion feature;

[0084] T3-2, the first scale contour feature and the first scale contour fusion feature are input into a fifth module to obtain a second scale contour fusion feature;

[0085] T3-3, the first scale contour fusion feature and the second scale contour fusion feature are input into a sixth module to obtain a first scale contour enhanced feature;

[0086] T3-4, the first scale contour enhanced feature and the third scale contour feature are input into a seventh module to obtain a second scale contour enhanced feature;

[0087] T4, input the first scale contour enhancement feature, the second scale contour enhancement feature and the second scale contour fusion feature into a shared detection head to obtain a passenger training recognition result; the passenger training recognition result is a boundary box coordinate, a confidence and an appearance feature vector of each simulated passenger under different frames, the boundary box coordinate is used for subsequent motion trajectory drawing, the confidence is used for judging whether it is a simulated passenger, and the appearance feature vector is used for matching the same simulated passenger in a multi-target tracking algorithm. In the embodiment, the boundary box coordinate is a center point coordinate of the boundary box.

[0088] T5, calculate a detection loss function based on the passenger training recognition result; and adjust parameters of the improved YOLOv8 neural network model based on the detection loss function.

[0089] The detection loss function includes a boundary box loss function , a classification loss function and a distillation loss function , and the corresponding formulas are:

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] wherein, , , respectively represent a boundary box weight, a classification weight and a distillation weight, , represent a passenger training detection box and a passenger training actual detection box, represent a complete intersection union ratio, represent a logarithmic function, , respectively represent a real classification label and a classification result (whether it is a passenger), represent a KL divergence, , respectively represent an output of a student model and an output of a teacher model. In the embodiment, , , respectively take values of 0.5, 0.3 and 0.2.

[0095] Specifically, the distillation loss is based on the concept of "knowledge distillation", which transfers the "knowledge" learned by the complex teacher model, such as fine discrimination of passenger features, classification tendency in different scenarios, etc., to the lightweight student model. The teacher model can capture more details, such as subtle passenger poses and distinguishing features of easily confused targets. The student model learns these "knowledge" as much as possible by imitating the output distribution of the teacher model, while keeping its parameter amount small and structure simple, thereby improving performance without significantly increasing computational cost. In this embodiment, the teacher model is a traditional YOLOv8 neural network model, and the student model is an improved YOLOv8 neural network model.

[0096] As shown in Table 1, the improved YOLOv8 neural network model (YOLOv8-tiny) has a speed improvement of 26% and a parameter reduction of 75% in the passenger detection task. Although the accuracy decreases by 15%, it still meets the demand of "quickly positioning passengers" (the accuracy loss can be compensated by a tracking algorithm later).

[0097] Table 1

[0098]

[0099] In addition, compared with the traditional YOLOv8 neural network model, the improved YOLOv8 neural network model is more lightweight and has simpler branches in its network structure to achieve fast passenger recognition and facilitate subsequent response. On the premise of sacrificing a small amount of accuracy, the inference speed is improved to 120 FPS, meeting the real-time demand of "extracting passenger positions per second", and providing continuous and stable input (position coordinates) for subsequent tracking algorithms.

[0100] S3-2-3, using a multi-target tracking algorithm (MOT algorithm) to process the passenger recognition results of different frames, matching the same simulated passenger in different frames to obtain the position information of different simulated passengers in different frames; the position information includes position and moving direction;

[0101] The S3-2-3 includes:

[0102] Using the bounding box coordinates in consecutive frames, the moving speed of different simulated passengers is calculated; based on each moving speed, the bounding box coordinates in the current frame t are processed using a Kalman filter algorithm to obtain the predicted coordinates of the next frame t+1 The corresponding formula is:

[0103] ;

[0104] ;

[0105] wherein,​ represents the displacement state matrix of the i-th simulated passenger, represents the horizontal and vertical coordinates of the bounding box coordinates of the i-th simulated passenger, represents the horizontal and vertical velocities of the bounding box coordinates of the i-th simulated passenger.

[0106] Calculate the cosine distance between each pair of different appearance feature vectors as the appearance similarity; the closer the cosine distance is to 1, the more likely the two appearance feature vectors belong to the same simulated passenger;

[0107] Based on the appearance similarity, the predicted coordinates of the next frame t+1 , use the Hungarian algorithm to match the simulated passengers of the current frame t and the next frame t+1, determine that “the simulated passenger A of the current frame t = the simulated passenger A of the next frame t+1”, and obtain the tracking ID and position information of each simulated passenger. The tracking ID is used to determine the identity of the same simulated passenger.

[0108] S3-2-4, based on the position information of different simulated passengers under different frames, draw the simulated passenger motion trajectory diagram.

[0109] For each simulated passenger, convert the position in the corresponding position information to actual coordinates; draw different motion trajectories of the actual coordinates and moving directions of the simulated passenger under different consecutive frames on the vehicle plan according to the order of the frames; the motion trajectories of different simulated passengers are distinguished by different colors;

[0110] Based on the motion trajectories of different simulated passengers, draw the motion trajectory diagram in the corresponding carriage of the fire site in a three-dimensional space composed of spatial horizontal coordinates, spatial vertical coordinates and time axis.

[0111] Use Kalman filtering and cosine distance to predict the position of the next frame and match the appearance features, which improves the anti-interference ability and can adapt to the complex environment in the vehicle. It specifically solves the problems of “dynamicity (moving, changing direction), complexity (occlusion, similar appearance), and practicality (physical meaning, real-time)” in passenger tracking in rail transit. The final output of “continuous physical trajectory + direction feature” can directly serve the fire judgment (such as combining sound, monitoring abnormal data, etc.), and provide reliable basis for emergency decision-making.

[0112] S3-3, based on the simulated passenger motion trajectory diagram and the last frame of the monitoring image, calculate the clustering coefficient, average distance, average consistency and synchronization coefficient;

[0113] ​​​​​​​Divide the last frame of the surveillance image into For each grid cell, the number of simulated passengers within that grid cell is counted, and the corresponding clustering coefficient is calculated. The corresponding formula is:

[0114] ;

[0115] in, Indicates the first Line number The number of simulated passengers in the column, This represents the summation function.

[0116] Based on the motion trajectory map, calculate the Euclidean distance between each pair of simulated passengers and obtain the corresponding average distance; the smaller the average distance, the closer the simulated passengers are to each other and the higher the degree of clustering.

[0117] Based on the motion trajectory, calculate the displacement vector of each simulated passenger in the last frame;

[0118] Calculate the cosine similarity matrix between each pair of simulated passengers; based on the cosine similarity matrix, calculate the average consistency. The closer the average consistency is to 1, the more likely the simulated passengers are moving in the same direction; the closer it is to 0, the more likely the simulated passengers are moving randomly. Therefore, the corresponding formula is:

[0119] ;

[0120] ;

[0121] in, Represents the magnitude of the vector. Indicates the first The simulated passenger and the first Cosine similarity matrix between simulated passengers , They represent the first The simulated passenger and the first The displacement vector of a simulated passenger. This indicates the number of simulated passengers.

[0122] Calculate the corresponding average velocity based on each displacement vector. and standard deviation ; Calculate the synchronization coefficient based on the standard deviation and average speed. The corresponding formula is:

[0123] .

[0124] Synchronization coefficient The greater, the closer the passenger speed is (the collective action feature is obvious).

[0125] S3-4, based on the aggregation coefficient, the average distance , the average consistency and the synchronization coefficient, the aggregation degree is calculated by weighted average , the corresponding formula is:

[0126] ;

[0127] The aggregation degree The value range of the aggregation degree is [0, 1], and the higher the value is, the more the simulated passengers are aggregated, which proves that the fire occurs. 、 、 、 Indicates the weight.

[0128] S3-5, calculate the mel spectrum of the monitored sound; input the mel spectrum into the sound risk assessment model to obtain the sound risk value; the sound risk assessment model adopts a CNN model;

[0129] The S3-5:

[0130] S3-5-1, perform short-time Fourier transform on the monitored sound to obtain the corresponding time-frequency graph; frame the monitored sound, add a Hanning window to each frame of sound and perform Fourier transform to obtain the time-frequency graph;

[0131] S3-5-2, process the time-frequency graph by using a mel filter bank to obtain an initial mel spectrum; perform logarithmic calculation on the initial mel spectrum to obtain the mel spectrum; the mel filter bank includes 40 mel filters with a frequency range of [20Hz, 20kHz];

[0132] S3-5-3, calculate the energy of the mel spectrum as an energy feature, count the zero-crossing rate of the mel spectrum, and extract the fundamental frequency feature of the mel spectrum by using an autocorrelation algorithm; wherein the energy feature is used to detect sound bursts, such as explosion sound; the zero-crossing rate is the number of zero-crossing points of each frame of signal, which is used to distinguish between speech (high zero-crossing rate) and mechanical noise (low zero-crossing rate); the fundamental frequency feature is used to identify a distress call (the fundamental frequency is usually >200Hz and fluctuates greatly).

[0133] S3-5-4, fuse the mel spectrum, the energy feature, the zero-crossing rate and the fundamental frequency feature to obtain global sound information features and input them into the CNN model for risk assessment to obtain the sound risk value. Since the CNN model is an existing publicly disclosed network model, the process of the CNN model processing the global sound information features will not be described.

[0134] S4, process the fire multi-source data by using a machine learning algorithm to obtain a corresponding fire evaluation value;

[0135] In the present embodiment, the machine learning algorithm employs LightGBM. The training process of LightGBM includes:

[0136] Obtain fire multi-source training data and its labels and pre-process to obtain fire multi-source training processed data; the pre-processing includes missing value processing, outlier processing and standardization processing; for example, if a certain sensor data (temperature, concentration, concentration, smoke concentration) is missing, fill it with the "mean of the previous 3 seconds at the same position" (short-term missing); if the continuous missing is > 5 seconds, mark it as "invalid data" and replace it with other sensor data in the same car compartment (to ensure data integrity); remove obvious error values, such as temperature > 1000℃, which exceeds the sensor range.

[0137] Input the fire multi-source training processed data and labels into LightGBM and adjust the network parameters of LightGBM using grid search algorithm.

[0138] Fire assessment value The corresponding formula is:

[0139] ;

[0140] ;

[0141] wherein, denotes the learning rate, denotes the total number of decision trees, denotes the output of the th decision tree, denotes the splitting function of the decision tree (mapping the input data to a certain leaf node), denotes the weight value of the leaf node.

[0142] S5, based on the fire assessment value, the aggregation degree and the sound risk value, perform fault detection on the FAO system to obtain a fault detection result, determine a corresponding control instruction and send it to the FAS system. The control instruction includes an alarm instruction and a maintenance instruction.

[0143] The S5 includes:

[0144] S5-1, based on the formula:

[0145] ;

[0146] obtain a fault detection result ; wherein, denotes a sigmoid function, which compresses the result to between 0 and 1, , respectively represent the fire assessment value and the sound risk value, represents a constant.

[0147] S5-2, judging the fault detection result whether it exceeds the fault detection threshold; if yes, it is determined that the vehicle has a fire and proceeds to S5-3; otherwise, it is determined that the vehicle has no fire and proceeds to S5-4;

[0148] S5-3, sending an alarm instruction through the TIAS system and sending it to the FAS system for fire alarm; in addition, the TIAS system sends an emergency instruction to the ATP system to control the operation of the train vehicle through the ATP system, stops the train vehicle at the next platform and keeps the train door open, facilitating passenger evacuation and subsequent staff fire extinguishing.

[0149] S5-4, sending a maintenance instruction to the cloud through the TIAS system to remind the staff to check.

[0150] As shown in Figure 4 , a vehicle simulation fault test system of a FAO system comprises:

[0151] a fire multi-source data acquisition module for simulating a vehicle fire scenario of the FAO system through a vehicle simulation model and acquiring corresponding fire multi-source data;

[0152] a monitoring calling module for transmitting the fire multi-source data to the TIAS system and calling monitoring data of the CCTV system;

[0153] a monitoring data separation module for separating the monitoring data, acquiring monitoring images and monitoring sounds, and filtering the monitoring images to obtain monitoring filtered images;

[0154] a simulated passenger identification module for identifying the monitoring filtered images using an image recognition algorithm to obtain passenger identification results of different frames;

[0155] a simulated passenger motion trajectory acquisition module for processing the passenger identification results of different frames using a multi-target tracking algorithm (MOT algorithm) to obtain position information of different simulated passengers in different frames; and drawing a motion trajectory graph based on the position information of different simulated passengers in different frames;

[0156] an aggregation degree calculation module for calculating an aggregation coefficient, an average distance, an average consistency, and a synchronization coefficient based on the simulated passenger motion trajectory graph and the monitoring image of the last frame; and calculating the aggregation degree by weighted average based on the aggregation coefficient, the average distance, the average consistency, and the synchronization coefficient;

[0157] a sound risk value calculation module for calculating a mel spectrum of the monitoring sound; inputting the mel spectrum into a sound risk assessment model to obtain a sound risk value;

[0158] a fire assessment value calculation module configured to process the fire multi-source data by using a machine learning algorithm to obtain corresponding fire assessment values;

[0159] a fault detection result determination module configured to perform fault detection on the FAO system based on the fire assessment values, the aggregation degree and the sound risk value, obtain a fault detection result and send an alarm instruction;

[0160] a fire alarm and train operation module configured to receive the alarm instruction of the fault detection result determination module and perform alarm; control the train vehicle to stop at the next platform and keep the train door open.

[0161] In summary, the present application

[0162] It should be noted that, as for the system in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0163] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0164] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of testing a FAO system vehicle simulation failure, characterized by, The application relates to a fire accident detection and control system (FAO system) for a vehicle, and a vehicle fire accident detection and control method. The vehicle fire accident scenario of the FAO system is simulated through a vehicle simulation model, and corresponding fire multi-source data is obtained; The fire multi-source data is transmitted to a TIAS system and monitoring data of a CCTV system is called; The monitoring data is separated to obtain monitoring images and monitoring sounds; based on the monitoring images and the monitoring sounds, a simulated passenger motion trajectory graph, an aggregation degree and a sound risk value are obtained; The fire multi-source data is processed by using a machine learning algorithm to obtain a corresponding fire evaluation value; Based on the fire evaluation value, the aggregation degree and the sound risk value, fault detection is performed on the FAO system to obtain a fault detection result, corresponding control instructions are determined and sent to an FAS system; The simulated passenger motion trajectory graph, the aggregation degree and the sound risk value are obtained by: using an audio and video separation tool or a demultiplexing tool to separate the monitoring data and extract monitoring videos and monitoring sounds; processing the monitoring images by using an image recognition algorithm and a multi-target tracking algorithm to obtain passenger recognition results of different frames; based on the passenger recognition results of different frames, a simulated passenger motion trajectory graph is drawn by combining the multi-target tracking algorithm; the passenger training recognition result is the boundary box coordinates, the confidence and the appearance feature vector of each simulated passenger under different frames; based on the simulated passenger motion trajectory graph and the monitoring image of the last frame, an aggregation coefficient, an average distance, an average consistency and a synchronization coefficient are calculated; based on the aggregation coefficient, the average distance, the average consistency and the synchronization coefficient, the aggregation degree is calculated by weighted average; the mel spectrum of the monitoring sound is calculated; the mel spectrum is input into a sound risk evaluation model to obtain a sound risk value.

2. The method of claim 1, wherein the FAO system vehicle simulation fault test method is characterized by, The simulated passenger motion trajectory graph is drawn by: performing Gaussian filtering on the monitoring images to eliminate snowflakes and obtain monitoring filtered images; recognizing the monitoring filtered images by using an image recognition algorithm to detect the positions of all passengers and obtain passenger recognition results of different frames; processing the passenger recognition results of different frames by using a multi-target tracking algorithm to match the same simulated passenger under different frames and obtain position information of different simulated passengers under different frames; the position information includes positions and moving directions; based on the position information of different simulated passengers under different frames, a simulated passenger motion trajectory graph is drawn.

3. The method of claim 1, wherein the FAO system vehicle simulation fault test method is characterized by, The image recognition algorithm adopts an improved YOLOv8 neural network model, including a Backbone network, a Neck network and a Head network; the Backbone network includes a first module, a second module, a third module and an SPPF layer connected in series; the first to third modules each include a convolutional layer, a ShuffleNetV2 module connected in series; the Neck network includes a fourth module, a fifth module, a sixth module and a seventh module connected in series; the fourth module and the sixth module each include an up-sampling layer, a splicing layer and a Ghost-C2f layer; the sixth module and the seventh module each include a Ghost-C2f layer, a splicing layer and a Ghost-CBS layer; each Ghost-C2f layer includes a Ghost-Conv layer, a division layer, N Bottleneck modules, a splicing layer and a Ghost-Conv layer connected in series; each Bottleneck module includes two Ghost-Conv layers connected in series; each Ghost-CBS layer includes a Ghost-Conv layer, a batch normalization layer and a linear layer connected in series; the linear layer adopts a Swish activation function.

4. The method of claim 2, wherein the FAO system vehicle simulation fault test method is characterized by, The position information of the different simulated passengers in different frames is obtained by: Using the bounding box coordinates in the continuous frames, the moving speeds of the different simulated passengers are calculated; based on the moving speeds, the Kalman filtering algorithm is used to process the bounding box coordinates in the current frame t to obtain the predicted coordinates of the next frame t+1; The cosine distances between the appearance feature vectors of any two are calculated and used as appearance similarities; Based on the appearance similarities and the predicted coordinates of the next frame t+1, the Hungarian algorithm is used to match the simulated passengers in the current frame t and the next frame t+1, to obtain the tracking ID and position information of each simulated passenger.

5. The method of claim 1, wherein the FAO system vehicle simulation fault test method is characterized by, The aggregation coefficient The corresponding formula is: ； wherein, represents the number of analog passengers in the i-th row and j-th column, represents the number of analog passengers in the i-th row and j-th column, represents the number of analog passengers in the i-th row and j-th column, represents a summation function, represents the number of grids; Average Consistency The corresponding formula is: ； ； wherein, denotes the vector norm, denotes the cosine similarity matrix between the th simulated passenger and the th simulated passenger, , denote the displacement vector of the th simulated passenger and the th simulated passenger, respectively, denotes the number of simulated passengers; Synchronization coefficient The corresponding formula is: ； wherein, represents the average speed, represents the standard deviation.

6. The method of claim 1, wherein the FAO system vehicle simulation fault test method further comprises: The sound risk value is obtained by: Performing short-time Fourier transform on the monitored sound to obtain a corresponding time-frequency graph; Processing the time-frequency graph using a Mel filter bank to obtain an initial Mel spectrum; performing logarithmic calculation on the initial Mel spectrum to obtain a Mel spectrum; Calculating the energy of the Mel spectrum as an energy feature, counting the zero-crossing rate of the Mel spectrum, and extracting the fundamental frequency feature of the Mel spectrum using an autocorrelation algorithm; Fusing the Mel spectrum, the energy feature, the zero-crossing rate and the fundamental frequency feature to obtain global sound information features and inputting the global sound information features into a sound risk assessment model for risk assessment to obtain a sound risk value; The sound risk assessment model adopts a CNN model.

7. A FAO system vehicle simulation fault test system for implementing the FAO system vehicle simulation fault test method according to any one of claims 1 to 6, characterized in that, It comprises: a fire multi-source data acquisition module configured to simulate a vehicle fire scenario of a FAO system by using a vehicle simulation model and acquire corresponding fire multi-source data; a monitoring calling module configured to transmit the fire multi-source data to a TIAS system and call monitoring data of a CCTV system; a monitoring data separation module configured to separate the monitoring data, acquire monitoring images and monitoring sound, and filter the monitoring images to obtain monitoring filtered images; a simulated passenger identification module configured to identify the monitoring filtered images by using an image recognition algorithm to obtain passenger identification results of different frames; and a sound risk assessment module configured to acquire sound information features of the monitoring sound, input the sound information features into a sound risk assessment model, and output a sound risk value. The simulation passenger motion trajectory acquisition module is configured to process passenger identification results of different frames by using a multi-target tracking algorithm to obtain position information of different simulation passengers in different frames. Based on the position information of the different simulation passengers in different frames, a simulation passenger motion trajectory diagram is drawn. The aggregation degree calculation module is configured to calculate an aggregation coefficient, an average distance, an average consistency, and a synchronization coefficient based on the simulation passenger motion trajectory diagram and the monitoring image of the last frame. Based on the aggregation coefficient, the average distance, the average consistency, and the synchronization coefficient, the aggregation degree is calculated by weighted average. The sound risk value calculation module is configured to calculate a mel spectrum of the monitoring sound; input the mel spectrum into a sound risk assessment model to obtain a sound risk value. The fire assessment value calculation module is configured to process fire multi-source data by using a machine learning algorithm to obtain a corresponding fire assessment value. The fault detection result determination module is configured to perform fault detection on the FAO system based on the fire assessment value, the aggregation degree, and the sound risk value, obtain a fault detection result, and send an alarm instruction. The fire alarm and train operation and maintenance module is configured to receive the alarm instruction of the fault detection result determination module and perform alarm; control the train vehicle to stop at the next station and keep the train door open.

Images