A method, device, equipment and medium for detecting abnormal events in an audio-video track of a rail transit carriage

Abstract

The application discloses a method, device, equipment and medium for detecting abnormal events in an audio-video track transportation carriage, comprising: collecting monitoring video in the track carriage by using a camera, extracting related features of fighting and quarreling through an image posture detection branch to obtain image branch confidence of fighting and quarreling behavior; collecting original audio in the carriage through a microphone, extracting related features of fighting and quarreling through an audio detection analysis branch to obtain audio branch confidence of fighting and quarreling; extracting personnel density grading results in the track carriage; performing parallel processing on the image branch confidence and the audio branch confidence, and dynamically distributing branch weights of the image branch confidence and the audio branch confidence based on the personnel density grading results in the track carriage to obtain fusion confidence, thereby obtaining a carriage abnormal event determination result. Therefore, the application can accurately detect fighting and quarreling behavior in different personnel dense scenes in the track carriage.

Description

Technical Field

[0001] This invention relates to the field of rail transit safety monitoring technology, specifically to a method, apparatus, equipment, and medium for detecting abnormal events in rail transit carriages using audio and video based on density grading. Background Technology

[0002] With the rapid development of rail transit (subways, intercity railways, etc.), passenger flow within carriages is frequent, and the density of people varies significantly during peak, off-peak, and low-peak hours. Abnormal incidents such as fights and arguments occur frequently, disrupting order and directly threatening passenger safety. To ensure the safe operation of rail transit and protect passenger rights, abnormal event detection technology has been upgraded from traditional manual inspection to "intelligent audio-visual fusion detection." Existing abnormal event detection solutions for public scenarios are mainly divided into two categories: single-modal detection and multi-modal fusion detection.

[0003] The existing single-modal detection technologies mainly refer to visual detection technologies based on human posture estimation. These technologies mainly capture limb movements by extracting key human body information, and then identify abnormal behaviors such as fighting. The typical implementation scheme is as follows: (1) Using 17 or more human body key points (such as head, torso, limb joints, etc.) defined by the COCO standard, the human posture in the monitoring video stream is estimated in real time by computer vision algorithms (such as convolutional neural networks). By analyzing the positional relationship, motion trajectory and movement amplitude between key points, it is determined whether there are violent limb movements such as fighting. For example, the existing technology discloses a fighting behavior recognition method and device based on video stream data. This technology uses a video analysis network model to detect whether there is limb intersection in the video stream. If there is limb intersection, the limb intersection personnel are labeled. According to the limb intersection personnel labels and related information, the fighting alarm information is output, and the alarm information is encapsulated and uploaded to the alarm database. This technology only relies on the limb intersection features in the video stream to recognize fighting behavior, which is essentially a visual single-modal detection scheme based on human posture association features. The shortcomings of this type of technology are: poor occlusion adaptability: in the highly dense scene of rail transit carriages, the limbs of people are seriously occluded with each other, and the detection of redundant key points is prone to increase the misjudgment rate due to the occlusion of key action features, and it is impossible to accurately capture the core limb movements of fighting behavior; weak scene adaptability: the generalized detection logic cannot cope with the dynamic changes of the environment in densely populated scenes. (2) Identify abnormal events such as quarrels by analyzing the feature parameters of environmental sounds. The typical implementation scheme is: to obtain environmental sound signals through audio acquisition equipment, extract the decibel value, speech spectrum, speech rate and other feature parameters of the sound, and compare them with the preset abnormal sound threshold or feature template to determine whether there are abnormal sounds such as quarrels. For example, existing dispute recognition and processing methods and devices are designed for complex sound environments with multiple rooms in a house. They can identify dispute sounds within the rooms. They first acquire sound signals from each room and identify the dispute sounds. To address interference such as ambient noise in the house, they use specific audio recognition logic to lock down the characteristics of the dispute sounds. They can also process the dispute sounds by changing their voices and then broadcast them through a central control device. This technology is highly effective in identifying dispute sounds in noisy home environments. However, this type of technology has the following shortcomings: it has a single information dimension: relying solely on audio features cannot fully reflect the whole picture of the event. It cannot effectively identify physical conflicts without obvious sound output or abnormal events where the sound is severely blocked.

[0004] Existing multimodal fusion detection technologies mainly refer to audio-video fusion detection technologies. These technologies overcome the limitations of single-modal approaches in multimodal anomaly detection by leveraging the complementary advantages of both audio signals and video visual features. This improves the accuracy and anti-interference capabilities of identifying anomalies such as fights and arguments. A typical implementation involves: acquiring ambient sound signals in real-time using audio acquisition devices (such as microphones) to extract key audio information such as decibel levels, spectral characteristics, and abnormal noises; acquiring video stream data via cameras and extracting key video features such as human body detection, pose recognition, or 3D reconstruction techniques; and constructing a joint audio-video detection model using strategies such as time alignment, feature fusion, and adaptive weight adjustment. This model deeply integrates and analyzes the abnormal features of the audio with the visual motion features of the video to comprehensively determine whether anomalies such as fights or arguments exist. Some solutions also optimize for different hardware and scenario requirements through edge computing and model lightweighting. For example, existing technologies for detecting violent events based on 3D reconstruction of audio and video using multiple cameras involve acquiring video data from multiple cameras and aligning it with audio data over time; reconstructing 3D human key points from the video data to generate 3D coordinate data; extracting features from the 3D coordinate data and audio data respectively to construct a detection model and output the probability of violent behavior; setting initial weights based on scene characteristics and adaptively adjusting them to obtain the fused violent probability; and determining violent behavior based on a preset threshold. However, this type of technology has shortcomings: while it improves recognition accuracy compared to a single modality, the weight fusion strategy used is mostly based on adaptive adjustment of initial weights according to scene characteristics, which cannot adjust the proportion of sound branches in the decision-making process in real time and does not fully consider the unique environment of subway cars, such as confined space, frequent passenger flow, dynamic density changes, and complex background noise (e.g., train noise, crowd noise). Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. Therefore, in response to the above-mentioned problem, the purpose of this invention is to provide a method, apparatus, device, and medium for detecting abnormal events in rail transit carriages based on density-level audio and video, capable of accurately detecting fighting and arguing behavior in different densely populated scenarios within rail carriages.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: In a first aspect, the present invention provides a method for detecting abnormal events in rail transit carriages using audio and video, comprising: The system uses cameras to collect surveillance videos inside the railcar, extracts fighting and arguing-related features through image pose detection branches, and obtains the image branch confidence of fighting and arguing behavior. The original audio inside the carriage was collected by a microphone. The fighting and arguing-related features were extracted by audio detection and analysis branches to obtain the audio branch confidence of the fighting and arguing. Extract the grading results of personnel density inside the railcar; The image branch confidence and audio branch confidence are processed in parallel, and the branch weights of the image branch confidence and audio branch confidence are dynamically allocated based on the personnel density classification results in the railcar to obtain the fusion confidence and obtain the abnormal event judgment result of the car.

[0007] In some possible implementations, a camera is used to capture surveillance video inside the railcar, and features related to fighting and arguing are extracted through image pose detection branches to obtain image branch confidence scores for fighting and arguing behavior, including: Extracting the core pose key points of the human body from surveillance video and realizing the three-segment pose feature extraction of the human body; Extract the core conflict features of each part of the fight, and obtain the quantitative judgment score of each part according to the defined fight quantitative judgment rules; Based on the passenger density classification results in the carriage, the weight and priority of the characteristics of various parts of the human body are dynamically adjusted through the "density classification screening layer". A dual mechanism of coplanar verification and error filtering is constructed to avoid misjudgment of non-contact actions. Combined with temporal consistency verification and feature reliability verification, false detection caused by instantaneous actions and occlusion is eliminated. Based on the weights of features of different parts of the human body and the quantitative judgment scores of each part, the image branch confidence of fighting and arguing behavior is obtained by calculating the confidence level.

[0008] Some possible implementations include extracting core pose key points of the human body from surveillance video and achieving three-segment pose feature extraction, including: Based on surveillance video, the core posture key points of the human body are extracted, including nose 0, left eye 1, right eye 2, left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10, left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15, and right ankle 16. The three-segment pose feature extraction process involves dividing the 17 key points of human pose into a three-segment structure, including: Upper limbs: left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10; Lower limbs: left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15, right ankle 16; Trunk: Nose 0, eyes 1-2, shoulders 5-6, hips 11-12.

[0009] In some possible implementations, the core conflict features of each part of the fight are extracted, and quantitative judgment scores for each part are obtained according to the defined fight quantification judgment rules, including: The rules for quantifying fighting are defined as follows: Upper limb punching features include a shoulder-elbow angle ≤ 90° with the elbow as the apex and an elbow-wrist angle ≤ 60° with the wrist as the apex, with an angle change rate ≥ 30° / frame for 3 consecutive frames; Lower limb kicking features include a hip-knee angle ≤ 60° with the knee as the apex and a knee-ankle angle ≤ 45° with the ankle as the apex, with ankle displacement ≥ 50% of the lower leg length / frame for 4 consecutive frames; Trunk confrontation features include an angle ≥ 120° between the trunk axes of the two targets (i.e., the line connecting the midpoints of the shoulders to the midpoints of the hips) or an angle ≥ 15° between the trunk axis and the vertical direction, with key points of the target's hands contacting the ground or a low-lying object, and key points of the target's lower limbs being stationary and without displacement. For each part of the fight, if the core conflict characteristics are met, 1 point is awarded; if not, 0 points are awarded, thus obtaining the quantitative judgment score for each part.

[0010] In some possible implementations, based on the weights of features of different parts of the human body and the quantification scores of each part, the image branch confidence of the fighting / arguing behavior is calculated through confidence level calculation. The process is as follows: Calculate the initial confidence level P init : P init =S 上肢 ×W 上肢 +S 躯干 ×W 躯干 +S 下肢 ×W 下肢 ; In the formula, S is the quantitative judgment score of the part, and W is the weight ratio under the current density; The initial confidence level P init Multiplying the corrected confidence level by the coplanar validation correction factor α, the temporal consistency correction factor β, and the feature reliability correction factor γ, we obtain the corrected confidence level: P 修正 =P init ×α×β×γ; The corrected confidence level P 修正 Normalized to the [0,1] interval, it is used as the image branch confidence P_vision of the fighting behavior.

[0011] In some possible implementations, raw audio from inside the carriage is captured via microphone, and features related to fighting and arguing are extracted through audio detection and analysis branches to obtain the audio branch confidence scores for fighting and arguing, including: The raw audio captured by the microphone is sampled and preprocessed, and various types of interference are uniformly suppressed by "basic noise modeling + spectral subtraction". Based on the set energy threshold, the target sound features after coarse energy screening are obtained; Core features of 12-dimensional Mel frequency cepstral coefficients are extracted based on target sound features; One-dimensional short-time zero-crossing rate feature is extracted by counting the number of sign changes between adjacent sampling points; The 12-dimensional Mel frequency cepstral coefficient core feature and the 1-dimensional short-time zero-crossing rate feature are concatenated in sequence to form a 13-dimensional feature vector, which is directly used as the input of the CNN model and outputs the normalized confidence score P_audio of the fighting and arguing audio branch. The target sound filtering threshold and temporal smoothing window are dynamically adjusted based on the personnel density information. If the number of consecutive frames P_audio ≥ 0.5, it is determined to be a valid target sound segment, and the final confidence score P_audio of the fighting and arguing audio branch is output.

[0012] In some possible implementations, the image branch confidence and audio branch confidence are processed in parallel, and the branch weights of the image branch confidence and audio branch confidence are dynamically allocated based on the passenger density classification results in the railcar to obtain the fused confidence, thus obtaining the abnormal event determination result of the railcar, including: The fusion confidence level is calculated using a weighted summation formula: P_fusion=W_audio×P_audio+W_vision×P_vision; Wherein, W_audio+W_vision are the weights of the two branches, satisfying W_audio+W_vision=1. The weights of the two branches are assigned based on the personnel density classification results: low density scene W_audio=0.7-0.8, W_vision=0.2-0.3; medium density scene W_audio=0.5-0.6, W_vision=0.4-0.5; high density scene W_audio=0.3-0.45, W_vision=0.55-0.7; Set a fusion confidence threshold. If P_fusion ≥ the threshold, it is initially judged as abnormal. A majority vote is performed on the fusion results of several consecutive frames. If ≥15 frames are judged as abnormal, a final alarm is output.

[0013] Secondly, the present invention also provides an audio-visual detection device for abnormal events in rail transit carriages, comprising: The human detection module is configured to use cameras to collect surveillance video inside the railcar, extract fighting and arguing-related features through image pose detection branch, and obtain the image branch confidence of fighting and arguing behavior. The audio detection module is configured to collect the original audio inside the carriage through a microphone, extract the fighting and arguing-related features through audio detection and analysis branches, and obtain the audio branch confidence of the fighting and arguing. The density grading module is configured to extract the density grading results of people inside the railcar; The anomaly detection module is configured to process the image branch confidence and the audio branch confidence in parallel, and dynamically allocate the branch weights of the image branch confidence and the audio branch confidence based on the personnel density classification results in the railcar to obtain the fusion confidence and obtain the anomaly event detection result of the car.

[0014] Thirdly, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform any of the methods described herein.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including computer instructions for causing a computer to perform any of the methods described herein.

[0016] Because the present invention adopts the above technical solution, it has the following characteristics: 1. Overcome the limitations of single-modal detection and improve the comprehensiveness and accuracy of abnormal event identification: This invention compensates for the misjudgment defects of single visual detection in high-density occlusion scenarios and the blind spot of single audio detection in recognizing physical conflicts without obvious sounds by using parallel processing and deep fusion of audio and video branches. At the same time, the three-segment feature extraction and effectiveness screening mechanism of the image branch, combined with the interference suppression and energy coarse screening design of the audio branch, greatly improves the feature capture capability of different types of abnormal events such as fighting and arguing, and reduces the missed detection and false detection rates caused by environmental interference.

[0017] 2. Enable dynamic adaptation of personnel density to enhance the versatility of different scenarios: Based on personnel density grading technology, this invention can dynamically adjust the feature weights, detection priorities, and timing verification rules of each part of the image branch, and simultaneously adjust the screening threshold and timing smoothing window of the audio branch. This allows the detection strategy to accurately match different operating scenarios such as empty (low density), off-peak (medium density), and peak (high density) subways / high-speed trains. It solves the problem that the generalized detection logic of existing technologies cannot cope with the dynamic changes in passenger density in carriages, ensuring the reliability of detection in various scenarios.

[0018] 3. Optimize the accuracy of fusion decision-making and enhance detection robustness: This invention employs a density-linked dynamic weight allocation strategy, which, compared to existing multimodal technologies with fixed or initial scene weight adjustments, can match changes in the carriage environment in real time, ensuring that the weights of audio and video branches are adapted to the effectiveness of features under the current density. Combined with coplanar verification, temporal consistency verification, and dual-branch linkage optimization mechanisms, it further corrects errors caused by instantaneous actions and occlusions, significantly improving the accuracy of fusion decisions. At the same time, the temporal verification rules based on majority voting enhance the stability of anomaly detection.

[0019] 4. It possesses advantages in engineering deployment and is adapted to the actual operational needs of rail transit: The image branch of this invention can adopt a lightweight YOLOv8-Pose model combined with pruning optimization, while the audio branch adopts a minimalist 2-layer CNN architecture with low single-frame inference time. At the same time, the basic noise modeling and spectral subtraction of audio preprocessing simplify the interference suppression logic. The overall solution balances detection performance and computational efficiency, is adaptable to the hardware deployment environment of rail transit carriages, and the standardized confidence output and millisecond-level time-series alignment design facilitates integration with existing operation monitoring systems, making it highly practical.

[0020] In summary, this invention can be widely applied to the accurate detection of fighting and arguing behavior in rail transit. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1 This is a flowchart of a method for detecting abnormal events in rail transit carriages based on density-level audio and video, according to an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating the three-stage posture feature extraction and fighting behavior determination method according to an embodiment of the present invention.

[0023] Figure 3 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0025] Although terms such as first, second, third, etc., may be used in this document to describe multiple elements, components, regions, layers, and / or segments, these elements, components, regions, layers, and / or segments should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or segment from another. Unless the context clearly indicates otherwise, terms such as "first," "second," and other numerical terms used herein do not imply order or sequence. Therefore, the first element, component, region, layer, or segment discussed below may be referred to as the second element, component, region, layer, or segment without departing from the teachings of the exemplary embodiments.

[0026] For ease of description, spatial relative terms may be used in the text to describe the relationship of one element or feature relative to another element or feature as shown in the figure. These relative terms include, for example, "inside," "outside," "middle," "outer," "below," "above," etc. Such spatial relative terms are intended to include different orientations of the device in use or operation, other than those depicted in the figure.

[0027] Existing methods for detecting abnormal behaviors such as fighting and arguing in subway and high-speed rail carriages have inherent limitations, with insufficient reliability adaptability to different personnel density scenarios. This invention provides a method, apparatus, equipment, and medium for detecting abnormal events in rail transit carriages based on density-based audio and video detection. These include: a dynamic weighted fusion decision method for audio and video based on personnel density: using the carriage's personnel density classification results as the core basis, dynamically allocating the weight ratios of the image pose detection branch and the audio detection and analysis branch (adapting to different weight requirements in low, medium, and high density scenarios), calculating the fusion confidence score through weighted summation, and outputting the anomaly judgment result by combining a sequential verification rule of majority voting in consecutive frames. Simultaneously, it achieves linked optimization of dual-branch parameters, improving the robustness of abnormal event detection in different scenarios; and an audio interference suppression and target sound screening method adapted to rail transit carriage scenarios: establishing a basic noise spectrum library by collecting background interference samples during subway empty periods, and using "basic noise modeling + spectral subtraction" to uniformly suppress interference such as operational noise and announcements. Simultaneously, it dynamically adjusts the high-energy target sound screening threshold and temporal smoothing window based on personnel density, solving the problem of general audio detection... The problem of not being able to adapt to the complex noise environment of the carriage; a three-segment posture feature extraction and density adaptive occlusion adaptation detection method: divides the 17 core posture key points of the human body into a three-segment structure of upper limbs, lower limbs and torso, defines quantitative judgment rules for the core conflict of each part of the body, and dynamically adjusts the detection priority and weight ratio of each part of the body based on the passenger density classification results in the carriage, filters invalid features in occluded scenes, and improves the accuracy of fighting behavior detection by combining mechanisms such as coplanar verification and temporal consistency verification; a fusion method of quantitative judgment and multi-dimensional error correction for fighting behavior posture: defines clear quantitative judgment rules (such as angle and displacement threshold) for upper limb punching, lower limb kicking and torso confrontation, and integrates a triple error correction mechanism of coplanar verification (foot distance / detection box height verification), temporal consistency verification (density setting of consecutive effective frames), and feature reliability verification (key point missing threshold), which breaks through the simple judgment logic of existing technologies that only rely on key point displacement or limb intersection. Compared with existing technologies, this invention directly solves the inherent defects of single-modality detection and the problem of insufficient detection reliability adaptation under different personnel density scenarios by using the core technology design of dual-branch dynamic adaptation driven by personnel density classification. Furthermore, through dual-branch parallel collaboration, visual and auditory features are extracted in parallel and aligned in sequence, combined with a linkage optimization mechanism to improve the comprehensiveness and accuracy of detection. Moreover, by dynamically adjusting the weights of audio and video branches based on personnel density, it can improve the detection capability of fighting and arguing behavior in subway and high-speed rail scenarios.

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0029] Example 1: As Figure 1 As shown, the method for detecting abnormal events in rail transit carriages based on density grading provided by this invention starts with "multi-source data acquisition," extracts relevant features of fighting and arguing through parallel "image pose detection branch" and "audio detection and analysis branch," uses "personnel density grading" to provide scene adaptation criteria, and finally outputs accurate anomaly judgment results through "dynamic weight fusion decision." The entire method uses personnel density as the core adaptation criterion, realizes dynamic optimization of dual-branch feature extraction, parameter adjustment, and weight allocation, and solves the occlusion and interference problems under different density scenes. The specific implementation process is as follows: S1. Use cameras to collect surveillance videos inside subway / high-speed rail carriages, extract fighting and arguing-related features through image pose detection branches, and obtain the image branch confidence of fighting and arguing behavior.

[0030] In this embodiment, a camera is used to capture surveillance video inside subway / high-speed rail carriages. Fighting and arguing-related features are extracted through image pose detection, and the image branch confidence of the fighting and arguing behavior is obtained, including: S11. Extract the core pose key points of the human body based on surveillance video.

[0031] In this embodiment, a lightweight pose detection model, YOLOv8-Pose, can be used. The backbone core components are the CBS module (composed of Conv, BN, and SiLU activation functions) and the C2f module, with the SPPF module at the network end. The Neck module fuses the multi-scale feature maps output by the backbone. Data augmentation is achieved by expanding the labeled poses related to fighting in low-density, medium-density, and high-density scenes in a railcar. Combined with model pruning optimization, 17 core pose key points of the human body are extracted in real time from the input video frame, providing basic data support for subsequent three-segment feature extraction. These 17 core pose key points include: torso (nose 0), left eye (1), right eye (2), left ear (3), right ear (4), left shoulder (5), right shoulder (6), left elbow (7), right elbow (8), left wrist (9), right wrist (10), left hip (11), right hip (12), left knee (13), right knee (14), left ankle (15), and right ankle (16).

[0032] Furthermore, this implementation can also use models such as the SOSD-Pose model to replace the YOLOv8-Pose series models for extracting key points of human core pose. By adjusting the model input resolution and anchor frame parameters to adapt to the carriage scene, it can still achieve effective extraction of three-segment pose features and adaptive density filtering. The specific process will not be elaborated here.

[0033] S12. Based on the extracted human core pose key points, realize three-segment pose feature extraction.

[0034] In this embodiment, as Figure 2 As shown, the three-stage pose feature extraction process includes: Using a sequence of 17 human posture key points as input, the sequence is divided into three parts—upper limbs, lower limbs, and torso—through a "three-segment structure division," and core features of each part are extracted. Specifically, the 17 human posture key points are divided into three segments: upper limbs (left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10), lower limbs (left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15, right ankle 16), and torso (nose 0, eyes 1-2, shoulders 5-6, hips 11-12). Core conflict features of fighting are extracted from each part. The limb crossing angle within a frame corresponds to the change in fighting angle and appears within the extracted consecutive frames. Clear fighting quantification rules are defined for upper limb punches, lower limb kicks, and torso confrontation in fighting behavior. This embodiment uses a monitoring video with a frame rate of 25fps (frame interval 40ms) to illustrate the fighting quantification rules. This is an example, but not limited to this. Specifically: The characteristics of upper limb punching are determined by the rate of change of shoulder-elbow angle and elbow-wrist angle. The shoulder-elbow angle (angle between shoulder, elbow, and wrist) with the elbow as the apex is ≤90°, the elbow-wrist angle (angle between elbow, wrist, and shoulder) with the wrist as the apex is ≤60°, and the angle change rate for three consecutive frames is ≥30° / frame. Lower limb kicking characteristics are determined by hip-knee angle, knee-ankle angle and ankle displacement: hip-knee angle (angle between hip, knee and ankle) with the knee as the apex is ≤60°, knee-ankle angle (angle between knee, ankle and hip) with the ankle as the apex is ≤45°, and ankle displacement is ≥50% of the lower leg length per frame for 4 consecutive frames; The trunk confrontation characteristics are determined by the trunk axis angle and key point position: the angle between the trunk axes of the two targets (the line connecting the midpoint of the shoulders to the midpoint of the hips) is ≥120° or the angle between the trunk axis and the vertical direction is ≥15°, and the key points of the target's hands (9 / 10) are in contact with the ground or low objects (such as picking something up), and the key points of the target's lower limbs (15 / 16) are in a static state and have no displacement (such as tying shoelaces).

[0035] S13. Based on the passenger density classification results in the carriage, the weight and priority of the features of each part are dynamically adjusted through the "density classification screening layer".

[0036] In this embodiment, the process of dynamically adjusting the weight and priority of features of each part through the "density grading screening layer" based on the passenger density grading results of the carriage is as follows: Based on the number of human detections within the carriage area in the current frame and the density calculated using the standard carriage area, the detection priority and validity filtering rules for each part are dynamically adjusted according to the carriage personnel density classification results, filtering out invalid features in occluded scenes: In high-density scenarios, the focus is on "upper limb features as the core, trunk features as the auxiliary, and lower limb features as the supplement," for example, setting the weight of the upper limbs to 0.7, the weight of the trunk to 0.2, and the weight of the lower limbs to 0.1. Medium-density scenes prioritize upper limb and trunk features, with lower limb features as secondary features. For example, the weight of upper limbs is set to 0.5, trunk to 0.3, and lower limbs to 0.2. In low-density scenes, all three-segment features are retained and used in a balanced manner for judgment. For example, the weights of the upper limbs are set to 0.4, the torso to 0.3, and the lower limbs to 0.3, to ensure the accuracy of feature utilization in different occlusion scenarios.

[0037] S14. The validity of the feature is ensured through “double verification” (including target coplanar verification and error filtering).

[0038] In this embodiment, a dual mechanism of coplanar verification and error filtering is constructed: the vertical distance difference between the target landing points (distance difference ≤ 0.1 times the image height) or the difference in detection box height (difference ≤ 0.3 times) is used to verify whether two targets are on the same plane, avoiding misjudgment of non-contact actions; combined with temporal consistency verification and feature reliability verification—high-density scenes require effective features to appear continuously for ≥ 3 frames, medium-density scenes require ≥ 4 frames, and low-density scenes require ≥ 5 frames, while verifying the integrity of core key points of each part (upper limb missing ≤ 1, trunk missing ≤ 2, lower limb missing ≤ 1), eliminating false detections caused by instantaneous actions and occlusion.

[0039] S15. The feature score, weighting and correction are completed through "confidence calculation", and finally the standardized confidence of fighting and arguing behavior is output to achieve accurate judgment in different scenarios.

[0040] In this embodiment, the confidence level calculation and standardized output are implemented. The specific process is as follows: Calculate the initial confidence level P init For each part, a validity score is calculated based on the principle of "1 point for meeting the feature quantification criteria, and 0 points for not meeting them." Then, the score is determined according to the weighting of the corresponding density scene using the formula: P init =S 上肢 ×W 上肢+S 躯干 ×W 躯干 +S 下肢 ×W 下肢 In the formula, S is the quantitative judgment score of the part, and W is the weight ratio under the current density.

[0041] The initial confidence level P init Multiplying the corrected confidence level by the coplanar verification correction coefficient α (1 for passing, 0.1 for failing), the temporal consistency correction coefficient β (1 for meeting the consecutive frame count requirement, 0.2 for failing), and the feature reliability correction coefficient γ (1 for complete keypoints, 0.5 for incomplete keypoints), we obtain the corrected confidence level. P 修正 =P init ×α×β×γ The corrected confidence score is normalized to the [0,1] interval and used as the image branch confidence score P_vision for the fighting behavior. It can simultaneously output density level, target number (multiple human bodies will be detected in one frame, and key points of different human limbs belong to different targets and will have an assigned number), effective parts (upper limb / lower limb / torso), number of consecutive effective frames and millisecond-level timestamp, forming a standardized output result that is consistent with the audio branch confidence score format, which can be directly used for fusion decision.

[0042] S2. Collect the original audio inside the carriage through the microphone, extract the relevant features of fighting and arguing through audio detection and analysis branches, and obtain the audio branch confidence P_audio of fighting and arguing.

[0043] In this embodiment, the original audio inside the carriage is collected through a microphone, and features related to fighting and arguing are extracted through audio detection and analysis branches. The confidence score P_audio of the audio branch for fighting and arguing is obtained. The specific process is as follows: S21. The raw audio captured by a single microphone is sampled at 16kHz and normalized with a 16-bit bit depth. It is then framed at 20ms / frame (frame overlap rate of 40%), and a Hanning window is added to reduce spectral leakage. DC noise is removed by a 150Hz high-pass filter. A millisecond-level timestamp is added to each audio frame to ensure that the timing alignment error with the video frame is ≤10ms.

[0044] S22, Unified interference suppression characteristics.

[0045] In this embodiment, a simplified interference suppression method of "basic noise modeling + spectral subtraction" is adopted. Various background interference samples (operating noise, broadcast sounds, etc.) are collected during the subway's empty period to establish a basic noise spectral library. Spectral subtraction is then directly applied to the preprocessed audio. (X(ω)=|Y(ω)|-β×|N(ω)|) In the formula, β is an adjustable coefficient of 0.85-0.95, X is the spectrum after noise reduction, Y is the original spectrum, and N is the spectrum of the noise library, so as to achieve unified suppression of various interferences and ensure that the target sound has no obvious distortion.

[0046] S23, Target sound coarse screening features.

[0047] In this embodiment, the audio energy baseline E_base during a stable and interference-free period in the subway is statistically analyzed. Using "current frame energy ≥ 1.8 × E_base" as the threshold, high-energy candidate frames are quickly screened (fighting and arguing sounds are mostly high-energy signals), and low-energy background interference frames are directly filtered out, reducing the complexity of subsequent processing.

[0048] S24, binary classification detection features.

[0049] In this embodiment, feature extraction and combination are performed as follows: For high-energy candidate frames (20ms / frame, 16kHz sampling, 16bit depth) after preprocessing, interference suppression, and energy coarse screening, 12-dimensional Mel frequency cepstral coefficients (MFCC) features are extracted. Specifically, a 512-point FFT (20ms / frame, 16kHz sampling, 320 sampling points in frame length, with the remainder padded with 0) is performed to convert the signal into a 0-8kHz frequency domain signal. The signal is then filtered by 12 triangular Mel filter banks to obtain the Mel spectrum energy vector, and then the 12-dimensional MFCC core features are generated through logarithmic transformation + 12-point DCT transformation. Extract 1D short-time zero-crossing rate feature: count the number of times the symbol changes between adjacent sampling points within a frame (the number of times the symbol (positive / negative) of two adjacent sampling points changes in a frame of speech signal), divide by the frame length (320 sampling points) and normalize to the [0,1] interval; The 12-dimensional MFCC features and the 1-dimensional short-time zero-crossing rate features are concatenated sequentially to form a 13-dimensional feature vector, which can be directly used as the input to the CNN model without dimensionality reduction. The CNN model can adopt a 2-layer CNN model structure and inference: through a minimalist architecture of "convolution-pooling-convolution-pooling-fully connected", it focuses on the extraction of sound features in the subway scene, with a single-frame inference time of ≤1ms, and finally outputs the normalized confidence score P_audio (∈[0,1]) of the fighting and arguing audio.

[0050] S25, density adaptive robustness optimization feature.

[0051] In this embodiment, the target sound screening threshold (low density E≥1.8×E0, medium density E≥2×E0, high density E≥2.5×E0) and the temporal smoothing window (3 frames for low density, 5 frames for medium density, and 8 frames for high density) are dynamically adjusted based on personnel density information. If P_audio≥0.5 for ≥6 consecutive frames, it is determined to be a valid target sound segment, further improving detection stability. The final audio confidence P_audio and timestamp are output synchronously to reduce the single-frame false positive rate and adapt to fusion requirements.

[0052] Furthermore, interference suppression of audio branches can also be achieved by using wavelet thresholding to replace spectral subtraction. Combined with the core logic of energy coarse screening and MFCC feature extraction, it can also achieve accurate identification and temporal alignment of the target sound.

[0053] S3. Extract personnel density grading technical features.

[0054] In this embodiment, the technical features for extracting personnel density grading include: S31. Classification based on characteristics: Based on the number of human detections output by the image pose detection branch, combined with the standard area of ​​the carriage covered by a single camera (approximately 20m²), the personnel density value (number of people / 20m²) is calculated.

[0055] S32, Three-level hierarchical rule characteristics: It is divided into three levels: low density (≤10 people, density ≤0.5 people / m²), medium density (11-30 people, 0.5 people / m² < density ≤1.5 people / m²), and high density (>30 people, density >1.5 people / m²), which are suitable for subway / high-speed rail empty, off-peak and peak operation scenarios.

[0056] S33. Synchronization characteristics of hierarchical results: The image pose detection branch outputs density grading results in real time, which are synchronized to the fusion decision and audio detection analysis branches, providing a basis for adjusting the parameters of the two branches and dynamically allocating weights.

[0057] S4. The image pose detection branch and the audio detection and analysis branch are processed in parallel, and the weights of the two branches are dynamically allocated based on the personnel density classification results to obtain the fusion confidence and the final anomaly judgment result.

[0058] In this embodiment, the image pose detection branch and the audio detection and analysis branch are processed in parallel, and the weights of the two branches are dynamically allocated based on the personnel density classification results to obtain the fusion confidence score and the final anomaly determination result. S41, Density-linked dynamic weight allocation feature: The weights of the two branches are assigned based on the personnel density classification results, satisfying W_audio+W_vision=1: low density scene W_audio=0.7-0.8, W_vision=0.2-0.3; medium density scene W_audio=0.5-0.6, W_vision=0.4-0.5; high density scene W_audio=0.3-0.45, W_vision=0.55-0.7.

[0059] S42. Calculate the fusion confidence level using the weighted summation formula: P_fusion=W_audio×P_audio+W_vision×P_vision, Where P_audio is the confidence score of the audio branch, P_vision is the confidence score of the image branch, and the output fusion confidence score P_fusion∈[0,1].

[0060] S43. Characteristics for determining anomalies in timing verification: Set a fusion confidence threshold (adjustable from 0.7 to 0.9). If P_fusion ≥ the threshold, it is initially judged as abnormal. Perform majority voting on the fusion results of 20 consecutive frames (200ms). If ≥15 frames are judged as abnormal, a final alarm is output. This is an example, but not limited to this.

[0061] Furthermore, when the confidence level of the audio branch P_audio ≥ 0.6, a key region detection signal is sent to the image branch. The image branch improves the detection accuracy of the pose key points in the corresponding region, so that the fusion decision is linked with the parameters of the two branches. For example, in high-density scenes, the audio branch increases the screening threshold, and the fusion strategy adjusts the weight ratio of the video branch in sync.

[0062] Example 2: Following the method for detecting abnormal events in rail transit carriages using audio and video as described in Example 1, this example provides an apparatus for detecting abnormal events in rail transit carriages using audio and video. The apparatus provided in this example can implement the method for detecting abnormal events in rail transit carriages using audio and video as described in Example 1. This apparatus can be implemented through software, hardware, or a combination of both. For ease of description, this example is described by dividing the functions into various units. Of course, in implementation, the functions of each unit can be implemented in one or more software and / or hardware components. For example, the apparatus may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Example 1. Since the apparatus in this example is basically similar to the method example, the description process of this example is relatively simple. Relevant details can be found in the description of the parts in Example 1. The embodiment of the apparatus for detecting abnormal events in rail transit carriages using audio and video provided by this invention is merely illustrative.

[0063] Specifically, the device for detecting abnormal events in rail transit carriages using audio and video provided in this embodiment includes: The human detection module is configured to use cameras to collect surveillance video inside the railcar, extract fighting and arguing-related features through image pose detection branch, and obtain the image branch confidence of fighting and arguing behavior. The audio detection module is configured to collect the original audio inside the carriage through a microphone, extract the fighting and arguing-related features through audio detection and analysis branches, and obtain the audio branch confidence of the fighting and arguing. The density grading module is configured to extract the density grading results of people inside the railcar; The anomaly detection module is configured to process the image branch confidence and the audio branch confidence in parallel, and dynamically allocate the branch weights of the image branch confidence and the audio branch confidence based on the personnel density classification results in the railcar to obtain the fusion confidence and obtain the anomaly event detection result of the car.

[0064] Example 3: This example provides an electronic device corresponding to the method for detecting abnormal events in rail transit carriages using audio and video provided in Example 1. The electronic device can be an electronic device for client use, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Example 1.

[0065] like Figure 3 As shown, the electronic device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the method of Embodiment 1. The implementation principle and technical effects are similar to those of Embodiment 1, and will not be repeated here. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device on which the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0066] In a preferred embodiment, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), and optical discs.

[0067] In a preferred embodiment, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0068] Example 4: This example provides a computer-readable storage medium for storing one or more programs, the one or more programs including computer instructions, which, when executed by a computer, cause the computer to perform the method provided in Example 1 above.

[0069] In a preferred embodiment, the computer-readable storage medium may be a tangible device for holding and storing instructions executable, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to perform the method provided in Embodiment 1 above.

[0070] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0073] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In the description of this specification, the terms "a preferred embodiment," "furthermore," "specifically," "in this embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting abnormal events in rail transit carriages using audio and video, characterized in that, include: The system uses cameras to collect surveillance videos inside the railcar, extracts fighting and arguing-related features through image pose detection branches, and obtains the image branch confidence of fighting and arguing behavior. The original audio inside the carriage was collected by a microphone. The fighting and arguing-related features were extracted by audio detection and analysis branches to obtain the audio branch confidence of the fighting and arguing. Extract the grading results of personnel density inside the railcar; The image branch confidence and audio branch confidence are processed in parallel, and the branch weights of the image branch confidence and audio branch confidence are dynamically allocated based on the personnel density classification results in the railcar to obtain the fusion confidence and obtain the abnormal event judgment result of the car.

2. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 1, characterized in that, The system uses cameras to capture surveillance video inside the railcar, extracts features related to fighting and arguing through image pose detection, and obtains the image branch confidence scores for fighting and arguing behavior, including: Extracting the core pose key points of the human body from surveillance video and realizing the three-segment pose feature extraction of the human body; Extract the core conflict features of each part of the fight, and obtain the quantitative judgment score of each part according to the defined fight quantitative judgment rules; Based on the passenger density classification results in the carriage, the weight and priority of the characteristics of various parts of the human body are dynamically adjusted through the "density classification screening layer". A dual mechanism of coplanar verification and error filtering is constructed to avoid misjudgment of non-contact actions. Combined with temporal consistency verification and feature reliability verification, false detections caused by instantaneous actions and occlusion are eliminated. Based on the weights of features of different parts of the human body and the quantitative judgment scores of each part, the image branch confidence of fighting and arguing behavior is obtained by calculating the confidence level.

3. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 2, characterized in that, Based on surveillance video, the core pose key points of the human body are extracted, and a three-segment human pose feature extraction is achieved, including: Based on surveillance video, the core posture key points of the human body are extracted, including nose 0, left eye 1, right eye 2, left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10, left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15, and right ankle 16. The three-segment pose feature extraction process involves dividing the 17 key points of human pose into a three-segment structure, including: Upper limbs: left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10; Lower limbs: left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15, right ankle 16; Trunk: Nose 0, eyes 1-2, shoulders 5-6, hips 11-12.

4. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 2, characterized in that, Extract the core conflict features of each part of the fight, and obtain the quantitative judgment score for each part according to the defined fight quantification judgment rules, including: The rules for quantifying fighting are defined as follows: Upper limb punching features include a shoulder-elbow angle ≤ 90° with the elbow as the apex and an elbow-wrist angle ≤ 60° with the wrist as the apex, with an angle change rate ≥ 30° / frame for 3 consecutive frames; Lower limb kicking features include a hip-knee angle ≤ 60° with the knee as the apex and a knee-ankle angle ≤ 45° with the ankle as the apex, with ankle displacement ≥ 50% of the lower leg length / frame for 4 consecutive frames; Trunk confrontation features include an angle ≥ 120° between the trunk axes of the two targets (i.e., the line connecting the midpoints of the shoulders to the midpoints of the hips) or an angle ≥ 15° between the trunk axis and the vertical direction, with key points of the target's hands contacting the ground or a low-lying object, and key points of the target's lower limbs being stationary and without displacement. If the core conflict characteristics of each part of the fight are met, 1 point is awarded; if they are not met, 0 points are awarded, thus obtaining the quantitative judgment score for each part.

5. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 4, characterized in that, Based on the weights of features of different parts of the human body and the quantified judgment scores of each part, the image branch confidence of fighting and arguing behavior is calculated through confidence level calculation. The process is as follows: Calculate the initial confidence level P init : P init =S 上肢 ×W 上肢 +S 躯干 ×W 躯干 +S 下肢 ×W 下肢 ; In the formula, S is the quantitative judgment score of the part, and W is the weight ratio under the current density; The initial confidence level P init Multiplying the corrected confidence level by the coplanar validation correction factor α, the temporal consistency correction factor β, and the feature reliability correction factor γ, we obtain the corrected confidence level: P 修正 =P init ×α×β×γ; The corrected confidence level P 修正 Normalized to the [0,1] interval, it is used as the image branch confidence P_vision of the fighting behavior.

6. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 1, characterized in that, Raw audio from inside the carriage was captured using a microphone. Through audio detection and analysis, features related to fighting and arguing were extracted, and the confidence scores of the audio branches related to fighting and arguing were obtained, including: The raw audio captured by the microphone is sampled and preprocessed, and various types of interference are uniformly suppressed by "basic noise modeling + spectral subtraction". Based on the set energy threshold, the target sound features after coarse energy screening are obtained; Core features of 12-dimensional Mel frequency cepstral coefficients are extracted based on target sound features; One-dimensional short-time zero-crossing rate feature is extracted by counting the number of sign changes between adjacent sampling points; The 12-dimensional Mel frequency cepstral coefficient core feature and the 1-dimensional short-time zero-crossing rate feature are concatenated in sequence to form a 13-dimensional feature vector, which is directly used as the input of the CNN model and outputs the normalized confidence score P_audio of the fighting and arguing audio branch. The target sound filtering threshold and temporal smoothing window are dynamically adjusted based on the personnel density information. If the number of consecutive frames P_audio ≥ 0.5, it is determined to be a valid target sound segment, and the final confidence score P_audio of the fighting and arguing audio branch is output.

7. The method for detecting abnormal events in rail transit carriages using audio and video as described in claim 6, characterized in that, Image branch confidence and audio branch confidence are processed in parallel, and the branch weights of image branch confidence and audio branch confidence are dynamically assigned based on the passenger density classification results in the railcar to obtain the fused confidence score, thus obtaining the abnormal event determination result of the railcar, including: The fusion confidence level is calculated using a weighted summation formula: P_fusion=W_audio×P_audio+W_vision×P_vision; Wherein, W_audio+W_vision are the weights of the two branches, satisfying W_audio+W_vision=1. The weights of the two branches are assigned based on the personnel density classification results: low density scene W_audio=0.7-0.8, W_vision=0.2-0.3; medium density scene W_audio=0.5-0.6, W_vision=0.4-0.5; high density scene W_audio=0.3-0.45, W_vision=0.55-0.7; Set a fusion confidence threshold. If P_fusion ≥ the threshold, it is initially judged as abnormal. A majority vote is performed on the fusion results of several consecutive frames. If ≥15 frames are judged as abnormal, a final alarm is output.

8. A device for detecting abnormal events in rail transit carriages using audio and video, characterized in that, include: The human detection module is configured to use cameras to collect surveillance video inside the railcar, extract fighting and arguing-related features through image pose detection branch, and obtain the image branch confidence of fighting and arguing behavior. The audio detection module is configured to collect the original audio inside the carriage through a microphone, extract the fighting and arguing-related features through audio detection and analysis branches, and obtain the audio branch confidence of the fighting and arguing. The density grading module is configured to extract the density grading results of people inside the railcar; The anomaly detection module is configured to process the image branch confidence and the audio branch confidence in parallel, and dynamically allocate the branch weights of the image branch confidence and the audio branch confidence based on the personnel density classification results in the railcar to obtain the fusion confidence and obtain the anomaly event detection result of the car.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images