A method, device, equipment, and medium for detecting conflicts between passengers in a train carriage.

Abstract

This invention relates to a method, apparatus, equipment, and medium for detecting conflicts among passengers in a train carriage. Specifically, the detection method involves: acquiring real-time video data using a camera sensor and monitoring the video data quality in real time; extracting frames from the video data according to a pre-set video frame sampling time interval and performing preprocessing operations to obtain keyframe images; identifying potential conflict behaviors based on the keyframe images using a pre-trained model; identifying the target to be monitored based on the identification results, and acquiring audio data of the target, as well as surrounding audio data and other passenger behavior data; analyzing and determining the presence of conflict behaviors based on multimodal information identification; and triggering an early warning mechanism and issuing an early warning signal when conflict behaviors are detected at the target, thereby ensuring passenger safety, maintaining carriage order, and ensuring a harmonious train operation environment.

Description

Technical Field

[0001] This invention relates to the field of train safety monitoring technology, and in particular to a method, device, equipment and medium for detecting conflicts between people inside a train carriage. Background Technology

[0002] Train carriages are relatively enclosed and densely populated environments. Conflicts within a train can directly threaten passenger safety and disrupt normal order. The noise and violence during a conflict can cause panic and anxiety among other passengers, leading to a tense, chaotic, and unstable atmosphere that negatively impacts the travel experience. This not only affects the individuals involved but also introduces additional legal risks and pressures to train management and operations. Therefore, the rapid detection and prevention of conflicts on trains are crucial for ensuring passenger safety, maintaining carriage order, avoiding legal disputes and liabilities, and protecting the reputation and image of the railway company. Summary of the Invention

[0003] The main objective of this invention is to provide a method, device, equipment, and medium for detecting conflicts among passengers in train carriages, so as to ensure the safety of passengers in train carriages, maintain order in the carriages, and maintain a harmonious operating environment for the train.

[0004] To achieve the above objectives, the present invention provides a method for detecting conflicts among occupants in a train carriage, comprising the following steps:

[0005] Real-time video data is collected through camera sensors, and the quality of the video data is monitored in real time.

[0006] The video data is sampled at a pre-set video frame sampling time interval and preprocessed to obtain keyframe images.

[0007] Based on the keyframe images, potential conflict behaviors are identified using a pre-trained model.

[0008] Based on the identification results, the target to be monitored is located, and audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, are collected.

[0009] Based on multimodal information identification, analyze and determine multiple features that indicate the existence of conflicting behaviors;

[0010] When a conflict occurs at a target to be monitored, an early warning mechanism is triggered and an early warning signal is issued. The early warning signal includes the location of the carriage, the level of crowding in the carriage, and the number of people involved in the conflict.

[0011] Furthermore, the step of acquiring real-time video data through a camera sensor and monitoring the video data quality in real time includes:

[0012] The train continuously captures real-time video data inside the carriages using camera sensors, which are wide-angle lenses with night vision capabilities. Multiple camera sensors are installed in the train carriages.

[0013] Real-time monitoring of the clarity and smoothness of captured video;

[0014] When video data loss or quality degradation to a set threshold is detected, an alarm mechanism is triggered, and an alarm message is sent to designated personnel. The alarm message includes the time, location, and type of the anomaly.

[0015] While triggering the alarm mechanism, the system collects audio data from the carriage in real time via a voice device, and other camera sensors collect passenger behavior data from the carriage.

[0016] Further, the step of extracting frames from the video data according to a pre-set video frame sampling time interval and performing preprocessing operations to obtain keyframe images includes:

[0017] Based on stable real-time video data, frames are extracted at set time intervals to obtain a series of initial keyframe images.

[0018] The initial keyframe image is preprocessed by denoising, enhancement, and edge detection to obtain the final keyframe image.

[0019] Furthermore, the step of identifying potential conflict behaviors based on the keyframe images using a pre-trained model includes:

[0020] Collect video data of normal and abnormal human postures and behaviors in various carriage spaces and preprocess them to obtain a passenger behavior video dataset;

[0021] Based on the aforementioned passenger behavior video dataset, a potential conflict behavior model is obtained by training a three-dimensional convolutional neural network model.

[0022] The keyframe images are input into the potential conflict behavior model to identify potential conflict behaviors of people in the carriage.

[0023] Furthermore, the step of training a three-dimensional convolutional neural network model based on the passenger behavior video dataset to obtain a potential conflict behavior model includes:

[0024] Based on the preprocessed passenger behavior video dataset, a pose estimation algorithm is used to identify key human points and label the corresponding location information.

[0025] Convert the key point location information into a data format suitable for machine learning models;

[0026] The converted key point location information is used as an additional feature channel and spliced ​​with the spatiotemporal feature channel of the video data; the spatiotemporal features are used to capture motion information and spatial structure information between image frames;

[0027] The spliced ​​behavioral features are used as input to a 3D convolutional neural network model for deep learning to perform conflict behavior recognition tasks and obtain the final potential conflict behavior model.

[0028] Furthermore, the step of locating the target to be monitored based on the identification result, and collecting audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, includes:

[0029] Based on the model's identification results, identify targets that have potential conflict behaviors and require monitoring.

[0030] According to a preset duration, the system continuously monitors the video data of the target to be monitored, collects audio data and other passenger behavior data within a specified range of the target to be monitored.

[0031] Furthermore, the step of identifying, analyzing, and determining whether conflicting behaviors exist based on multimodal information includes:

[0032] The collected multimodal information is identified, including video data of the target to be monitored, audio data within a specified range of the target to be monitored, and other passenger behavior data;

[0033] The data in the multimodal information are identified separately to obtain the feature judgment of conflict behavior;

[0034] When the characteristics of the audio data or the passenger behavior data indicate potential conflict behavior, it is determined that the target to be monitored is engaging in conflict behavior.

[0035] The present invention also provides a train carriage passenger conflict detection device, comprising:

[0036] The data acquisition module is used to acquire real-time video data through a camera sensor and monitor the video data quality in real time.

[0037] The preprocessing module is used to extract frames from the video data according to a preset video frame sampling time interval and perform preprocessing operations to obtain keyframe images.

[0038] The model recognition module is used to identify potential conflict behaviors based on the keyframe images using a pre-trained model.

[0039] The monitoring module is used to locate the target to be monitored based on the identification results, and to collect the audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data;

[0040] The analysis module is used for identification based on multimodal information, analyzing and determining multiple features that indicate the existence of conflicting behaviors;

[0041] The early warning module is used to trigger an early warning mechanism and issue an early warning signal when there is conflict behavior in the target to be monitored.

[0042] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for detecting conflicts between people in a train carriage.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for detecting conflicts among occupants in a train carriage.

[0044] The train carriage passenger conflict detection method, device, equipment, and medium provided by this invention have the following beneficial effects: Real-time acquisition of video data via camera sensors and monitoring of data quality ensure timely alarms when video data is abnormal, thereby guaranteeing the continuous effectiveness of the monitoring system; by combining video data, audio data, and other passenger behavior data, the system can more comprehensively identify and analyze the multi-features of conflict behavior, improving the accuracy and reliability of conflict detection; when potential conflict behavior is detected, the system can quickly trigger an early warning mechanism and issue an early warning signal. The detailed information contained in the early warning signal, such as carriage location, crowding level, and number of people involved in the conflict, helps train staff quickly locate the conflict scene, assess the conflict situation, and take appropriate measures. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a method for detecting conflicts among passengers in a train carriage according to an embodiment of the present invention.

[0046] Figure 2 This is a structural block diagram of a train carriage passenger conflict detection device according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] Reference Figure 1 This is a flowchart illustrating a method for detecting conflicts among passengers in a train carriage, as proposed in this invention, comprising the following steps:

[0051] S1 collects real-time video data through a camera sensor and monitors the quality of the video data in real time;

[0052] S2, the video data is sampled at a preset video frame sampling time interval and preprocessed to obtain keyframe images;

[0053] S3, Based on the keyframe images, identify potential conflict behaviors using a pre-trained model;

[0054] S4. Based on the identification results, locate the target to be monitored, and collect the audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data;

[0055] S5, based on multimodal information recognition, analyzes and determines whether there are multiple features of conflicting behavior;

[0056] S6. When the target to be monitored is involved in conflict behavior, an early warning mechanism is triggered and an early warning signal is issued. The early warning signal includes the location of the carriage, the degree of crowding in the carriage, and the number of people involved in the conflict.

[0057] As described in step S1 above, real-time video data is collected through camera sensors, and the quality of the video data is monitored in real time. Multiple camera sensors are installed inside the train carriages to comprehensively monitor the situation inside. These sensors continuously capture real-time video data from inside the train carriages, serving as the basis for subsequent conflict behavior identification and analysis. While collecting real-time video data, it is necessary to monitor the quality of the video data in real time, including both image clarity and smoothness. Image clarity refers to the resolution and detail representation of the video image, while smoothness refers to the continuity and stability of video playback.

[0058] As described in step S2 above, the video data is sampled at a pre-set video frame sampling time interval, and preprocessed to obtain keyframe images. The video frame sampling time interval refers to the time interval for extracting image frames from continuous video data. This interval needs to be adjusted according to actual conditions to minimize computational and storage requirements without affecting the accuracy of conflict behavior recognition. Frame extraction refers to extracting image frames from continuous video data at pre-set time intervals; these image frames will serve as the basis for subsequent analysis and processing. During frame extraction, the system extracts a series of image frames from the video stream at set time intervals, forming the initial keyframe images. Preprocessing involves performing a series of operations on the initial keyframe images to improve their quality and analyzability. These preprocessing operations include denoising, enhancement, and edge detection. Denoising aims to remove noise points from the image, which may be caused by sensor noise, distortion during transmission, or other factors. Denoising helps improve image clarity and recognizability. Enhancement aims to improve image attributes such as contrast and brightness, making details in the image clearer and helping to better identify the characteristics of potential conflict behaviors in subsequent analysis. Edge detection aims to identify edge information in the image, which typically corresponds to the boundaries between objects or the contours of objects, helping to more accurately identify objects and scenes in the image in subsequent analysis. The initial keyframe images after preprocessing become clearer and easier to analyze. These preprocessed image frames are called keyframe images and serve as an important foundation for subsequent conflict behavior recognition and analysis.

[0059] As described in step S3 above, potential conflict behaviors are identified based on the keyframe images using a pre-trained model. The pre-trained model refers to a deep learning model pre-trained on a large amount of data, which has already learned prior knowledge about image recognition and classification. The pre-processed keyframe images are input into the pre-trained model, which extracts features from the input keyframe images, including information such as color, texture, and shape. Based on the extracted features, the pre-trained model classifies the keyframe images to determine whether they contain potential conflict behaviors. This classification process is based on the knowledge and rules learned by the model during training. The model outputs a recognition result indicating whether potential conflict behaviors exist in the keyframe images. Potential conflict behaviors refer to actions that may cause conflict or adverse consequences within a train carriage, such as physical altercations, verbal arguments, and overcrowding. These behaviors typically have specific characteristics and patterns and can be detected and identified using image recognition and classification techniques.

[0060] As described in step S4 above, the target to be monitored is identified based on the identification results, and audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, are collected. According to step S3, the identification results are obtained, which may include, but are not limited to, physical conflict, verbal arguments, overcrowding, abnormal behavior, etc. Based on these identification results, targets deemed to require further monitoring are identified. These targets may be specific individuals or groups, or specific areas or events. For the identified targets to be monitored, the system will prioritize collecting their audio data, through a microphone array or a single microphone arranged in the train carriage. In addition to the audio data directly targeting the monitored target, the system will also collect audio data around the target, such as the conversations of other passengers and the sound of the train running, so as to more comprehensively understand the environmental background and behavioral reasons of the target to be monitored. In addition to audio data, the system will also collect behavioral data of other passengers, such as tracking passenger movement trajectories and analyzing passenger posture and facial expressions.

[0061] As described in step S5 above, based on the identification of multimodal information, the multi-features of whether conflict behavior exists are analyzed and determined. Multimodal information refers to perceiving and understanding the external world through various different information representations. In the conflict behavior detection of this invention, the multimodal information includes video data of the target to be monitored, audio data within a specified range of the target to be monitored, and other passenger behavior data collected in step S4. Image features are crucial in conflict behavior detection because physical tension, angry or fearful expressions, and aggressive actions can all be precursors or manifestations of conflict behavior. Audio features help determine the emotional state of the person, such as anger, anxiety, or tension, and whether there is arguing or aggressive speech. After identifying the multimodal information, this information needs to be comprehensively analyzed to determine whether conflict behavior exists; for example, whether the person exhibits a tense physical posture, such as leaning forward or clenching their hands; whether the person exhibits negative emotional expressions such as anger, fear, or disgust; whether the intensity of the voice suddenly increases, or whether the tone becomes sharp or angry, to determine the person's emotional state. After comprehensively analyzing the multimodal information and the multi-features of conflict behavior, a judgment can be made as to whether conflict behavior exists.

[0062] As described in step S6 above, when a target to be monitored exhibits conflict behavior, an early warning mechanism is triggered, issuing an early warning signal. This early warning signal includes the location of the target in the carriage, the level of crowding in the carriage, and the number of people involved in the conflict. Through comprehensive identification and judgment of multimodal information, if it is determined that the target to be monitored exhibits conflict behavior and that this behavior meets preset triggering conditions (such as the intensity or duration of the conflict), the early warning mechanism will be triggered, sending an early warning signal. The early warning signal needs to contain sufficient information so that relevant personnel or systems can quickly understand the specific situation of the conflict and make an appropriate response. This includes: the location of the target in the carriage, clearly indicating the specific carriage where the conflict occurred, so that relevant personnel can quickly locate and proceed to the scene; the level of crowding in the carriage, providing information on the level of crowding within the carriage, which helps determine whether additional rescue personnel or equipment are needed, and what method is most appropriate to enter the carriage; and the number of people involved in the conflict, indicating the number of people involved, which helps assess the scale of the conflict and the potential level of danger, thereby determining the level of response and the resources required. Warning signals can be issued in various ways, including but not limited to: visual signals, such as displays and indicator lights inside the carriage, to visually display warning information; auditory signals, such as alarms and broadcast announcements, to issue emergency notifications inside the carriage and surrounding areas; and wireless communication, such as sending warning information to relevant personnel or systems via mobile phone text messages, emails, or dedicated wireless communication systems.

[0063] In one embodiment, step S1, which involves acquiring real-time video data via a camera sensor and monitoring the video data quality in real time, includes:

[0064] S11, continuously capture real-time video data inside the train carriage through a camera sensor. The camera sensor is a wide-angle lens with night vision function, and multiple camera sensors are configured in the train carriage.

[0065] S12, real-time monitoring of the clarity and smoothness of the captured video;

[0066] S13, when video data loss or quality degradation to a set threshold is detected, an alarm mechanism is triggered and an alarm message is sent to a designated person. The alarm message includes the time, location and type of the anomaly.

[0067] S14 triggers the alarm mechanism and simultaneously collects real-time audio data from the carriage via a voice device, while other camera sensors collect passenger behavior data from the carriage.

[0068] As described in step S11 above, the train carriages are equipped with multiple wide-angle lens camera sensors with night vision capabilities to continuously capture real-time video data within the carriages. The night vision function ensures that even in low-light environments, the camera sensors can capture clear video images, while the wide-angle lenses guarantee a large field of view, enabling effective monitoring of the entire carriage. As described in step S12 above, during the video data acquisition process, the system monitors the clarity and smoothness of the video image in real time to ensure that the quality of the acquired video data meets certain standards, thereby enabling accurate subsequent analysis and processing.

[0069] As described in step S13 above, if video data loss is detected during real-time monitoring (e.g., video signal interruption due to camera sensor failure or network problems) or video quality deteriorates below a set threshold (e.g., blurry image, pixelation, or delay due to camera obstruction or insufficient light), the system will immediately trigger an alarm mechanism and send alarm information to the designated management personnel or monitoring center. The alarm information includes the time, location, and type of anomaly (e.g., video loss, quality degradation). As described in step S14 above, when the alarm mechanism is triggered, in addition to sending alarm information, the system will immediately activate the voice device to collect audio data in the carriage in real time (e.g., conversations between passengers, arguments, loud noises, etc.); simultaneously, other camera sensors will continue to collect passenger behavior data (e.g., passenger actions, postures, expressions, and interactions between passengers) to keep track of the situation inside the carriage.

[0070] In one embodiment, step S2, which involves extracting frames from the video data according to a pre-set video frame sampling time interval and performing preprocessing operations to obtain keyframe images, includes:

[0071] S21, based on stable real-time video data, frames are extracted at set time intervals to obtain a series of initial keyframe images;

[0072] S22, perform preprocessing operations such as denoising, enhancement, and edge detection on the initial keyframe image to obtain the final keyframe image.

[0073] As described in step S21 above, in this embodiment of the invention, OpenCV (an open-source, cross-platform computer vision and machine learning software library) is used to extract a series of frames from the acquired stable real-time video data (with image clarity and smoothness within a preset range) at specified time intervals (e.g., 0.2 seconds). These frames are the initial keyframe images, which are evenly distributed over time and can roughly reflect the overall situation of the video content. As described in step S22 above, the initial keyframe images are preprocessed, including: denoising, since video data may be affected by various noises (such as environmental noise, equipment noise, etc.) during acquisition and transmission, the initial keyframe images need to be denoised to eliminate the impact of these noises on image quality; enhancement, improving the contrast and brightness of the image to make it clearer, easier to observe and analyze; and edge detection, identifying edge features in the image (such as the outline of objects, texture boundaries, etc.). These edge features have important reference value for subsequent image analysis, recognition, and understanding tasks, and finally, the keyframe images are obtained.

[0074] In one embodiment, step S3, which identifies potential conflict behaviors based on the keyframe image using a pre-trained model, includes:

[0075] S31, collect passenger behavior video data of various conflict behaviors and normal behaviors in the train environment, and passenger audio data synchronously corresponding to the passenger behavior video data;

[0076] S32, preprocess the passenger behavior video data and the passenger audio data, and mark conflicting behaviors in the video and audio respectively;

[0077] S33, Based on the passenger behavior video data and the passenger audio data, select an existing model architecture for training to obtain a potential conflict behavior model;

[0078] S34, input the keyframe image into the potential conflict behavior model to identify the potential conflict behaviors of people in the carriage.

[0079] As described in step S31 above, video data of passenger behavior in various train environments (such as crowded environments, various lighting conditions, various angles, etc.) is collected. This data includes various conflict behaviors (such as arguing, fighting, etc.) and normal behaviors (such as talking, reading, etc.). Simultaneously, audio data synchronously corresponding to this video data needs to be collected to capture sound characteristics such as volume, speech rate, and emotion. Synchronization of video and audio data is crucial because sound and image complement each other, together forming a complete picture of passenger behavior. As described in step S32 above, the video and audio data collected in step S31 are cleaned, converted in format, and feature extracted. Video data requires frame rate adjustment, size scaling, and color correction; audio data requires noise reduction and volume normalization. After preprocessing, conflict behaviors in the video and audio data need to be marked, indicating the start and end times of the conflict behavior and the type of conflict behavior. As described in step S33 above, a suitable machine learning or deep learning model architecture is selected based on the nature of the conflict behavior to be identified and the characteristics of the collected data. The selected model architecture is then trained using preprocessed video and audio data, along with corresponding conflict behavior labels. During training, the model learns how to identify conflict behavior based on the input video and audio data. As described in step S34 above, the keyframe images extracted in step S2 are input into the trained potential conflict behavior model. The model uses the knowledge and features learned during training to identify potential conflict behaviors among people in the carriage based on the input keyframe images.

[0080] During the collection, processing, storage, and transmission of real-time video data, audio data, and other passenger behavior data within train carriages, strict data privacy and security protection measures are implemented. These measures include: encrypting all collected data, including video, audio, and other passenger behavior data, to ensure data security during transmission and storage; and enhancing data protection through the use of encryption algorithms. During data processing, sensitive information such as passenger facial features and identity information is anonymized to prevent the leakage of passenger privacy; specialized data encryption equipment and software are used for data encryption and anonymization. A strict access control mechanism is established to ensure that only authorized personnel can access and process this data; multi-factor authentication (such as passwords and biometrics) is used to enhance access control security. Data is backed up regularly to prevent data loss or corruption, and a data recovery plan is developed to ensure rapid recovery in the event of data loss or corruption. Regular security assessments and vulnerability scans are conducted on the system to promptly identify and remediate potential security risks.

[0081] In one embodiment, step S33, which involves selecting an existing model architecture for training based on the passenger behavior video data and the passenger audio data to obtain a potential conflict behavior model, includes:

[0082] S321, label the conflicting behaviors and normal behaviors in the passenger behavior video data and passenger audio data, wherein the conflicting behaviors include at least the start and end times of the conflict, the location of the conflict participants, and the behavior of the surrounding passengers when the conflict occurs;

[0083] S322, in the video processing unit, dynamic features of labeled passenger behavior video data are extracted through a convolutional neural network, and an existing video behavior recognition model is selected for training.

[0084] S323, on the audio unit, extract the spectral features of each frame of audio signal in the labeled passenger audio data, use a recurrent neural network to build a model, learn the temporal features in the audio signal, and input the spectral features into the recurrent neural network model for training and optimization;

[0085] S324, through the fusion module, fuses the output features of each model in the video processing module and the audio processing module;

[0086] S325, based on labeled passenger behavior video data and passenger audio data, trains and optimizes the fused model to obtain a potential conflict behavior model that simultaneously identifies whether conflict behaviors exist in video data and audio data.

[0087] As described in step S321 above, conflicting and normal behaviors in passenger behavior video data and passenger audio data are labeled. The labeled information includes at least the start and end times of the conflict, the locations of the conflict participants, and the behavior of surrounding passengers when the conflict occurs. The labeled information serves as the basis for subsequent model training, helping the model learn the characteristics of conflicting behaviors. As described in step S322 above, a convolutional neural network (CNN) is used in the video processing unit to extract the dynamic features of the labeled passenger behavior video data. CNN performs well in processing image and video data and can capture spatial and temporal features in the video. In this embodiment of the invention, the existing YOLOv8 model (an object detection model that can identify and locate objects in images) is selected. The YOLOv8 model is trained based on the captured spatial and temporal features, enabling the model to recognize behaviors in the video. As described in step S323 above, in the audio unit, the spectral features of each frame of audio signal in the labeled passenger audio data are extracted. The extracted spectral features reflect the characteristics of the audio signal in the frequency domain, which helps to identify information in speech. A recurrent neural network (RNN) is used to build a model to learn the temporal features in the audio signal. RNNs have advantages in processing sequential data (such as audio signals) and can capture the time dependence in the signal. The spectral features are input into the RNN model for training and optimization, enabling the model to recognize volume, speech rate, emotion, etc. in the audio. As described in step S324 above, the output features of each model in the video processing module and the audio processing module are fused through the fusion module. By combining video and audio information, the accuracy of the model in recognizing conflict behavior is improved. As described in step S325 above, based on the labeled passenger behavior video data and passenger audio data, the fused model is trained and optimized to obtain a potential conflict behavior model that can simultaneously identify whether conflict behavior exists in video data and audio data. Through training and optimization, the model can learn the characteristics of conflict behavior for judgment on actual data. Alternatively, the potential conflict behavior model can also employ other conventional model training methods in the existing technology.

[0088] In one embodiment, step S4, which involves locking the target to be monitored based on the identification result and collecting audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, includes:

[0089] S41, Based on the model's identification results, identify targets that have potential conflict behaviors and require monitoring;

[0090] S42, according to a preset duration, continuously monitor the video data of the target to be monitored, collect audio data and other passenger behavior data within a specified range of the target to be monitored.

[0091] As described in step S41 above, based on the output of the potential conflict behavior identification model in step S3 and the analysis of video data, such as people's actions, expressions, and positional relationships, the system determines which targets (such as passengers, staff, etc.) may be exhibiting conflict behavior or have a tendency to do so. Once a target with potential conflict behavior is identified, the system locks it as a target to be monitored. As described in step S42 above, after locking the target to be monitored, the system enters a continuous monitoring phase to gain a deeper understanding of the target's behavioral patterns and their interactions with the surrounding environment. The system continues to collect video data of the target to observe whether their actions, expressions, etc., have changed, and whether new conflict behaviors have emerged. In addition to video data, the system also collects audio data within a specified range of the target to understand the target's speech content, tone, etc., thereby more comprehensively assessing their emotional state and potential conflict behaviors. Furthermore, the system also pays attention to the behavioral data of other passengers around the target, including their actions, expressions, and interactions with the target, to help the system more accurately determine the nature and possible consequences of the conflict behavior. In this process, the preset duration of continuous monitoring is set according to the specific scenario and needs to ensure that the system can collect enough information to make accurate judgments.

[0092] In one embodiment, step S5, which involves identifying, analyzing, and determining whether conflicting behaviors exist based on multimodal information, includes:

[0093] S51, identify the collected multimodal information, which includes video data of the target to be monitored, audio data within a specified range of the target to be monitored, and other passenger behavior data;

[0094] S52, the data in the multimodal information are identified respectively through the potential conflict behavior model;

[0095] S53, based on the recognition results, determine whether there are potential conflict behaviors among the data in the multimodal information;

[0096] S54, when there is potential conflicting behavior in the audio data or other passenger behavior data within the specified range of the target to be monitored, it is determined that there is indeed conflicting behavior in the target to be monitored.

[0097] As described in step S51 above, the multimodal information collected in step S4 is identified. This multimodal information includes video data of the target to be monitored (actions, expressions, location, etc.), audio data within a specified range of the target to be monitored (voice content, tone, speech rate, etc.), and behavioral data of other passengers (actions, expressions, interactions with the target to be monitored, etc.). As described in step S52 above, after identifying the multimodal information, a pre-constructed potential conflict behavior model is used to identify key information related to conflict behavior for each modality of data. As described in step S53 above, the model's identification results determine whether potential conflict behavior exists in each piece of data within the multimodal information. Audio data within a specified range of the target to be monitored or other passenger behavioral data serve as auxiliary indicators for judging conflict behavior. If one or both of these indicate potential conflict behavior, then the target to be monitored is considered to indeed exhibit conflict behavior. For example, the model identifies obvious signs of conflict behavior in the audio data, such as arguing or insults; while other passenger behavioral data shows passengers suddenly gathering in one place, etc.

[0098] Reference Figure 2 Here is a structural block diagram of a train carriage passenger conflict detection device according to an embodiment of the present invention, comprising:

[0099] The data acquisition module is used to acquire real-time video data through a camera sensor and monitor the video data quality in real time.

[0100] The preprocessing module is used to extract frames from the video data according to a preset video frame sampling time interval and perform preprocessing operations to obtain keyframe images.

[0101] The model recognition module is used to identify potential conflict behaviors based on the keyframe images using a pre-trained model.

[0102] The monitoring module is used to locate the target to be monitored based on the identification results, and to collect the audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data;

[0103] The analysis module is used for identification based on multimodal information, analyzing and determining multiple features that indicate the existence of conflicting behaviors;

[0104] The early warning module is used to trigger an early warning mechanism and issue an early warning signal when there is conflict behavior in the target to be monitored.

[0105] For the specific implementation of each module in the above device example, please refer to the above method embodiments, which will not be repeated here.

[0106] Reference Figure 3 This invention also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3 As shown, the computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0107] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the computer devices on which the present invention is applied.

[0108] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0109] In summary, real-time video data is collected by camera sensors, and the quality of the video data is monitored in real time. The video data is then sampled at pre-set time intervals to obtain keyframe images. Based on these keyframe images, a pre-trained model is used to identify potential conflict behaviors. The target to be monitored is located based on the identification results, and audio data of the target, as well as surrounding audio data and other passenger behavior data, are collected. Based on multimodal information recognition, multiple features are analyzed and used to determine whether conflict behaviors exist. When conflict behaviors are detected at the target, an early warning mechanism is triggered, issuing a warning signal to ensure passenger safety, maintain order within the train carriages, and ensure a harmonious train operation environment.

[0110] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0111] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0112] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting conflicts among passengers in a train carriage, characterized in that, Includes the following steps: Real-time video data is collected through camera sensors, and the quality of the video data is monitored in real time. The video data is sampled at a pre-set video frame sampling time interval and preprocessed to obtain keyframe images. Based on the keyframe images, potential conflict behaviors are identified through a pre-trained model, including: collecting passenger behavior video data of various conflict behaviors and normal behaviors in a train environment, and passenger audio data synchronously corresponding to the passenger behavior video data; preprocessing the passenger behavior video data and the passenger audio data, and labeling the conflict behaviors in the video and audio respectively; training an existing model architecture based on the passenger behavior video data and the passenger audio data to obtain a potential conflict behavior model; and inputting the keyframe images into the potential conflict behavior model to identify potential conflict behaviors of people in the carriage. Based on the identification results, the target to be monitored is identified, and audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, are collected, including: based on the identification results of the model, identifying the target to be monitored that has potential conflict behavior; and continuously monitoring the video data of the target to be monitored, and collecting audio data and other passenger behavior data within a specified range of the target to be monitored for a preset duration. Based on the identification of multimodal information, the method analyzes and determines whether conflicting behavior exists using multiple features, including: identifying the collected multimodal information, which includes video data of the target to be monitored, audio data within a specified range of the target to be monitored, and other passenger behavior data; identifying the data in the multimodal information using a potential conflicting behavior model; determining whether potential conflicting behavior exists in each data in the multimodal information based on the identification results; and determining whether potential conflicting behavior exists in the audio data or other passenger behavior data within the specified range of the target to be monitored. When a conflict occurs at a target to be monitored, an early warning mechanism is triggered and an early warning signal is issued. The early warning signal includes the location of the carriage, the level of crowding in the carriage, and the number of people involved in the conflict.

2. The method for detecting conflicts among passengers in a train carriage according to claim 1, characterized in that, The step of acquiring real-time video data through a camera sensor and monitoring the video data quality in real time includes: The train continuously captures real-time video data inside the carriages using camera sensors, which are wide-angle lenses with night vision capabilities. Multiple camera sensors are installed in the train carriages. Real-time monitoring of the clarity and smoothness of captured video; When video data loss or quality degradation to a set threshold is detected, an alarm mechanism is triggered, and an alarm message is sent to designated personnel. The alarm message includes the time, location, and type of the anomaly. While triggering the alarm mechanism, the system collects audio data from the carriage in real time via a voice device, and other camera sensors collect passenger behavior data from the carriage.

3. The method for detecting conflicts among passengers in a train carriage according to claim 1, characterized in that, The step of extracting frames from the video data according to a pre-set video frame sampling time interval and performing preprocessing operations to obtain keyframe images includes: Based on stable real-time video data, frames are extracted at set time intervals to obtain a series of initial keyframe images. The initial keyframe image is preprocessed by denoising, enhancement, and edge detection to obtain the final keyframe image.

4. The method for detecting conflicts among passengers in a train carriage according to claim 1, characterized in that, The step of training an existing model architecture based on the passenger behavior video data and the passenger audio data to obtain a potential conflict behavior model includes: The conflicting and normal behaviors in the passenger behavior video data and passenger audio data are labeled. The conflicting behaviors include at least the start and end times of the conflict, the location of the conflict participants, and the behavior of the surrounding passengers when the conflict occurs. In the video processing unit, dynamic features of labeled passenger behavior video data are extracted using a convolutional neural network, and an existing video behavior recognition model is selected for training. In the audio unit, the spectral features of each frame of audio signal in the labeled passenger audio data are extracted, a recurrent neural network is used to build a model, the temporal features in the audio signal are learned, and the spectral features are input into the recurrent neural network model for training and optimization. The fusion module integrates the output features of each model in the video processing module and the audio processing module. Based on labeled passenger behavior video data and passenger audio data, the fused model is trained and optimized to obtain a potential conflict behavior model that can simultaneously identify whether conflict behaviors exist in video data and audio data.

5. A conflict detection device for passengers inside a train carriage, characterized in that, include: The data acquisition module is used to acquire real-time video data through a camera sensor and monitor the video data quality in real time. The preprocessing module is used to extract frames from the video data according to a preset video frame sampling time interval and perform preprocessing operations to obtain keyframe images. The model recognition module is used to identify potential conflict behaviors based on the keyframe images using a pre-trained model. This includes: collecting passenger behavior video data of various conflict behaviors and normal behaviors in a train environment, and passenger audio data synchronized with the passenger behavior video data; preprocessing the passenger behavior video data and the passenger audio data, and labeling the conflict behaviors in the video and audio respectively; training an existing model architecture based on the passenger behavior video data and the passenger audio data to obtain a potential conflict behavior model; and inputting the keyframe images into the potential conflict behavior model to identify potential conflict behaviors of people in the carriage. The monitoring module is used to locate the target to be monitored based on the identification results, and collect the audio data of the target to be monitored, as well as surrounding audio data and other passenger behavior data, including: locating the target to be monitored with potential conflict behavior based on the model's identification results; and continuously monitoring the video data of the target to be monitored and collecting audio data and other passenger behavior data within a specified range of the target to be monitored for a preset duration. The analysis module is used to identify and determine the existence of conflicting behaviors based on multimodal information, including: identifying the collected multimodal information, which includes video data of the target to be monitored, audio data within a specified range of the target to be monitored, and other passenger behavior data; identifying the data in the multimodal information using a potential conflicting behavior model; determining whether potential conflicting behaviors exist in each data in the multimodal information based on the identification results; and determining that the target to be monitored does indeed have conflicting behaviors when potential conflicting behaviors are found in the audio data within the specified range of the target to be monitored or in the other passenger behavior data. The early warning module is used to trigger an early warning mechanism and issue an early warning signal when there is conflict behavior in the target to be monitored.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the train carriage conflict detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the train carriage conflict detection method as described in any one of claims 1 to 4.

Images