An intelligent ship safety monitoring method and system based on deep learning

Abstract

This invention discloses a deep learning-based intelligent ship safety monitoring method and system, belonging to the field of ship safety monitoring technology. It includes: acquiring data collected by monitoring equipment on the ship; inputting the collected image data into a target detection model; executing subsequent procedures based on the detection results; inputting the collected video data into a behavior recognition model to identify crew behavior; generating an alarm directly if the event is an emergency; tracking the involved crew member if the event is a general event, collecting crew member facial data using a face detection model; establishing a face recognition network, comparing the collected crew member facial images with images in a crew member face database to confirm the crew member's identity, and storing the data collected in the above steps as evidence, triggering an alarm. This invention can monitor various types of events, using detection algorithms to complete the entire process from event discovery to evidence collection. This makes ship safety monitoring more comprehensive and detection efficiency higher.

Description

Technical Field

[0001] This invention relates to the field of ship safety monitoring technology, and in particular to an intelligent ship safety monitoring method and system based on deep learning. Background Technology

[0002] For a long time, safety has been the top priority for ships navigating at sea. Ensuring ship safety is the responsibility and obligation of every crew member, and ensuring the safety of crew members' behavior and personal safety is also a crucial aspect of ensuring ship safety. Current safety management primarily relies on human supervision, using data from monitoring equipment to identify and warn of potential safety hazards that have not yet caused accidents, and to investigate and hold accountable those responsible for accidents that have occurred. With the development of technology, the drawbacks of this traditional method have become increasingly apparent. It is not only inefficient but also consumes significant human and material resources, making it difficult to meet the needs of modern intelligent ship safety management. How to ensure the safety of crew members' behavior and achieve unified intelligent ship safety management are issues that the shipping industry highly values ​​and urgently needs to address.

[0003] Currently, with the continuous development and popularization of artificial intelligence (AI) technology, more and more shipping companies and research institutions are committed to integrating AI technology into ship safety monitoring systems. Chinese Patent Publication No. CN113657201A discloses a method, device, equipment, and storage medium for monitoring and analyzing crew behavior, identifying crew behavior, and issuing timely alarms. Chinese Patent Publication No. CN113486843A discloses a multi-scenario unsafe crew behavior detection and distribution system based on improved YOLOv3, capable of detecting six types of unsafe crew behavior in different scenarios. Chinese Patent Publication No. CN114419607A discloses a method and system for detecting non-standard behavior in the ship's bridge, which is beneficial for real-time early warning and prevention of non-standard behavior in the ship's bridge. Regarding the ship safety monitoring systems and methods disclosed in the above patents, while they are helpful for detecting ship safety, the following three problems still exist:

[0004] The monitoring scope is not comprehensive enough. The technology disclosed in Chinese Patent Publication No. CN113486843A only detects six unsafe behaviors of crew members; the technology disclosed in Chinese Patent Publication No. CN114419607A is used to detect non-standard behaviors in the ship's bridge. Existing ship safety inspection technologies are limited to detecting only crew behavior, while the actual scope of ship safety monitoring should not only include crew violations or unsafe behaviors, but also consider crew personal safety, shipboard environmental safety, etc.

[0005] The monitoring process is incomplete. One of the goals of an intelligent ship safety monitoring system is to automate the monitoring process, from incident detection to evidence collection and incident handling. Except for incident handling, which still relies on human intervention, the preceding processes can be automated using artificial intelligence technology. Current technology only automates the incident detection process, neglecting evidence collection. Currently, combining target tracking technology and pedestrian retrieval technology can automate evidence collection.

[0006] Monitoring efficiency needs improvement. The technology disclosed in Chinese Patent Publication No. CN113657201A uses an image classification model of crew behavior to identify crew members' actions; the technology disclosed in Chinese Patent Publication No. CN113486843A improves the target detection model YOLOv3 network to detect crew behavior. Currently, video-based behavior recognition technology is becoming increasingly mature, while existing monitoring methods still rely on image classification and target detection to identify human behavior, which not only fails to improve monitoring efficiency but also reduces the automation level of the entire process.

[0007] To promote the development of intelligent ships and achieve the efficient integration of ship safety management and artificial intelligence technology, building a complete, comprehensive, and efficient intelligent ship safety monitoring system is a problem that the shipping industry attaches great importance to and urgently needs to solve. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a deep learning-based intelligent ship safety monitoring method and system, focusing on solving three problems in existing intelligent ship safety monitoring systems: incomplete monitoring scope, incomplete monitoring process, and low monitoring efficiency.

[0009] To achieve the above objectives, this invention proposes a deep learning-based intelligent ship safety monitoring method, comprising:

[0010] (1) Construct a deep learning model and the required training dataset. The deep learning model includes an object detection model, an action recognition model and a pedestrian retrieval model. Use the training dataset to complete the model training.

[0011] (2) Acquire data collected by the monitoring equipment on board the ship, including image data and video data;

[0012] (3) Input the collected image data into the target detection model to identify the target in the image data. If the detection result shows a direct state target, then execute step 5; if the detection result shows a behavioral state target, then execute step 4.

[0013] (4) Input the collected video data into the behavior recognition model to identify the behavior of the crew in the video data;

[0014] (5) If the event is an emergency, an alarm will be generated directly; if the event is a normal event, step 6 will be executed.

[0015] (6) Track the crew members involved in the incident. During the tracking process, use a face detection model to collect the crew members' face data. Once a clear face image of the crew members involved in the incident is obtained, stop tracking.

[0016] (7) Establish a facial recognition network, compare the collected facial images of the crew members involved with the images in the crew member facial database, confirm the crew member's identity, and save the data collected in the above steps as evidence, and issue an alarm.

[0017] Furthermore, the target detection model uses the YOLOv5 model; the behavior recognition model uses the SlowFast model; and the pedestrian re-identification model uses the CAL model.

[0018] Furthermore, the direct state target includes at least one of a safety helmet, a fire source, and a person who has fallen to the ground. The existence of such a target at a certain moment can be determined as an event occurring.

[0019] Furthermore, the behavioral state target includes at least one of cigarettes, mobile phones, or people in a fighting state. Such targets exist at a certain moment and need to be further detected by the behavior recognition model.

[0020] Furthermore, the emergency includes at least one of the following: discovery of a fire source and absence from a critical post.

[0021] Furthermore, the general incidents include at least one of the following: not wearing a safety helmet during engineering operations and smoking in important positions.

[0022] Furthermore, the tracking of the crew members involved in step 6 specifically involves:

[0023] (6.1) Single camera tracking: Crew target tracking in the same camera scene can be performed using a single target tracking algorithm. The image sequence of the crew is collected. If a clear image of the crew face is not collected before the crew target leaves the camera's field of view, cross-camera tracking is performed.

[0024] (6.2) Cross-camera tracking: First, determine the camera locations in the vicinity, detect the scene of crew members appearing through the target detection model, and track them using the single target tracking algorithm. Collect a continuous sequence of crew member images, and match the collected image sequence with the sequence from the previous camera through the pedestrian re-identification model. The sequence with the highest score is determined as the sequence of the crew member involved.

[0025] (6.3) Use the face detection model to collect crew member face data. After obtaining a clear face image of the crew member involved in the incident, stop tracking.

[0026] Furthermore, the single-target tracking algorithm uses the KCF single-target tracking algorithm, and the face detection model uses the LFFD model.

[0027] Furthermore, the face recognition network used in step 7 is FaceNet.

[0028] This invention also provides a deep learning-based intelligent ship safety monitoring system, comprising:

[0029] Monitoring data sampling module: used to acquire data collected by monitoring equipment on board the ship, including image data and video data;

[0030] Shipboard target detection module: used to carry a target detection model, input the collected image data into the target detection model, and identify targets in the image data;

[0031] Crew behavior recognition module: used to carry a behavior recognition model, input the collected video data into the behavior recognition model, and recognize the behavior of the crew in the video data;

[0032] Crew target tracking module: used to track the crew members involved in the incident. During the tracking process, a face detection model is used to collect the crew members' facial data. Once a clear image of the crew member involved in the incident is obtained, the tracking stops.

[0033] Crew identity matching module: This module is equipped with a facial recognition network to compare the collected facial images of the crew members involved with images in the crew facial database to confirm the crew members' identities. The data collected in the above steps is stored as evidence, and an alarm is triggered.

[0034] The beneficial effects of this invention are:

[0035] 1. Comprehensive monitoring range: This invention can be used to monitor a variety of events occurring on board. In addition, even if the crew member involved leaves the original camera's field of view by entering the cabin from the deck, this method can still achieve comprehensive tracking of the crew member involved through cross-camera tracking technology.

[0036] 2. Completely Automated Process: This invention first detects unsafe events using target detection and behavior recognition algorithms; then, it tracks the crew members involved using target tracking algorithms, and can combine target detection and pedestrian re-identification algorithms to achieve cross-camera tracking when necessary; during the tracking process, it captures the facial images of the crew members involved using face detection algorithms; finally, it confirms the crew members' identities through a facial feature extraction network. From event discovery to evidence collection, the entire process is automated by computers.

[0037] 3. Highly Efficient Monitoring Capabilities: This invention departs from the previous methods of using image classification or object detection to identify crew behavior, instead employing a video-based behavior recognition model to directly identify crew behavior. It utilizes object detection and behavior recognition algorithms to achieve the primary function of safety monitoring, and combines these with target tracking and face matching algorithms to retrieve target crew members. The models used in the system are representative models that have demonstrated outstanding performance in their respective fields.

[0038] Based on the aforementioned advantages and characteristics, this invention can be widely applied in the field of ship safety monitoring. This invention will promote the development of the intelligent ship sector, especially in ship safety management, enabling the timely detection of safety hazards, effectively reducing the total number of ship safety accidents, ensuring the safety of ships and crew, and promoting the stable development of safe production in the shipping industry. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the intelligent ship safety monitoring method based on deep learning, according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the architecture of an intelligent ship safety monitoring system based on deep learning, according to an embodiment of the present invention.

[0041] Figure 3 This is a flowchart illustrating the monitoring process of the intelligent ship safety monitoring method based on deep learning, according to an embodiment of the present invention.

[0042] Figure 4 This is a schematic diagram of the YOLOv5 model architecture used in the embodiments of the present invention.

[0043] Figure 5 This is a schematic diagram of the SlowFast model architecture used in the embodiments of the present invention.

[0044] Figure 6 This is a schematic diagram of the LFFD model architecture used in the embodiments of the present invention.

[0045] Figure 7 This is a schematic diagram of the CAL model training structure used in an embodiment of the present invention.

[0046] Figure 8 This is a schematic diagram of the FaceNet network framework used in an embodiment of the present invention. Detailed Implementation

[0047] To clearly illustrate the technical solution and features of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments.

[0048] like Figure 1 and Figure 2As shown in the figure, this embodiment provides a deep learning-based intelligent ship safety monitoring method and system, which includes the following steps:

[0049] S101. Construct a deep learning model and the required training dataset. The deep learning model includes an object detection model, an action recognition model, and a pedestrian retrieval model. Use the training dataset to complete the model training.

[0050] Dataset construction: It is necessary to collect image data and video data from shipboard monitoring cameras and construct datasets corresponding to different models.

[0051] Model training: Using the datasets obtained from the dataset construction module, train the object detection model M1 based on camera images, the behavior recognition model M2 based on video, and the pedestrian re-identification model M3 based on video.

[0052] The datasets used for models M1 and M2 were constructed by processing data recorded by shipboard surveillance cameras. The cameras had a resolution of 1920*1080, and the average duration of the video clips was 5 seconds, assuming that 5 seconds was enough to confirm the occurrence of an action. M3 was trained using a publicly available pedestrian re-identification dataset.

[0053] The object detection model used is the YOLOv5 model, such as Figure 4 As shown. The YOLOv5 model is one of the representative models of the YOLO (You Only Look Once) series. From YOLOv1 to YOLOv5, this model has made significant improvements in many aspects. Some of the more noteworthy improvements in YOLOv5 are:

[0054] First, the Backbone model uses a Focus structure to slice the image, which is similar to downsampling. The purpose of this structure is to reduce the number of model parameters and computational cost while ensuring that features are not lost.

[0055] Secondly, there is the CSP (Cross Stage Partial) structure. In YOLOv4, this structure was only applied to the Backbone, while YOLOv5 extended the structure to two forms, applying them to the Backbone and Neck respectively. This greatly reduced the computational and memory costs of the model while ensuring its accuracy.

[0056] Third, an adaptive image scaling method is used at the model input. The original scaling method leads to a lot of information redundancy, which affects the model's inference speed. The adaptive image scaling method adds a minimum amount of black borders to the scaled image to avoid information redundancy and thus improve the model's inference speed.

[0057] The behavior recognition model uses the SlowFast model. For example... Figure 5 As shown, the SlowFast model is a dual-branch structure model for video recognition. The Slow branch is responsible for capturing spatial semantic information and operates at a lower frame rate and slower refresh rate, while the Fast branch is responsible for capturing rapidly changing motion and operates at a faster refresh rate and higher temporal resolution. The two branches are fused together by lateral connection.

[0058] The slow branch can be any type of convolutional model. Its key concept lies in the large time span 's' of the input frames, meaning that only one frame is processed every 's' frames. The value of 's' is usually 16. Assuming that the slow branch samples 't' frames, the original clip length is 's' * 't' frames.

[0059] The Fast branch runs parallel to the Slow branch and has three characteristics. First, it has a high frame rate, meaning it samples faster, with a step size of s / n, where n is the frame rate ratio between the two branches. The sampling density of the Fast branch is n times that of the Slow branch, and n is typically 8. Second, it has high temporal resolution, as it does not use a temporal downsampling layer. Third, it has low channel capacity. The Fast branch becomes very lightweight by reducing channel capacity, having m times the number of channels in the Slow branch, where m is typically 1 / 8. This also means that the Fast branch is more computationally efficient.

[0060] The Slow and Fast branches are merged through horizontal connections, a common technique for fusing different levels of spatial resolution and semantics.

[0061] The pedestrian re-identification model uses the CAL model, a type of model designed for re-identifying pedestrians in different clothing. It exhibits good accuracy even when identifying different pedestrians wearing the same clothing. However, since crew members typically wear uniform clothing, traditional pedestrian re-identification models are significantly affected by factors such as clothing appearance, making them unsuitable for crew re-identification. The main feature of the CAL model is its Clothes-based Adversarial Loss (CAL) function. By penalizing the model's predictive ability regarding clothing, it extracts clothing-independent features from the original RGB image. For example... Figure 7 As shown, in addition to the identity classifier, the CAL model also includes an additional clothing classifier. During the learning process, the clothing-based adversarial loss forces the backbone network to mine features unrelated to clothing.

[0062] S102. Acquire data collected by the ship's monitoring equipment, including image data and video data;

[0063] In practice, the acquired data is video data, and image data is obtained by cropping video data. Different data types are input into corresponding models for detection. The sampled image data resolution is 1920*1080, and the average video clip duration is 20 seconds.

[0064] S103. Input the collected image data into the target detection model to identify the target in the image data. If the detection result shows a direct state target, then execute step S105; if the detection result shows a behavioral state target, then execute step S104.

[0065] In practice, the target detection model is an image-based target detection model. It identifies the location and type of targets in the image by recognizing the image data, and executes different processes depending on the detected target.

[0066] The target detection model identifies targets in two categories:

[0067] Direct state target T1. The existence of this type of target at a certain moment can be determined as an event occurring. The direct state targets include at least a safety helmet, a fire source, and a person who has fallen to the ground.

[0068] Behavioral state target T2. The existence state of this type of target at a certain moment is insufficient to confirm that an event has occurred, and further detection is required through a behavioral recognition model. The behavioral state targets include at least cigarettes, mobile phones, and people in a fighting state.

[0069] The two target classifications are based on whether the target's state at a given moment is sufficient to determine if an event has occurred. In the events monitored by the system, most crew violations require behavioral analysis of video footage for identification. For example, if a crew member is sleeping on duty, simply using a target detection model to identify a moment in the image might misinterpret a blinking or closed-eye state as sleeping. Therefore, a behavioral recognition model is needed for further video analysis. Conversely, for events related to shipboard environmental safety, such as the discovery of a fire source, the presence of a fire source in the image, when detected by the target detection model, can directly indicate a dangerous event.

[0070] S104. Input the collected video data into the behavior recognition model to identify the behavior of the crew members in the video data;

[0071] In practice, the behavior recognition model is a video analytics-based model primarily used to identify irregular behaviors of crew members, including at least smoking, using mobile phones, and fighting. The behavior recognition model receives sampled video data and combines it with target information generated by a target detection model to further detect and identify behavioral events in the video.

[0072] S105. If the event is an emergency, an alarm will be generated directly; if the event is a normal event, step S106 will be executed.

[0073] In practice, the events monitored by the system are divided into two categories:

[0074] Emergency Event E1. This type of event may lead to a serious accident, requiring timely notification and resolution of management. Evidence investigation can be conducted after the event has been resolved. The emergency events mentioned include at least the discovery of a source of fire and absence of personnel in key positions.

[0075] General Incident E2. This type of incident poses certain safety hazards and requires timely notification to management personnel and recording of the incident's occurrence. These general incidents are mostly due to crew violations, including at least not wearing safety helmets during engineering operations and smoking in important positions.

[0076] Different types of incidents require different handling procedures. For example, if a fire source is discovered, an alarm should be raised immediately and relevant personnel should be dispatched to handle the situation promptly. In cases where safety helmets are not worn during engineering operations, evidence can be investigated first. While preserving video data, facial recognition technology can be used to match the information of the crew members involved with information in the identity database to identify the crew members involved.

[0077] S106. Track the crew member involved in the incident. During the tracking process, use a face detection model to collect the crew member's face data. Once a clear face image of the crew member involved in the incident is obtained, stop tracking.

[0078] In practice, tracking the crew members involved includes:

[0079] Single-camera tracking. Crew target tracking in the same camera scene can be performed using a single-target tracking algorithm. This involves acquiring a sequence of crew images. If a clear image of the crew's face is not acquired before the crew target leaves the camera's field of view, then cross-camera tracking is performed.

[0080] Cross-camera tracking. First, determine the camera locations in the vicinity. Then, use a target detection model to detect scenes where crew members appear and use a single-target tracking algorithm to track them, collecting a continuous sequence of crew member images. The collected image sequence is then matched with the sequence from the previous camera using a pedestrian re-identification model. The sequence with the highest score is identified as the sequence of the crew member involved, and tracking continues until a clear image of the crew member's face is collected.

[0081] The single-target tracking algorithm uses the KCF single-target tracking algorithm. Its basic idea is to find a filter template, convolve the next frame image with the selected filter template, and the region with the largest response is the predicted target. The tracking algorithm based on kernelized correlation filters (KCF) is the method adopted in this module. It utilizes properties such as diagonalization of circulant matrices to make target tracking fast and accurate.

[0082] The face detection model uses the LFFD model, a novel single-object detection model suitable for faces, pedestrians, vehicles, and other targets. The model uses a receptive field instead of anchors, making it an anchor-free method. Its main advantages are: first, by adding more convolutional layers, it can cover targets of a larger scale with limited increase in latency; second, it excels in detecting small targets; and third, its network structure is common and can be deployed on mainstream edge devices, making it highly adaptable to this system. The network structure of the LFFD model is as follows: Figure 6 As shown, the model mainly consists of four parts: tiny part, small part, medium part, and large part. Based on the basic model structure, eight feature maps are extracted to detect faces from small to large. The detection module is divided into binary classification and boundary regression.

[0083] S107. Establish a facial recognition network, compare the collected facial images of the crew members involved with the images in the crew member facial database to confirm the crew member's identity, and save the data collected in the above steps as evidence, and issue an alarm.

[0084] In practical implementation, a facial recognition network is established. By matching the facial images of the crew members involved with facial image features in the crew member facial database, the crew member's identity is confirmed, previously obtained data is saved as evidence, and an alarm is triggered. The facial recognition network used is FaceNet. FaceNet is a general-purpose recognition network that can be used for facial verification, facial recognition, and facial clustering. It learns to map images to Euclidean space through deep convolutional neural networks. Spatial distance is directly related to facial image similarity: different images of the same person have small spatial distances, while images of different people have larger spatial distances. After determining the feature mapping relationship of the obtained images, facial recognition becomes a K-NN classification problem. The overall framework of FaceNet is as follows: Figure 8 As shown, the main model uses a deep network, Inception ResNet-v2.

[0085] According to another embodiment, a deep learning-based intelligent ship safety monitoring system is also provided, comprising:

[0086] Monitoring data sampling module: used to acquire data collected by monitoring equipment on board the ship, including image data and video data;

[0087] Shipboard target detection module: used to carry a target detection model, input the collected image data into the target detection model, and identify targets in the image data;

[0088] Crew behavior recognition module: used to carry a behavior recognition model, input the collected video data into the behavior recognition model, and recognize the behavior of the crew in the video data;

[0089] Crew target tracking module: used to track the crew members involved in the incident. During the tracking process, a face detection model is used to collect the crew members' facial data. Once a clear image of the crew member involved in the incident is obtained, the tracking stops.

[0090] Crew identity matching module: This module is equipped with a facial recognition network to compare the collected facial images of the crew members involved with images in the crew facial database to confirm the crew members' identities. The data collected in the above steps is stored as evidence, and an alarm is triggered.

[0091] Finally, it should be noted that the above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any substitution, modification or improvement made to the technical solution and inventive concept of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based intelligent ship safety monitoring method, characterized in that, include: (1) Construct a deep learning model and the required training dataset. The deep learning model includes an object detection model, an action recognition model and a person re-identification model. Use the training dataset to complete the model training. The target detection model uses the YOLOv5 model; the behavior recognition model uses the SlowFast model; and the person re-identification model uses the CAL model. (2) Acquire data collected by the monitoring equipment on board the ship, including image data and video data; (3) Input the collected image data into the target detection model to identify the target in the image data. If the detection result shows a direct state target, then execute step 5; if the detection result shows a behavioral state target, then execute step 4. The direct state targets include at least one of a safety helmet, a fire source, and a person who has fallen to the ground. The existence of such a target at a certain moment can be determined as an event occurring. The behavioral state target includes at least one of the following: a cigarette, a mobile phone, or a person in a fighting state. Such targets exist at a certain moment and need to be further detected by the behavioral recognition model. (4) Input the collected video data into the behavior recognition model to identify the behavior of the crew in the video data and execute step (5). (5) If the incident is an emergency, an alarm will be generated directly; If the event is a general event, proceed to step 6; (6) Track the crew members involved in the incident. During the tracking process, use a face detection model to collect the crew members' face data. Once a clear face image of the crew members involved in the incident is obtained, stop tracking. (6.1) Single camera tracking: The single target tracking algorithm is used to track the crew members in the same camera scene. The image sequence of the crew members is collected. If a clear image of the crew member's face is not collected before the crew member leaves the field of view of the camera, cross-camera tracking is performed. The single-target tracking algorithm uses the KCF single-target tracking algorithm; (6.2) Cross-camera tracking: First, determine the camera locations in the vicinity, detect the scene of the crew members by the target detection model, and use the single target tracking algorithm to track them, collect a continuous sequence of crew member images, and match the collected image sequence with the sequence of the previous camera by the pedestrian re-identification model. The sequence with the highest score is determined as the sequence of the crew member involved. (6.3) Use the face detection model to collect crew member face data, and stop tracking after obtaining a clear face image of the crew member involved in the incident; The face detection model used is the LFFD model; (7) Establish a facial recognition network, compare the collected facial images of the crew members involved with the images in the crew member facial database, confirm the crew member's identity, and save the data collected in the above steps as evidence, and issue an alarm. The face recognition network used is FaceNet.

2. The intelligent ship safety monitoring method based on deep learning according to claim 1, characterized in that: The emergency includes at least one of the following: discovery of a fire source and absence of personnel in critical positions.

3. The intelligent ship safety monitoring method based on deep learning according to claim 1, characterized in that: The general incidents include at least one of the following: not wearing a safety helmet during engineering operations and smoking in important positions.

4. A deep learning-based intelligent ship safety monitoring system, based on the deep learning-based intelligent ship safety monitoring method according to any one of claims 1-3, characterized in that, include: Monitoring data sampling module: used to acquire data collected by monitoring equipment on board the ship, including image data and video data; Shipboard target detection module: used to carry a target detection model, input the collected image data into the target detection model, and identify targets in the image data; Crew behavior recognition module: used to carry a behavior recognition model, input the collected video data into the behavior recognition model, and recognize the behavior of the crew in the video data; Crew target tracking module: used to track the crew members involved in the incident. During the tracking process, a face detection model is used to collect crew member face data. Once a clear face image of the crew member involved in the incident is obtained, the tracking stops. Crew identity matching module: This module is equipped with a facial recognition network to compare the collected facial images of the crew members involved with images in the crew facial database to confirm the crew members' identities. The data collected by the above module is stored as evidence and an alarm is triggered.

Images