On-duty ammunition handover management system and method

Abstract

The invention relates to the technical field of on-duty information management, and discloses an on-duty ammunition handover management system, which comprises a main processor MCU, a fingerprint identification module, a weighing module, a camera module, a data storage module, a cloud reading module, a video abnormity identification module and a touch screen display, the main processor MCU is respectively connected with the fingerprint identification module, the weighing module, the camera module, the cloud reading module and the touch display, and the cloud reading module is also connected with the video abnormity identification module. According to the on-duty ammunition handover management system and the on-duty ammunition handover management method provided by the invention, technologies such as visual identification, weight sensing and fingerprint identification are comprehensively utilized, and video abnormal event detection based on significant features and a time-space diagram network is set, so that long monitoring video abnormal detection and identification in an on-duty handover process are realized. The method solves a basic-level practical problem, and achieves the purposes of effective monitoring and scientific management of the handover process.

Description

Technical Field

[0001] The present invention relates to the technical field of duty information management, and in particular to a duty ammunition handover management system and method. Background Art

[0002] At present, on-duty posts still use the handover method of counting with mouth and observing with hands when handing over ammunition. There is a lack of scientific supervision of the handover process. There are problems such as difficulty in handing over the exact quantity of ammunition, difficulty in judging the authenticity of ammunition, difficulty in controlling on-duty personnel, and difficulty in grasping the handover time.

[0003] At the same time, the duty posts can only perform simple video surveillance during ammunition handover to output long videos, and are unable to detect anomalies in the surveillance videos. Currently, the technology for detecting and classifying abnormal events in long videos is still immature, and it is not possible to achieve accurate video anomaly detection and classification based on the unedited video samples provided by users. Summary of the Invention

[0004] The present invention provides a duty ammunition handover management system and method, which comprehensively utilizes visual recognition, weight sensing, fingerprint recognition and other technologies, and sets up video abnormal event detection based on significant features and spatiotemporal graph networks to realize abnormal detection and identification of long monitoring videos during the duty handover process. The above integrated duty ammunition handover management system solves practical problems at the grassroots level and realizes the purpose of effective monitoring and scientific management of the handover process.

[0005] The present invention provides an on-duty ammunition handover management system, comprising a main processor MCU, a fingerprint recognition module, a weighing module, a camera module, a data storage module, a cloud reading module, a video anomaly recognition module, and a touch screen display. The main processor MCU is respectively connected to the fingerprint recognition module, the weighing module, the camera module, the cloud reading module, and the touch screen display. The cloud reading module is also connected to the video anomaly recognition module.

[0006] The fingerprint recognition module is used to identify the fingerprint of the authenticating inspector and turn on the camera module when the fingerprint recognition authentication is passed;

[0007] The camera module includes multiple cameras for capturing handover videos at handover posts and transmitting the handover videos to the main processor MCU;

[0008] The weighing module uses weighing and measuring equipment, including tray magazine empty positions and tray bullet hole positions, to record and verify the weight and quantity of ammunition on duty;

[0009] The data storage module is used to store the handover video captured by the camera module, the on-duty ammunition handover record transmitted by the weighing module, all operation steps, personnel information, and perform data backup regularly;

[0010] The cloud reading module is used to read the handover video captured by the camera module and transmit the handover video to the video anomaly recognition module;

[0011] The video anomaly recognition module is used to monitor video anomaly events based on the handover video using a preset significant feature and spatiotemporal graph network;

[0012] The touch screen display is used to display in real time the weight and quantity of the on-duty ammunition recorded and verified by the weighing module during the handover and delivery process, and to display in real time the detection results of the video anomaly recognition module.

[0013] Furthermore, the video anomaly recognition module receives the handover video and inputs the handover video into a preset network based on significant features and spatiotemporal graphs for abnormal event monitoring, specifically in the following steps:

[0014] S1. Using the I3D network as a feature extraction network to extract the spatiotemporal features of the handover video, and applying the spatiotemporal features to the video anomaly detection model STFGCN to obtain the anomaly score of the clip, and establishing a feature sequence screening model to screen for significant feature sequences of video anomalies;

[0015] S2. Refine the abnormal region of the significant feature sequence of the abnormal event to obtain the precise location of the abnormal event;

[0016] S3. In the area of abnormal events, the I3D features and anomaly scores of the corresponding segments are obtained respectively, and the feature similarity graph and anomaly similarity graph of each segment are established. The abnormal event features are learned and integrated through the graph convolutional network.

[0017] S4. Establish an abnormal category imbalance loss function to realize abnormal event category classification.

[0018] Furthermore, the step S1 specifically includes:

[0019] S101, using a multi-instance segmentation method to perform frame processing on the handover video, dividing the continuous video into frames; inputting 16 consecutive frames of images as a video segment into the I3D network to obtain 1024-dimensional high-order video information; dividing the entire video into 32 continuous and non-overlapping segments, and extracting features from the video segments, obtaining 32 1024-dimensional vectors for each video feature representation, and obtaining a feature vector X∈R for each video V after feature extraction. T×F , where T is the video length or the number of frames, F is the feature dimension, and the extraction operation formula is:

[0020] X=Extra I3D (V)

[0021] S102, using the spatiotemporal fusion graph network model STFGCN to learn and obtain the spatiotemporal features f of each video segment STFGCN (X), sigmoid is used as the binary classification function of the abnormal / normal classification layer, so the abnormality score of each segment is obtained as follows:

[0022] S={S1,S2,…,S N}

[0023] Among them, S i ∈[0,1] is the abnormality score of the i-th segment, i=1,2,…,N;

[0024] S103, the jth significant feature sequence Z of the abnormal event in the handover video j The screening model was established as:

[0025]

[0026] Among them, s j 、e j are the starting and ending segments of the j-th significant feature sequence of the abnormal event in the video; shot(i) is the i-th segment in the video; score(·) is the score obtained for the corresponding video segment.

[0027] Furthermore, the step S2 specifically includes:

[0028] S201, for the relevant nodes v at the head and tail of the significant feature sequence i , refine the length of the significant feature sequence of the jth abnormal event. If the length is ≤ 3 segments, no refinement is required; if the length is > 3 and ≤ 5, only refine the segments at the head and tail of the sequence; if the length is > 5, refine the two consecutive segments at the head and tail;

[0029] S202: Use a weight penalty method to correct the scores of the head and tail related segments. The weight of each segment is calculated as follows:

[0030] For 3 <e j -s j +1≤5, where s j ,e j ∈{1,2,…,N}, the head and tail nodes of the sequence v i The score correction weight is:

[0031]

[0032] For e j -s j +1>5, where s j ,e j∈{1,2,...,N}, the head and tail nodes v of the sequence i The score correction weight is:

[0033]

[0034] That is, the closer the fragment is to the head and tail of the jth region, the smaller the abnormal score weight is, and the farther the fragment is from the head and tail, the larger the abnormal score weight is;

[0035] S203. After obtaining the score correction weights of the head and tail of the significant feature sequence of the abnormal event, the weights are used to correct the abnormal scores of the head and tail related segments to refine the significant feature sequence, which is expressed as follows:

[0036] S(i)←a i ×S(i),i∈(s j ,e j ]

[0037] S204. After the abnormal scores of the head and tail related segments are corrected, the start and end segments of the significant feature sequence of the abnormal event are further narrowed and corrected to s according to the situation where the score is ≥ 0.5. j 、e j , in order to improve the accuracy of abnormal event positioning.

[0038] Furthermore, the step S3 specifically includes:

[0039] S301, sampling an equal amount of abnormal event sequences to obtain K equal amount of abnormal event sequence segment features;

[0040] S302, using an abnormal event feature learning and classification method based on a feature similarity graph and an abnormality similarity graph, constructing a feature similarity graph and an abnormality similarity graph respectively through the sampled abnormal sequence fragments, and using a graph convolutional network to implement feature learning of the abnormal sequence fragments, respectively, to obtain spatiotemporal appearance features and abnormal features;

[0041] S303: Fusing the spatiotemporal appearance features and abnormal features, and considering the imbalance of abnormal event categories, classifying video abnormal events; specifically, the following steps are performed:

[0042] After propagation through the nodes of GCN, a new feature matrix is obtained K is the number of sequence segments after sampling, and C is the number of abnormal event categories. All learned features are weighted and fused to obtain the characteristic expression of abnormal events. The formula is:

[0043]

[0044] in, is the score matrix; Z∈R Cis the feature representation of abnormal events; θ is the hyperparameter of the fusion process;

[0045] The softmax classification layer operation is performed on the feature representation Z after fusion learning, and the score of each abnormal event class is obtained as P = softmax (Z).

[0046] Furthermore, in step S302, a graph convolutional network is used to construct a feature similarity graph to obtain spatiotemporal appearance features, specifically including:

[0047] Select each segment of the sampled abnormal sequence and mark it as a node of the feature similarity graph Construct a feature similarity graph and use cosine similarity to generate edge weights and node and The edge weight of is calculated as:

[0048]

[0049] in, are the features of the i-th and j-th segments of the abnormal sequence after sampling;

[0050] By using the threshold operation to retain the effective similarity weight information between nodes, GCN and The edge weights of are modified to:

[0051]

[0052] in, is the adjacency matrix A f ∈R K×K The edge weight of , α is the adjustment threshold;

[0053] Will Embedded as a node on the graph The features of the adjacency matrix A f Edge information E in the embedded graph f , forming a feature similarity graph G f Input into the graph convolutional network for feature learning, the propagation process is as follows:

[0054]

[0055] Among them, W f is the weight matrix to be learned in GCN; norm(·) function is the Laplace regularization process; is the feature output of the l+1th layer of GCN. After the association learning of GCN, the spatiotemporal appearance characteristics of the sequence fragment can be obtained;

[0056] A graph convolutional network is used to construct an abnormal similarity graph to obtain abnormal features, including:

[0057] Select each fragment of the sampled abnormal sequence and mark them as abnormal similarity graph nodes Construct an abnormal similarity graph, use Gaussian kernel function as the edge weight measurement in the abnormal similarity graph, and node and The edge weight calculation formula is:

[0058]

[0059] in, are the abnormal probability scores corresponding to the i-th and j-th video segments of the abnormal sequence after sampling; A a is the adjacency matrix of the abnormal similarity graph generated by the Gaussian kernel;

[0060] By comparing the adjacency matrix of the abnormal similarity graph to the node V a Connect to form an unusually similar graph in Abnormal fragment features extracted by the STFGCN model, side information E a Take the adjacency matrix A a To express;

[0061] Similarly, similar to the feature similarity graph, G a Input into the GCN network for abnormal feature learning, and its propagation process is:

[0062]

[0063] Among them, W a is the GCN network layer parameter; is the output of the l+1th layer. After the association learning of GCN, the abnormal features of the sequence fragments can be obtained.

[0064] Furthermore, in step S4, a class imbalance loss function is used as the loss function for abnormal event classification, and its expression is:

[0065] L1=-β a；i (1-P a；i ) γ lgP a；i

[0066] in, N is the size of the training sample batch size, and C is the number of classification categories;

[0067] The sorting mean square error is used as the constraint function of the STFGCN model, and its formula is:

[0068]

[0069] Among them, S i；j is the abnormal score of the jth video clip of the i-th video, and the maximum value is taken as the representation value of the entire video; max(S i；j ) is closer to 1, the greater the abnormal probability of the i-th video; is the abnormal label {0,1} of the video that is irrelevant to the class, 1 represents the abnormal video label, and 0 represents the normal video label; batch is the total number of samples input into the network at one time;

[0070] The STFGCN model and the abnormal event classification loss function are combined to constrain the training of the overall detection and classification model. The overall loss is:

[0071] Loss=L1+λ d L2

[0072] Among them, λ d is the adjustment parameter of the loss function of the STFGCN model.

[0073] Furthermore, it also includes an indicator light, a switch and a power supply;

[0074] The indicator light is connected to the main processor MCU, and the indicator light includes three colors: red, yellow and green. Among them, the yellow and green light colors represent the models of different types of bullets, and red represents abnormal bullets;

[0075] The switches are connected to a power supply and a main processor MCU respectively.

[0076] The present invention also provides a method for managing the handover of on-duty ammunition. Based on the above-mentioned on-duty ammunition handover management system, the method specifically includes:

[0077] S1. The on-duty ammunition handover management system is in a normally closed state by default and cannot be opened without a specific operation process. When it is opened, the supervisor leads two personnel to the handover post. The supervisor verifies the fingerprint through the fingerprint recognition module to activate the on-duty ammunition handover management system. After the system successfully recognizes the supervisor's fingerprint, the camera module is automatically turned on to ensure that the entire handover process is under monitoring.

[0078] S2. The handover person places the full magazine into the magazine tray space of the weighing module. The system recognizes the weight of the full magazine and automatically switches to the handover mode.

[0079] S3. Bullet removal and recording: The handover person picks up the magazine again, takes out the bullets one by one and places them into the bullet holes on the tray. Each time a bullet is placed, the touch screen display shows the order and related information of the bullets in real time.

[0080] Confirmation of completion of ammunition inspection: After all bullets are taken out of the real magazine, the empty magazine is placed in the magazine space designated by the system, and the handover person system interface on the touch screen display displays "Ammunition inspection completed";

[0081] S4, handover mode activated: After the handover person takes out the empty magazine, the system automatically switches to handover mode;

[0082] Bullet receiving operation: The receiving personnel take out the bullets from the bullet holes on the tray and press them into the magazine one by one;

[0083] S5. Handover confirmation: The handover person places the full magazine into the empty magazine space on the tray, and the touch screen displays the weight in real time. When the weight is the same as before the handover, the handover person system interface on the touch screen displays "Handover Completed";

[0084] S6. Confirmation of completion of handover: The handover supervisor clicks the "End Handover" button in the system interface of the touch screen display and verifies the fingerprint through the fingerprint recognition module again. After the system recognizes the handover supervisor's fingerprint and confirms it, it closes the on-duty ammunition handover management system and completes the entire handover process.

[0085] Furthermore, it also includes:

[0086] Data storage and backup: Throughout the entire handover process, the system must store all operating steps, personnel information, actual magazine weight, number of bullets, and video surveillance content in real time, and perform regular data backup to ensure data integrity and traceability;

[0087] Inspection and call: The management personnel check the duty handover situation within a certain period of time, enter the management interface through the administrator account, and call up relevant handover data and video materials.

[0088] The beneficial effects of the present invention are:

[0089] The fingerprint recognition module in the present invention identifies and authenticates the fingerprint of the handover supervisor, and when the fingerprint recognition authentication is passed, the camera module is turned on; the camera module shoots the handover video at the handover post, and the weighing module includes a tray magazine empty space and a tray bullet hole position, records and verifies the weight and quantity of the on-duty ammunition; the cloud reading module reads the handover video and transmits it to the video anomaly recognition module, so as to use the preset significant features and spatiotemporal graph network based on the handover video to perform video anomaly event monitoring; the touch screen display displays in real time the weight and quantity of the on-duty ammunition recorded and verified by the weighing module during the handover and handover process, and displays the detection results of the video anomaly recognition module in real time; the visual recognition, weight sensing, fingerprint recognition and other technologies are comprehensively used, and video anomaly event detection based on significant features and spatiotemporal graph network is set to realize long-term monitoring video anomaly detection and identification during the on-duty handover process. The above integrated on-duty ammunition handover management system solves practical problems at the grassroots level and realizes the purpose of effective monitoring and scientific management of the handover process. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a structural diagram of the on-duty ammunition handover management system of the present invention.

[0091] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0092] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0093] like Figure 1 As shown, the present invention provides an on-duty ammunition handover management system, including a main processor MCU, a fingerprint recognition module, a weighing module, a camera module, a data storage module, a cloud reading module, a video anomaly recognition module, a touch screen display, an indicator light, a switch and a power supply. The main processor MCU is respectively connected to the fingerprint recognition module, the weighing module, the camera module, the cloud reading module and the touch display, the cloud reading module is also connected to the video anomaly recognition module, the indicator light is connected to the main processor MCU, and the switch is respectively connected to the power supply and the main processor MCU.

[0094] The fingerprint recognition module is used to identify and authenticate the fingerprint of the handover inspector, and when the fingerprint recognition authentication is passed, the camera module is turned on; the camera module includes multiple cameras for shooting the handover video at the handover post and transmitting the handover video to the main processor MCU; the weighing module uses a weighing measurement device, including a tray magazine empty space and a tray bullet hole position, for recording and verifying the weight and quantity of the on-duty ammunition; the data storage module is used to store the handover video shot by the camera module, the on-duty ammunition handover record transmitted by the weighing module, all operation steps, personnel information, and perform data backup regularly; the cloud reading module reads the handover video shot by the camera module through API calls, and transmits the handover video to the video anomaly recognition module; the video anomaly recognition module is used to monitor video anomaly events based on the handover video using a preset network based on significant features and spatiotemporal graphs; the touch screen display is used to display in real time the weight and quantity of the on-duty ammunition recorded and verified by the weighing module during the handover and handover process, as well as the detection results of the video anomaly recognition module in real time.

[0095] Among them, the user turns on the power, the supervisor verifies the fingerprint through the fingerprint recognition module, the camera turns on the recording mode, the camera module records the supervisor, and transmits the data to the main processor in real time.

[0096] When storing or removing ammunition, the weighing module increases or decreases the weight and transmits the data to the main processor in real time. The main processor identifies the nature of the ammunition and turns on the indicator light. Yellow and green light colors indicate different types of ammunition, and red indicates abnormal ammunition.

[0097] Video anomaly recognition module:

[0098] The video anomaly recognition module receives the handover video and inputs the handover video into a preset network based on significant features and spatiotemporal graphs for abnormal event monitoring. The specific steps are as follows:

[0099] S1. Use the I3D network as the feature extraction network to extract the spatiotemporal features of the handover video, and pass the spatiotemporal features through the video anomaly detection model STFGCN to obtain the anomaly score value of the clip, and establish a feature sequence screening model to perform significant feature sequence screening of video abnormal events.

[0100] S101, feature extraction: Use a multi-instance segmentation method to perform frame processing on the handover video, and divide the continuous video into frames; input 16 consecutive frames of images as a video segment into the I3D network to obtain 1024-dimensional high-order video information; divide the entire video into 32 continuous and non-overlapping segments, and extract features from the video segments, so that each video is represented by 32 1024-dimensional vectors. For each video V, after feature extraction, a feature vector X∈R is obtained.T×F , where T is the video length or the number of frames, F is the feature dimension, and the extraction operation formula is:

[0101] X=Extra I3D (V)

[0102] S102, video anomaly location: Anomaly classification requires locating the location where the event occurred in the video, where the probability of anomaly occurrence P(V i ) is used to quantify the possibility of abnormality in the i-th frame or i-th video segment, so positioning detection can be regarded as a regression task. The spatiotemporal fusion graph network model STFGCN is used to learn the spatiotemporal features f of each video segment. STFGCN (X), sigmoid is used as the binary classification function of the abnormal / normal classification layer, so the abnormality score of each segment is obtained as follows:

[0103] S={S1,S2,…,S N}

[0104] Among them, S i ∈[0,1] is the abnormality score of the i-th segment, i=1,2,…,N;

[0105] S103. Screening of significant feature sequences of abnormal events. The STFGCN model focuses on the probability of detecting that each video clip is abnormal, that is, the score. It uses a common method to default the clip to an abnormal clip when the score is greater than 0.5, thereby detecting the occurrence of abnormalities in the video. However, the occurrence of abnormal events has a certain continuous process. This model ignores the uncertainty of the probability of abnormality of the clip, which will cause a normal small clip to be mistaken for abnormal. It will also cause the relevant abnormal clips to be judged as normal during the duration of the abnormal event. Taking into account that most abnormal events do not occur in a flash, but have a certain time duration, and considering the uncertainty of the judgment of clip abnormality, a method for screening significant feature sequences of abnormal events is proposed to determine the starting and ending positions of the abnormal events. The model assumes that there are multiple significant feature sequences of abnormal events in the video. Each significant feature sequence of abnormal events starts from a segment with an abnormality score greater than or equal to 0.5 and ends with a segment with an abnormality score greater than or equal to 0.5. If a single segment has an abnormality score less than 0.5, it is considered to be an uncertainty in the calculation of the abnormality score probability, and the segment is still considered a continuous segment of the significant feature sequence of abnormal events; at the same time, if only one segment has an abnormality score greater than or equal to 0.5 and there are more than two consecutive segments with abnormality scores less than 0.5 before and after, then the segment is a normal segment. Therefore, the jth significant feature sequence Z of the abnormal event in the handover video is j The screening model was established as:

[0106]

[0107] Among them, s j 、e j where shot(i) represents the start and end of the jth significant feature sequence of an abnormal event in the video; shot(i) represents the i-th segment in the video; and score(·) represents the score obtained for the corresponding video segment. During an abnormal event, if the abnormal event sequence is interrupted due to uncertainty in abnormality judgment, the model can tolerate this uncertainty and maintain the abnormal event sequence. Furthermore, if a single segment in a normal area is occasionally judged as abnormal, it can be eliminated, thereby screening for the appropriate start and end locations of the significant feature sequence of the abnormal event.

[0108] S2. Refine the abnormal region of the significant feature sequence of the abnormal event to obtain the precise location of the abnormal event.

[0109] The screening of significant feature sequences of abnormal events described above depends largely on the anomaly score probabilities of the segments. From the perspective of the STFGCN model, the model's segment time graph construction involves n adjacent segments before and after. This results in the propagation of abnormal feature learning in the head and tail related segments of the abnormal event sequence. The calculation of the anomaly score probabilities for these segments is unrealistic and overly large, resulting in a certain spread in the start and end times of the abnormal event sequence. While the abnormal regions extracted by screening significant feature sequences of abnormal events can well maintain the temporal order of the abnormal segments, they still show anomaly sequence spread at the beginning and end of the sequence, resulting in inaccurate positioning. Therefore, to more accurately locate the abnormal event segment regions, this paper proposes an abnormal region refinement algorithm. This algorithm uses temporal continuity similarity to modify the anomaly score probabilities of the head and tail related segments in the abnormal event sequence, thereby refining the significant feature sequences of abnormal events.

[0110] S201, for the relevant nodes v at the head and tail of the significant feature sequence i , refine the length of the significant feature sequence of the jth abnormal event. If the length is ≤ 3 segments, no refinement is required; if the length is > 3 and ≤ 5, only refine the segments at the head and tail of the sequence; if the length is > 5, refine the two consecutive segments at the head and tail;

[0111] S202: Use a weight penalty method to correct the scores of the head and tail related segments. The weight of each segment is calculated as follows:

[0112] For 3 <e j -s j +1≤5, where s j ,e j ∈{1,2,…,N}, the head and tail nodes of the sequence v i The score correction weight is:

[0113]

[0114] For e j -s j +1>5, where s j ,e j ∈{1,2,...,N}, the head and tail nodes v of the sequence i The score correction weight is:

[0115]

[0116] That is, the closer the fragment is to the head and tail of the jth region, the smaller the abnormal score weight is, and the farther the fragment is from the head and tail, the larger the abnormal score weight is;

[0117] S203. After obtaining the score correction weights of the head and tail of the significant feature sequence of the abnormal event, the weights are used to correct the abnormal scores of the head and tail related segments to refine the significant feature sequence, which is expressed as follows:

[0118] S(i)←a i ×S(i),i∈(s j ,e j ]

[0119] S204. After the abnormal scores of the head and tail related segments are corrected, the start and end segments of the significant feature sequence of the abnormal event are further narrowed and corrected to s according to the situation where the score is ≥ 0.5. j 、e j , in order to improve the accuracy of abnormal event positioning.

[0120] S3. In the area of abnormal events, the I3D features and anomaly scores of the corresponding segments are obtained respectively, and the feature similarity graph and anomaly similarity graph of each segment are established. The abnormal event features are learned and integrated through the graph convolutional network.

[0121] Different types of video abnormal events have significant differences in the spatiotemporal appearance of the significant feature sequence segments and the anomaly scores of the segments. Furthermore, the length of the abnormal sequence can also vary. Therefore, in order to effectively learn and classify the features of abnormal event categories, we need to use a set of algorithms to identify and classify the abnormal events.

[0122] S301, sampling an equal amount of abnormal event sequences to obtain K equal amount of abnormal event sequence segment features;

[0123] S302. At the same time, there is an intrinsic correlation between the features of event sequence fragments. In order to learn this intrinsic correlation more effectively, an abnormal event feature learning and classification method based on feature similarity graph and abnormality similarity graph is adopted. The feature similarity graph and abnormality similarity graph are respectively constructed through the sampled abnormal sequence fragments, and the graph convolutional network is used to realize the feature learning of the abnormal sequence fragments, and the spatiotemporal appearance characteristics and abnormal characteristics are obtained.

[0124] (1) Using graph convolutional networks to construct feature similarity graphs to obtain spatiotemporal appearance features, specifically including:

[0125] Select each segment of the sampled abnormal sequence and mark it as a node of the feature similarity graph Construct a feature similarity graph and use cosine similarity to generate edge weights and node and The edge weight of is calculated as:

[0126]

[0127] in, are the features of the i-th and j-th segments of the abnormal sequence after sampling;

[0128] By using the threshold operation to retain the effective similarity weight information between nodes, GCN and The edge weights of are modified to:

[0129]

[0130] in, is the adjacency matrix A f ∈R K×K The edge weight of , α is the adjustment threshold;

[0131] Will Embedded as a node on the graph The features of the adjacency matrix A f Edge information E embedded in the graph f , forming a feature similarity graph G f Input into the graph convolutional network for feature learning, the propagation process is as follows:

[0132]

[0133] Among them, W f is the weight matrix to be learned in GCN; norm(·) function is the Laplace regularization process; is the feature output of the l+1th layer of GCN. After the association learning of GCN, the spatiotemporal appearance characteristics of the sequence fragment can be obtained;

[0134] (2) Using graph convolutional networks to construct anomaly similarity graphs to obtain anomaly features, specifically including:

[0135] Select each fragment of the sampled abnormal sequence and mark them as abnormal similarity graph nodes Construct an abnormal similarity graph, use Gaussian kernel function as the edge weight measurement in the abnormal similarity graph, and node and The edge weight calculation formula is:

[0136]

[0137] in, are the abnormal probability scores corresponding to the i-th and j-th video segments of the abnormal sequence after sampling; A a is the adjacency matrix of the abnormal similarity graph generated by the Gaussian kernel;

[0138] By comparing the adjacency matrix of the abnormal similarity graph to the node V a Connect to form an unusually similar graph in Abnormal fragment features extracted by the STFGCN model, side information E a Take the adjacency matrix A a To express;

[0139] Similarly, similar to the feature similarity graph, G a Input into the GCN network for abnormal feature learning, and its propagation process is:

[0140]

[0141] Among them, W a is the GCN network layer parameter; is the output of the l+1th layer. After the association learning of GCN, the abnormal features of the sequence fragments can be obtained.

[0142] S303: Fusing the spatiotemporal appearance features and abnormal features, and considering the imbalance of abnormal event categories, classifying video abnormal events; specifically, the following steps are performed:

[0143] After propagation through the nodes of GCN, a new feature matrix is obtained K is the number of sequence segments after sampling, and C is the number of abnormal event categories. All learned features are weighted and fused to obtain the characteristic expression of abnormal events. The formula is:

[0144]

[0145] in, is the score matrix; Z∈R Cis the feature representation of abnormal events; θ is the hyperparameter of the fusion process;

[0146] The softmax classification layer operation is performed on the feature representation Z after fusion learning, and the score of each abnormal event class is obtained as P = softmax (Z).

[0147] S4. Establish an abnormal category imbalance loss function to realize abnormal event category classification.

[0148] In a real environment, the probability of occurrence of various types of anomalies varies greatly. The collected data set will have an imbalanced training sample problem. Some abnormal event samples have a large proportion, while others have a small proportion. The loss function obtained by using cross entropy during training will be more sensitive to the classification decision of categories with a large number of samples, making it difficult for the model to learn reasonable abnormal event categories. Therefore, the category imbalance loss (abnormal-imbalance loss, AIL) is used as the loss function for abnormal event classification. Its formal expression is:

[0149] L1=-β a；i (1-P a；i ) γ lgP a；i

[0150] in, N is the size of the training sample batch size, and C is the number of classification categories;

[0151] The sorting mean square error is used as the constraint function of the STFGCN model, and its formula is:

[0152]

[0153] Among them, S i；j is the abnormal score of the jth video clip of the i-th video, and the maximum value is taken as the representation value of the entire video; max(S i；j ) is closer to 1, the greater the abnormal probability of the i-th video; is the abnormal label {0,1} of the video that is irrelevant to the class, 1 represents the abnormal video label, and 0 represents the normal video label; batch is the total number of samples input into the network at one time;

[0154] The STFGCN model and the abnormal event classification loss function are combined to constrain the training of the overall detection and classification model. The overall loss is:

[0155] Loss=L1+λ d L2

[0156] Among them, λ d is the adjustment parameter of the loss function of the STFGCN model.

[0157] The present invention also provides a method for managing the handover of on-duty ammunition. Based on the above-mentioned on-duty ammunition handover management system, the method specifically includes:

[0158] S1. The on-duty ammunition handover management system is in a normally closed state by default and cannot be opened without a specific operation process. When it is opened, the supervisor leads two personnel to the handover post. The supervisor verifies the fingerprint through the fingerprint recognition module to activate the on-duty ammunition handover management system. After the system successfully recognizes the supervisor's fingerprint, the camera module is automatically turned on to ensure that the entire handover process is under monitoring.

[0159] S2. The handover person places the full magazine into the magazine tray space of the weighing module. The system recognizes the weight of the full magazine and automatically switches to the handover mode.

[0160] S3. Handover process operation and system functions.

[0161] 1> Bullet removal and recording: The handover person picks up the magazine again, takes out the bullets one by one and places them into the bullet holes on the tray. Each time a bullet is placed, the touch screen display shows the order and related information of the bullets in real time;

[0162] 2> Confirmation of completion of ammunition inspection: After all bullets are taken out of the real magazine, the empty magazine is placed in the magazine space designated by the system, and the handover person system interface on the touch screen display displays "Ammunition inspection completed";

[0163] S4. Handover process operation and system functions.

[0164] 1> Handover mode activated: After the receiver takes out the empty magazine, the system automatically switches to handover mode;

[0165] 2> Bullet receiving operation: The receiving personnel take out the bullets from the bullet holes on the tray and press them into the magazine one by one;

[0166] S5. Handover confirmation: The handover person places the full magazine into the empty magazine space on the tray, and the touch screen displays the weight in real time. When the weight is the same as before the handover, the handover person system interface on the touch screen displays "Handover Completed";

[0167] S6. End the process operation and system functions.

[0168] Confirmation of handover completion: The handover supervisor clicks the "End Handover" button in the system interface of the touch screen display and verifies the fingerprint through the fingerprint recognition module again. After the system recognizes the handover supervisor's fingerprint and confirms that it is correct, it closes the on-duty ammunition handover management system and completes the entire handover process.

[0169] Data storage and backup: During the entire handover process, the system must store all operating steps, personnel information, item information (including weight, number of bullets, etc.), and video surveillance content in real time, and perform regular data backup to ensure data integrity and traceability;

[0170] S7. Check and retrieve: The management personnel check the duty handover situation within a certain period of time, enter the management interface through the administrator account, and retrieve relevant handover data and video materials.

[0171] The fingerprint recognition module in the present invention identifies and authenticates the fingerprint of the handover supervisor, and when the fingerprint recognition authentication is passed, the camera module is turned on; the camera module shoots the handover video at the handover post, and the weighing module includes a tray magazine empty space and a tray bullet hole position, records and verifies the weight and quantity of the on-duty ammunition; the cloud reading module reads the handover video and transmits it to the video anomaly recognition module, so as to use the preset significant features and spatiotemporal graph network based on the handover video to perform video anomaly event monitoring; the touch screen display displays in real time the weight and quantity of the on-duty ammunition recorded and verified by the weighing module during the handover and handover process, and displays the detection results of the video anomaly recognition module in real time; the visual recognition, weight sensing, fingerprint recognition and other technologies are comprehensively used, and video anomaly event detection based on significant features and spatiotemporal graph network is set to realize long-term monitoring video anomaly detection and identification during the on-duty handover process. The above integrated on-duty ammunition handover management system solves practical problems at the grassroots level and realizes the purpose of effective monitoring and scientific management of the handover process.

[0172] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0173] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A duty ammunition handover management system, characterized in that: It includes a main processor MCU, a fingerprint recognition module, a weighing module, a camera module, a data storage module, a cloud reading module, a video anomaly recognition module, and a touch screen display. The main processor MCU is respectively connected to the fingerprint recognition module, the weighing module, the camera module, the cloud reading module and the touch display. The cloud reading module is also connected to the video anomaly recognition module; The fingerprint recognition module is used to identify the fingerprint of the authenticating inspector and turn on the camera module when the fingerprint recognition authentication is passed; The camera module includes multiple cameras for capturing handover videos at handover posts and transmitting the handover videos to the main processor MCU; The weighing module uses weighing and measuring equipment, including tray magazine empty positions and tray bullet hole positions, to record and verify the weight and quantity of ammunition on duty; The data storage module is used to store the handover video captured by the camera module, the on-duty ammunition handover record transmitted by the weighing module, all operation steps, personnel information, and perform data backup regularly; The cloud reading module is used to read the handover video captured by the camera module and transmit the handover video to the video anomaly recognition module; The video anomaly recognition module is used to monitor video anomaly events based on the handover video using a preset significant feature and spatiotemporal graph network; The touch screen display is used to display in real time the weight and quantity of the on-duty ammunition recorded and verified by the weighing module during the handover and delivery process, and to display in real time the detection results of the video anomaly recognition module.

2. The on-duty ammunition handover management system according to claim 1, characterized in that: The video anomaly recognition module receives the handover video and inputs the handover video into a preset network based on significant features and spatiotemporal graphs for abnormal event monitoring. The specific steps are as follows: S1. Using the I3D network as a feature extraction network to extract the spatiotemporal features of the handover video, and applying the spatiotemporal features to the video anomaly detection model STFGCN to obtain the anomaly score of the clip, and establishing a feature sequence screening model to screen for significant feature sequences of video anomalies; S2. Refine the abnormal region of the significant feature sequence of the abnormal event to obtain the precise location of the abnormal event; S3. In the area of abnormal events, the I3D features and anomaly scores of the corresponding segments are obtained respectively, and the feature similarity graph and anomaly similarity graph of each segment are established. The abnormal event features are learned and integrated through the graph convolutional network. S4. Establish an abnormal category imbalance loss function to realize abnormal event category classification.

3. The on-duty ammunition handover management system according to claim 1, characterized in that: The step S1 specifically includes: S101, using a multi-instance segmentation method to perform frame processing on the handover video, dividing the continuous video into frames; inputting 16 consecutive frames of images as a video segment into the I3D network to obtain 1024-dimensional high-order video information; dividing the entire video into 32 continuous and non-overlapping segments, and extracting features from the video segments, obtaining 32 1024-dimensional vectors for each video feature representation, and obtaining a feature vector X∈R for each video V after feature extraction. T×F , where T is the video length or the number of frames, F is the feature dimension, and the extraction operation formula is: X=Extra I3D (V) S102, using the spatiotemporal fusion graph network model STFGCN to learn and obtain the spatiotemporal features f of each video segment STFGCN (X), sigmoid is used as the binary classification function of the abnormal / normal classification layer, so the abnormality score of each segment is obtained as follows: S={S1,S2,…,S N } Among them, S i ∈[0,1] is the abnormality score of the i-th segment, i=1,2,…,N; S103, the jth significant feature sequence Z of the abnormal event in the handover video j The screening model was established as: Among them, s j 、e j are the starting and ending segments of the j-th significant feature sequence of the abnormal event in the video; shot(i) is the i-th segment in the video; score(·) is the score obtained for the corresponding video segment.

4. The on-duty ammunition handover management system according to claim 3, characterized in that: The step S2 specifically includes: S201, for the relevant nodes v at the head and tail of the significant feature sequence i , refine the length of the significant feature sequence of the jth abnormal event. If the length is ≤ 3 segments, no refinement is required; if the length is > 3 and ≤ 5, only refine the segments at the head and tail of the sequence; if the length is > 5, refine the two consecutive segments at the head and tail; S202: Use a weight penalty method to correct the scores of the head and tail related segments. The weight of each segment is calculated as follows: For 3 <e j -s j +1≤5, where s j ,e j ∈{1,2,…,N}, the head and tail nodes of the sequence v i The score correction weight is: For e j -s j +1>5, where s j ,e j ∈{1,2,...,N}, the head and tail nodes v of the sequence i The score correction weight is: That is, the closer the fragment is to the head and tail of the jth region, the smaller the abnormal score weight is, and the farther the fragment is from the head and tail, the larger the abnormal score weight is; S203. After obtaining the score correction weights of the head and tail of the significant feature sequence of the abnormal event, the weights are used to correct the abnormal scores of the head and tail related segments to refine the significant feature sequence, which is expressed as follows: S(i)←a i ×S(i),i∈(s j ,e j ] S204. After the abnormal scores of the head and tail related segments are corrected, the start and end segments of the significant feature sequence of the abnormal event are further narrowed and corrected to s according to the situation where the score is ≥ 0.

5. j 、e j , in order to improve the accuracy of abnormal event positioning.

5. The on-duty ammunition handover management system according to claim 4, characterized in that: The step S3 specifically includes: S301, sampling an equal amount of abnormal event sequences to obtain K equal amount of abnormal event sequence segment features; S302, using an abnormal event feature learning and classification method based on a feature similarity graph and an abnormality similarity graph, constructing a feature similarity graph and an abnormality similarity graph respectively through the sampled abnormal sequence fragments, and using a graph convolutional network to implement feature learning of the abnormal sequence fragments, respectively, to obtain spatiotemporal appearance features and abnormal features; S303: Fusing the spatiotemporal appearance features and abnormal features, and considering the imbalance of abnormal event categories, classifying video abnormal events; specifically, the following steps are performed: After propagation through the nodes of GCN, a new feature matrix is obtained K is the number of sequence segments after sampling, and C is the number of abnormal event categories. All learned features are weighted and fused to obtain the characteristic expression of abnormal events. The formula is: in, is the score matrix; Z∈R C is the feature representation of abnormal events; θ is the hyperparameter of the fusion process; The softmax classification layer operation is performed on the feature representation Z after fusion learning, and the score of each abnormal event class is obtained as P = softmax (Z).

6. The on-duty ammunition handover management system according to claim 5, characterized in that: In step S302, a graph convolutional network is used to construct a feature similarity graph to obtain spatiotemporal appearance features, specifically including: Select each segment of the sampled abnormal sequence and mark it as a node of the feature similarity graph Construct a feature similarity graph and use cosine similarity to generate edge weights and node and The edge weight of is calculated as: in, are the features of the i-th and j-th segments of the abnormal sequence after sampling; By using the threshold operation to retain the effective similarity weight information between nodes, GCN and The edge weights of are modified to: in, is the adjacency matrix A f ∈R K×K The edge weight of , α is the adjustment threshold; Will Embedded as a node on the graph The features of the adjacency matrix A f Edge information E embedded in the graph f , forming a feature similarity graph G f Input into the graph convolutional network for feature learning, the propagation process is as follows: Among them, W f is the weight matrix to be learned in GCN; norm(·) function is the Laplace regularization process; is the feature output of the l+1th layer of GCN. After the association learning of GCN, the spatiotemporal appearance characteristics of the sequence fragment can be obtained; A graph convolutional network is used to construct an abnormal similarity graph to obtain abnormal features, including: Select each fragment of the sampled abnormal sequence and mark them as abnormal similarity graph nodes Construct an abnormal similarity graph, use Gaussian kernel function as the edge weight measurement in the abnormal similarity graph, and node and The edge weight calculation formula is: in, are the abnormal probability scores corresponding to the i-th and j-th video segments of the abnormal sequence after sampling; A a is the adjacency matrix of the abnormal similarity graph generated by the Gaussian kernel; By associating the adjacency matrix of the abnormal similarity graph to the node V a Connect to form an unusually similar graph in Abnormal fragment features extracted by the STFGCN model, side information E a Take the adjacency matrix A a To express; Similarly, similar to the feature similarity graph, G a Input into the GCN network for abnormal feature learning, and its propagation process is: Among them, W a is the GCN network layer parameter; is the output of the l+1th layer. After the association learning of GCN, the abnormal features of the sequence fragments can be obtained.

7. The on-duty ammunition handover management system according to claim 6, characterized in that: In step S4, the class imbalance loss function is used as the loss function for abnormal event classification, and its expression is: L1=-β a；i (1-P a；i ) γ lgP a；i in, N is the size of the training sample batch size, and C is the number of classification categories; The sorting mean square error is used as the constraint function of the STFGCN model, and its formula is: Among them, S i；j is the abnormal score of the jth video clip of the i-th video, and the maximum value is taken as the representation value of the entire video; max(S i；j ) is closer to 1, the greater the abnormal probability of the i-th video; is the abnormal label {0,1} of the video that is irrelevant to the class, 1 represents the abnormal video label, and 0 represents the normal video label; batch is the total number of samples input into the network at one time; The STFGCN model and the abnormal event classification loss function are combined to constrain the training of the overall detection and classification model. The overall loss is: Loss=L1+λ d L2 Among them, λ d is the adjustment parameter of the loss function of the STFGCN model.

8. The on-duty ammunition handover management system according to claim 1, characterized in that: It also includes indicator lights, switches, and power supplies; The indicator light is connected to the main processor MCU, and the indicator light includes three colors: red, yellow and green. Among them, the yellow and green light colors represent the models of different types of bullets, and red represents abnormal bullets; The switches are connected to a power supply and a main processor MCU respectively.

9. A method for managing the transfer of on-duty ammunition, characterized in that: Based on the on-duty ammunition handover management system according to any one of claims 1 to 8, the method specifically comprises: S1. The on-duty ammunition handover management system is in a normally closed state by default and cannot be opened without a specific operation process. When it is opened, the supervisor leads two personnel to the handover post. The supervisor verifies the fingerprint through the fingerprint recognition module to activate the on-duty ammunition handover management system. After the system successfully recognizes the supervisor's fingerprint, the camera module is automatically turned on to ensure that the entire handover process is under monitoring. S2. The handover person places the full magazine into the magazine tray space of the weighing module. The system recognizes the weight of the full magazine and automatically switches to the handover mode. S3. Bullet removal and recording: The handover person picks up the magazine again, takes out the bullets one by one and places them into the bullet holes on the tray. Each time a bullet is placed, the touch screen display shows the order and related information of the bullets in real time. Confirmation of completion of ammunition inspection: After all bullets are taken out of the real magazine, the empty magazine is placed in the magazine space designated by the system, and the handover person system interface on the touch screen display displays "Ammunition inspection completed"; S4, handover mode activated: After the handover person takes out the empty magazine, the system automatically switches to handover mode; Bullet receiving operation: The receiving personnel take out the bullets from the bullet holes on the tray and press them into the magazine one by one; S5. Handover confirmation: The handover person places the full magazine into the empty magazine space on the tray, and the touch screen displays the weight in real time. When the weight is the same as before the handover, the handover person system interface on the touch screen displays "Handover Completed"; S6. Confirmation of handover completion: The handover supervisor clicks the "End Handover" button in the system interface of the touch screen display and verifies the fingerprint through the fingerprint recognition module again. After the system recognizes the handover supervisor's fingerprint and confirms it, it closes the on-duty ammunition handover management system and completes the entire handover process.

10. The on-duty ammunition handover management method according to claim 1, characterized in that: Also includes: Data storage and backup: Throughout the entire handover process, the system must store all operating steps, personnel information, actual magazine weight, number of bullets, and video surveillance content in real time, and perform regular data backup to ensure data integrity and traceability; Inspection and call: The management personnel check the duty handover situation within a certain period of time, enter the management interface through the administrator account, and call up relevant handover data and video materials.

Images