An image recognition-based video monitoring abnormal behavior recognition method

Abstract

The application discloses a video monitoring abnormal behavior recognition method based on image recognition, which comprises the following steps: step one, acquiring a video monitoring stream and generating a scene anchoring video frame sequence; step two, performing image recognition to obtain a monitoring target recognition result; step three, performing cross-frame association and trajectory binding to obtain a trajectory binding behavior unit; step four, constructing a monitoring behavior relationship tensor; step five, inputting an improved Video Swin Transformer model to obtain an abnormal behavior representation vector; step six, generating an abnormal behavior candidate result; step seven, performing evidence scoring tracking and counter-evidence correction to obtain an evidence checking result; and step eight, outputting an abnormal behavior recognition result and updating a scene behavior sample library. The application adopts a scene anchoring counter-evidence gating mechanism and an improved Video Swin Transformer model to realize stable video monitoring abnormal behavior recognition and false alarm suppression.

Description

Technical Field

[0001] This invention relates to the field of abnormal behavior recognition technology, and in particular to a video surveillance abnormal behavior recognition method based on image recognition. Background Technology

[0002] With the increasing demand for intelligent video surveillance in public security, park management, transportation hubs, and production sites, image recognition-based abnormal behavior identification technology is gradually shifting from manual inspection to automated analysis. Existing methods typically identify behaviors such as falls, fighting, loitering, wrong-way walking, and abnormal gatherings through target detection, human posture estimation, trajectory tracking, or video classification models, which improves the processing efficiency of surveillance footage to some extent. However, existing video surveillance abnormal behavior identification methods still have many shortcomings: On the one hand, some methods mainly rely on single-frame image features or short-time action segments, which are insufficient in jointly expressing changes in personnel posture, target movement trajectory, human-object interaction, and the relationship between personnel and scene areas, making it difficult to accurately distinguish between normal passage, permitted lingering, and genuine abnormal behavior; on the other hand, situations such as personnel occlusion, short-term target loss, sudden changes in lighting, image overlap, and changes in area configuration often occur in surveillance scenarios. Existing methods lack scene anchoring processing that combines camera markings, scene boundaries, passage direction, and acquisition time, which easily leads to misidentification. While some deep learning models can extract spatiotemporal features from videos, they are insufficient in continuously tracking and correcting abnormal evidence, lack a stable chain of evidence verification in their outputs, and the anomaly detection threshold is difficult to update with the normal behavior benchmark in specific monitoring scenarios.

[0003] Therefore, how to provide a method for identifying abnormal behavior in video surveillance based on image recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a video surveillance abnormal behavior identification method based on image recognition. This invention proposes a video surveillance abnormal behavior identification method based on image recognition, which generates a scene-anchored video frame sequence by acquiring video surveillance stream and combining camera identifiers, monitored scene areas and acquisition time. Image recognition is performed on continuous video frames to extract personnel target features, related object features and scene boundary features to form a monitoring target identification result.

[0005] Cross-frame association and trajectory binding are performed on the target identification results to generate trajectory binding behavior units. The monitoring behavior relationship tensor is constructed through pose difference, trajectory encoding, interaction distance calculation and region constraint encoding.

[0006] The Video Swin Transformer abnormal behavior recognition model is improved by inputting the monitoring behavior relationship tensor. Abnormal behavior representation vectors are generated through scene-anchored video block embedding, hierarchical spatiotemporal shift window encoding, behavior relationship enhancement aggregation, and counter-evidence gating anomaly decoding.

[0007] After classification mapping, target localization, time regression, evidence scoring, continuous evidence scoring tracking, and rebuttal correction, the abnormal behavior identification results are output and the scene behavior sample library is updated, thereby improving the accuracy and stability of abnormal behavior identification in video surveillance.

[0008] A method for identifying abnormal behavior in video surveillance based on image recognition according to an embodiment of the present invention includes the following steps: Step 1: Acquire the video surveillance stream and generate a scene-anchored video frame sequence based on the camera identifier, monitored scene area, and acquisition time; Step 2: Perform image recognition on the scene-anchored video frame sequence to extract features of personnel targets, related objects, and scene boundaries to obtain the monitoring target recognition results; Step 3: Perform cross-frame association and trajectory binding processing on the monitoring target identification results to obtain trajectory binding behavior units; Step 4: Perform pose differentiation, trajectory encoding, interaction distance calculation, and region constraint encoding on the trajectory binding behavior unit to construct the monitoring behavior relationship tensor; Step 5: Input the monitoring behavior relationship tensor into the improved Video Swin Transformer abnormal behavior recognition model to obtain the abnormal behavior representation vector. The improved Video Swin Transformer abnormal behavior recognition model includes a scene anchoring video block embedding module, a hierarchical spatiotemporal shift window encoding module, a behavior relationship enhancement aggregation module, and a counter-evidence gating abnormal decoding module. The counter-evidence gating abnormal decoding module embeds a scene anchoring counter-evidence gating mechanism. Step 6: Perform classification mapping, target localization, temporal regression, and evidence scoring on the abnormal behavior representation vector to generate candidate results for abnormal behavior; Step 7: Perform continuous evidence scoring and tracking and rebuttal correction on the candidate results of abnormal behavior to obtain the evidence verification results of abnormal behavior; Step 8: Output the abnormal behavior identification result based on the abnormal behavior evidence verification result, write the abnormal behavior identification result into the scene behavior sample library, and update the normal behavior benchmark and abnormal behavior discrimination threshold in the scene behavior sample library.

[0009] Optionally, step one specifically includes: Read the video access address, camera identifier, and monitoring scene area configuration of the surveillance camera. The monitoring scene area configuration includes the boundary coordinates of the monitoring area, the boundary coordinates of the restricted area, the boundary coordinates of the permitted stay area, and the direction of passage. Configure the effective time and expiration time of the configuration for the monitoring scene area. When a new configuration for the monitoring scene area is written, update the expiration time of the configuration of the previous monitoring scene area to the new effective time. Continuous video frames are extracted from the video surveillance stream according to the sampling frame rate. One second is divided by the sampling frame rate to obtain the sampling interval. The acquisition time is generated based on the sampling interval and frame number, with the starting acquisition time as the reference. The data collection time is compared with the configuration effective time and the configuration expiration time. The monitoring scene area configuration is selected when the data collection time is greater than or equal to the configuration effective time and less than the configuration expiration time. When the data collection time falls into multiple valid time ranges, the monitoring scene area configuration is selected when the configuration effective time is closest to the data collection time. The continuous video frames, camera identifiers, acquisition time, and selected monitoring scene area configurations are bound according to the frame sequence number to generate a scene-anchored video frame sequence.

[0010] Optionally, step two specifically includes: Read the continuous video frames, camera identifier, acquisition time and monitoring scene area configuration in the scene anchored video frame sequence, scale the continuous video frames to the preset input resolution, and record the horizontal scaling factor and vertical scaling factor; Calculate the mean and standard deviation of grayscale values ​​for consecutive scaled video frames. Standardize the pixel grayscale values ​​based on the mean and standard deviation and crop them to a preset grayscale range to obtain standardized video frames. Standardized video frames are input into the target detection network to identify personnel and associated object bounding boxes. Low-confidence bounding boxes are removed according to the detection confidence threshold, and non-maximum suppression is performed on overlapping bounding boxes. Perform pose key point detection on the retained personnel target bounding boxes, extract the coordinates of key points corresponding to the head, shoulders, elbows, hands, hips, knees and feet of the human body, subtract the x-coordinate of the upper left corner of the personnel target bounding box from the x-coordinate of the key points and divide by the width of the personnel target bounding box, and subtract the y-coordinate of the upper left corner of the personnel target bounding box from the y-coordinate of the key points and divide by the height of the personnel target bounding box to obtain the personnel target features; The association relationship is determined based on the center distance and overlap area between the personnel target box and the associated object target box, and the center point coordinates of the associated object target box, the scale of the associated object target box and the association relationship are bound as associated object features; Based on the horizontal and vertical scaling factors, the area boundary coordinates in the monitoring scene area configuration are mapped to the standardized video frame coordinate system, and the passage direction identifier is bound to the mapped area boundary coordinates to generate scene boundary features. By binding the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifiers, and acquisition time according to the frame sequence number, the monitoring target recognition result is obtained.

[0011] Optionally, step three specifically includes: Read the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifier, acquisition time and frame number from the monitoring target recognition results; The target matching cost is generated based on the intersection-union ratio, center point distance and similarity of personnel target features in adjacent frames. Personnel targets with a target matching cost less than the matching threshold are identified as the same personnel target. The same trajectory identifier is assigned to the same person target, and the coordinates of the center point of the person target box, the coordinates of the attitude key points, the features of related objects and the scene boundary features are recorded in ascending order of frame number to form the target behavior trajectory; When a person reappears after the number of consecutive lost frames is less than the loss threshold, the reappearing person will be matched with the person at the end of the target's behavior trajectory, and the trajectory identifier will be continued if the match is successful. When the number of consecutive lost frames reaches the loss threshold, the corresponding target behavior trajectory is terminated, and a new trajectory identifier is assigned to the newly appearing and unmatched personnel target. By binding the target behavior trajectory, trajectory identifier, camera identifier, acquisition time, and number of short-term missing frames, a trajectory-bound behavior unit is obtained.

[0012] Optionally, step four specifically includes: Read the target behavior trajectory, trajectory identifier, coordinates of the center point of the person target box, coordinates of the posture key points, associated object features and scene boundary features from the trajectory binding behavior unit; The attitude keypoint coordinates of adjacent frames under the same trajectory identifier are subtracted by the keypoint number and the absolute value is taken to obtain the attitude difference feature. Subtract the center point coordinates of the personnel target box in adjacent frames to obtain the trajectory displacement vector. Divide the trajectory displacement vector by the acquisition time difference between adjacent frames to obtain the trajectory velocity feature. Generate trajectory direction feature based on the horizontal and vertical components of the trajectory displacement vector. The Euclidean distance is calculated based on the coordinates of the center point of the person's bounding box and the coordinates of the center point of the object in the associated object features. The Euclidean distance is then divided by the length of the diagonal of the person's bounding box to obtain the interaction distance feature. The coordinates of the center point of the personnel target box are compared with the monitoring area boundary, the restricted area boundary, and the permitted stay area boundary in the scene boundary features to generate area constraint features, and a scene grid index is generated based on the grid position of the center point of the personnel target box. The attitude difference features, trajectory velocity features, trajectory direction features, interaction distance features, regional constraint features, and scene grid index are concatenated according to trajectory identifiers and acquisition time to construct a monitoring behavior relationship tensor.

[0013] Optionally, step five specifically includes: The scene anchoring video block embedding module reads the monitoring behavior relationship tensor according to the preset time window, takes the sixteen consecutive tensor segments under the same trajectory identifier as the model input window, divides the model input window into time blocks in four frames, and performs grid aggregation according to the scene grid index to obtain continuous spatiotemporal video blocks. The scene-anchored video block embedding module flattens continuous spatiotemporal video blocks into one-dimensional block vectors, multiplies the one-dimensional block vectors with the projection weight matrix and superimposes the projection bias to obtain video block embedding features, encodes camera identifiers, region identifiers and acquisition time into scene-anchored embedding features, and adds them to the video block embedding features and location encoding features to obtain scene-anchored video block features. The hierarchical spatiotemporal shift window coding module divides the scene anchored video block features into a three-dimensional local spatiotemporal window, generates a query vector, a key vector, and a value vector within the window, multiplies the query vector by the transpose of the key vector and divides it by the square root of the key vector dimension to obtain the window attention weight, and then multiplies it by the value vector to obtain the window coding features. The hierarchical spatiotemporal shift window encoding module moves the starting position of the window along the time, height, and width dimensions in adjacent encoding layers, and performs residual summation and layer normalization on the window encoding features to obtain hierarchical spatiotemporal behavior features; The behavior relationship enhancement aggregation module aligns the hierarchical spatiotemporal behavior features with the monitoring behavior relationship tensor according to trajectory identifier, acquisition time, and scene grid index, and concatenates them by channel. The concatenated features are multiplied by the aggregation weight matrix and processed by the GELU activation function to obtain the mapped behavior features. Then, mean-variance normalization and average pooling are performed to obtain the behavior relationship enhancement features. The counter-evidence gated anomaly decoding module inputs the gated anomaly features generated by the scene-anchored counter-evidence gated mechanism into the anomaly decoding layer, performs global pooling and decoding mapping on the gated anomaly features to obtain target-level anomaly semantic features, and concatenates them with trajectory identifier embedding and acquisition time embedding to obtain anomaly decoding features; The counter-evidence gated anomaly decoding module calculates the mean and standard deviation of the anomaly decoding features in the channel dimension, performs layer normalization based on the mean, standard deviation and stability coefficient, and uses the GELU activation function for nonlinear mapping to obtain the anomaly behavior representation vector.

[0014] Optionally, the scenario-anchored counter-evidence gating mechanism specifically includes: The abnormal evidence branch reads and the behavior relationship enhancement features are aligned with the posture difference features, trajectory velocity features, trajectory direction features, interaction distance features and region constraint features in the monitoring behavior relationship tensor. The read features are multiplied by the abnormal evidence weight matrix and processed by the Sigmoid function to obtain the abnormal evidence weight. The counter-evidence branch determines the allowed stay matching value based on the regional constraint features, the passage consistency value based on the trajectory direction features and the passage direction identifier, the occlusion counter-evidence value based on the number of short-term missing frames of the personnel target box under the same trajectory identifier, and the illumination counter-evidence value based on the grayscale mean difference of adjacent consecutive video frames. The evidence of dissent branch concatenates the allowed stay matching value, passability consistency value, occlusion dissent value, and illumination dissent value. The concatenation result is multiplied by the evidence of dissent weight matrix and processed by the Sigmoid function to obtain the evidence of dissent weight. The scenario-anchored counter-evidence gating mechanism multiplies the result of subtracting the counter-evidence weight from the abnormal evidence weight to obtain the gating abnormal weight, and then multiplies the gating abnormal weight with the behavior relationship enhancement feature to obtain the gating abnormal feature. The counter-evidence gated anomaly decoding module inputs the gated anomaly features into the decoding layer to obtain the anomaly behavior representation vector.

[0015] Optionally, step six specifically includes: Read the abnormal behavior representation vector, input the abnormal behavior representation vector into the classification mapping branch, generate the class confidence through linear mapping and Softmax normalization, and take the abnormal class with the highest class confidence as the candidate abnormal behavior class; The abnormal behavior representation vector is input into the target localization branch, and the target center offset and target box scale offset are output. These are then superimposed on the personnel target boxes under the corresponding trajectory labels to obtain the candidate abnormal target locations. The abnormal behavior representation vector is input into the time regression branch, and the abnormal start offset and abnormal end offset are output. These are then added to the continuous acquisition time range corresponding to the generated abnormal behavior representation vector to obtain the candidate abnormal occurrence time period. The anomaly evidence score is calculated by weighting the category confidence, the overlap between the candidate anomaly target location and the target behavior trajectory, and the number of frames during the candidate anomaly occurrence time period. The result that reaches the candidate screening threshold is taken as the candidate result of the anomaly behavior.

[0016] Optionally, step seven specifically includes: Read the candidate results of abnormal behavior, and extract the abnormal evidence scores from continuous video frames according to the candidate abnormal behavior category, trajectory identifier and candidate abnormal occurrence time period; The evidence tracking window is formed by extending forward and backward a preset number of frames from the time period in which the candidate anomaly occurs, and the average score of the abnormal evidence and the number of consecutive frames that meet the standard are calculated within the evidence tracking window. Divide the number of consecutive compliant frames by the total number of frames in the evidence tracking window to obtain the evidence persistence coefficient, and multiply the evidence persistence coefficient by the mean score of abnormal evidence to obtain the continuous evidence score; The region counter-evidence value is calculated based on the positional relationship between the candidate abnormal target location and the boundary of the allowed stay area; the passage counter-evidence value is calculated based on the consistency between the trajectory direction features and the passage direction markers; the illumination counter-evidence value is calculated based on the grayscale mean difference between adjacent consecutive video frames within the evidence tracking window; and the region counter-evidence value, the passage counter-evidence value, and the illumination counter-evidence value are weighted and summed to obtain the counter-evidence score. When the continuous evidence score reaches the evidence retention threshold and the rebuttal score is lower than the rebuttal threshold, a retention mark is written for the candidate result of abnormal behavior; when the rebuttal score reaches the rebuttal threshold, the abnormal evidence score is reduced according to the rebuttal score, and the reduced abnormal evidence score is recompared with the candidate screening threshold, and a retention mark or a rejection mark is written; when the continuous evidence score is lower than the evidence rejection threshold, a rejection mark is written for the candidate result of abnormal behavior, and the abnormal behavior evidence verification result is obtained.

[0017] Optionally, step eight specifically includes: Read the abnormal behavior evidence verification results, convert the abnormal behavior candidate results with reserved tags into abnormal behavior recognition results, and generate abnormal behavior records according to camera identifiers, trajectory identifiers and collection time; Write abnormal behavior records into the scene behavior sample library, and write the abnormal behavior candidate results with the elimination mark into the misidentified sample set; Based on the normal passage trajectory, normal stay duration and normal population density within a preset period according to the monitored scene area, a normal behavior benchmark is generated; Based on the retention and removal markers in the abnormal behavior evidence verification results, the abnormal evidence score distribution of abnormal behavior records and misidentified sample sets are statistically analyzed respectively. The lowest valid evidence score in the abnormal behavior record is compared with the highest false identification evidence score in the false identification sample set to update the abnormal behavior discrimination threshold. The normal behavior benchmark and the abnormal behavior discrimination threshold are written into the scene behavior sample library.

[0018] The beneficial effects of this invention are: This invention binds camera identification, monitored scene area, and acquisition time to video frame sequence, so that video surveillance data has a clear scene source and time reference before entering the recognition process, reducing recognition instability caused by changes in area configuration and sampling time offset.

[0019] By extracting personnel targets, associated objects, and scene boundaries through image recognition, and forming trajectory-bound behavior units after cross-frame association, personnel posture changes, target movement trajectories, human-object interactions, and the relationship between personnel and scene areas can be uniformly organized into a monitoring behavior relationship tensor, enhancing the behavioral semantic expression required for abnormal behavior recognition.

[0020] The improved Video Swin Transformer abnormal behavior recognition model introduces behavior relationship enhancement aggregation on the basis of hierarchical spatiotemporal shift window encoding, which can simultaneously capture local action changes and cross-frame trajectory changes, and improve the ability to recognize continuous abnormal behaviors such as falls, wandering, reverse movement, and abnormal clustering.

[0021] The scene-anchored counter-evidence gating mechanism further utilizes counter-evidence information such as allowed stay, consistent traffic direction, short-term occlusion, and changes in lighting to suppress and correct abnormal candidate results, thereby reducing the false alarm rate.

[0022] By updating the normal behavior benchmark and abnormal behavior discrimination threshold through the results of abnormal behavior evidence verification, the method can improve its adaptability and continuous identification stability in different monitoring scenarios. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a video surveillance abnormal behavior identification method based on image recognition proposed in this invention; Figure 2 This is a flowchart illustrating the working principle of the improved Video Swin Transformer abnormal behavior recognition model, which is based on image recognition and proposed in this invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0025] refer to Figure 1 and Figure 2 A method for identifying abnormal behavior in video surveillance based on image recognition, comprising the following steps: Step 1: Acquire the video surveillance stream and generate a scene-anchored video frame sequence based on the camera identifier, monitored scene area, and acquisition time; Step 2: Perform image recognition on the scene-anchored video frame sequence to extract features of personnel targets, related objects, and scene boundaries to obtain the monitoring target recognition results; Step 3: Perform cross-frame association and trajectory binding processing on the monitoring target identification results to obtain trajectory binding behavior units; Step 4: Perform pose differentiation, trajectory encoding, interaction distance calculation, and region constraint encoding on the trajectory binding behavior unit to construct the monitoring behavior relationship tensor; Step 5: Input the monitoring behavior relationship tensor into the improved Video Swin Transformer abnormal behavior recognition model to obtain the abnormal behavior representation vector. The improved Video Swin Transformer abnormal behavior recognition model includes a scene anchoring video block embedding module, a hierarchical spatiotemporal shift window encoding module, a behavior relationship enhancement aggregation module, and a counter-evidence gating abnormal decoding module. The counter-evidence gating abnormal decoding module embeds a scene anchoring counter-evidence gating mechanism. Step 6: Perform classification mapping, target localization, temporal regression, and evidence scoring on the abnormal behavior representation vector to generate candidate results for abnormal behavior; Step 7: Perform continuous evidence scoring and tracking and rebuttal correction on the candidate results of abnormal behavior to obtain the evidence verification results of abnormal behavior; Step 8: Output the abnormal behavior identification result based on the abnormal behavior evidence verification result, write the abnormal behavior identification result into the scene behavior sample library, and update the normal behavior benchmark and abnormal behavior discrimination threshold in the scene behavior sample library.

[0026] In this embodiment, step one specifically includes: Read the video access address, camera identifier, and monitoring scene area configuration of the surveillance camera. The monitoring scene area configuration includes the boundary coordinates of the monitoring area, the boundary coordinates of the restricted area, the boundary coordinates of the permitted stay area, and the direction of passage. Configure the effective time and expiration time of the configuration for the monitoring scene area. When a new configuration for the monitoring scene area is written, update the expiration time of the configuration of the previous monitoring scene area to the new effective time. Continuous video frames are extracted from the video surveillance stream according to the sampling frame rate. One second is divided by the sampling frame rate to obtain the sampling interval. The acquisition time is generated based on the sampling interval and frame number, with the starting acquisition time as the reference. The data collection time is compared with the configuration effective time and the configuration expiration time. The monitoring scene area configuration is selected when the data collection time is greater than or equal to the configuration effective time and less than the configuration expiration time. When the data collection time falls into multiple valid time ranges, the monitoring scene area configuration is selected when the configuration effective time is closest to the data collection time. Bind the continuous video frames, camera identifiers, acquisition time, and selected monitoring scene area configurations according to the frame sequence number to generate a scene-anchored video frame sequence; In the specific implementation process, the video access address adopts the RTSP stream address format, the camera identification adopts the "CAM-area number-device serial number" format; the monitoring scene area configuration is stored in JSON format, the area boundary coordinates adopt normalized image coordinates, the horizontal and vertical coordinates are 0 to 1, the passage direction identification adopts a two-dimensional unit direction vector, and the configuration effective time and configuration expiration time adopt millisecond-level timestamps. When writing a new monitoring scene area configuration under the same camera identifier, the expiration time of the previous configuration will be updated to the effective time of the new configuration; when the configuration time ranges overlap, the area configuration with the effective time of the configuration closer to the collection time will be selected. The video surveillance stream is decoded at 25 frames per second, the sampling frame rate is set to 5 frames per second, and the sampling interval is 200 milliseconds; the first frame decoding timestamp is used as the starting acquisition time, and the acquisition time is generated according to the frame sequence number and sampling interval; when the difference between the decoding timestamp and the generated acquisition time exceeds 100 milliseconds, the decoding timestamp is used as the acquisition time and a time correction flag is written. Continuous video frames, camera identifiers, acquisition times, matching scene area configurations, and time correction markers are bound by frame numbers and arranged in ascending order of acquisition time. Through the above processing, each continuous video frame has a definite camera source, time reference, and scene configuration, avoiding inconsistencies in input caused by configuration changes or timestamp offsets.

[0027] In this embodiment, step two specifically involves: Read the continuous video frames, camera identifier, acquisition time and monitoring scene area configuration in the scene anchored video frame sequence, scale the continuous video frames to the preset input resolution, and record the horizontal scaling factor and vertical scaling factor; Calculate the mean and standard deviation of grayscale values ​​for consecutive scaled video frames. Standardize the pixel grayscale values ​​based on the mean and standard deviation and crop them to a preset grayscale range to obtain standardized video frames. Standardized video frames are input into the target detection network to identify personnel and associated object bounding boxes. Low-confidence bounding boxes are removed according to the detection confidence threshold, and non-maximum suppression is performed on overlapping bounding boxes. Perform pose key point detection on the retained personnel target bounding boxes, extract the coordinates of key points corresponding to the head, shoulders, elbows, hands, hips, knees and feet of the human body, subtract the x-coordinate of the upper left corner of the personnel target bounding box from the x-coordinate of the key points and divide by the width of the personnel target bounding box, and subtract the y-coordinate of the upper left corner of the personnel target bounding box from the y-coordinate of the key points and divide by the height of the personnel target bounding box to obtain the personnel target features; The association relationship is determined based on the center distance and overlap area between the personnel target box and the associated object target box, and the center point coordinates of the associated object target box, the scale of the associated object target box and the association relationship are bound as associated object features; Based on the horizontal and vertical scaling factors, the area boundary coordinates in the monitoring scene area configuration are mapped to the standardized video frame coordinate system, and the passage direction identifier is bound to the mapped area boundary coordinates to generate scene boundary features. By binding the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifiers, and acquisition time according to the frame sequence, the monitoring target recognition result is obtained. In the specific implementation process, the input resolution of the standardized video frame is set to 640×360 pixels; bilinear interpolation scaling is performed on continuous video frames, and the ratio of the original image width to 640 is recorded as the horizontal scaling factor, and the ratio of the original image height to 360 is recorded as the vertical scaling factor. Gray-scale normalization adopts the frame-by-frame mean standard deviation method, which subtracts the gray-scale mean from the gray-scale value of each pixel, divides it by the sum of the gray-scale standard deviation and the stability coefficient, and then crops it to the range of -3 to 3. The target detection network's detection confidence threshold is set to 0.45, and the overlap ratio (IoU) threshold for overlapping target boxes is set to 0.5. When the IoU of two target boxes reaches 0.5, the target box with the higher detection confidence is retained, and the other target box is removed. The top-left corner coordinates, bottom-right corner coordinates, center point, and box scale are recorded for both personnel target boxes and associated object target boxes. The coordinates of the pose key points are normalized using the personnel target box as a local reference box. The x-coordinate of the key point is subtracted from the x-coordinate of the upper left corner of the personnel target box and then divided by the width of the personnel target box. The y-coordinate of the key point is subtracted from the y-coordinate of the upper left corner of the personnel target box and then divided by the height of the personnel target box. When the confidence of key point detection is lower than 0.3, the corresponding key point coordinates are set to 0 and a missing marker is written. When the center distance between the person's bounding box and the bounding box of an associated object is less than 1.2 times the diagonal length of the person's bounding box, and the overlapping area accounts for 0.1 times the area of ​​the associated object's bounding box, a relationship between the two is determined. When mapping the region boundary coordinates, the original boundary x-coordinate is divided by the horizontal scaling factor, and the original boundary y-coordinate is divided by the vertical scaling factor. Through the above processing, the person, associated object, and scene boundary are unified to a standardized coordinate system, reducing the deviation caused by differences in size, lighting, and coordinate scale.

[0028] In this embodiment, step three specifically includes: Read the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifier, acquisition time and frame number from the monitoring target recognition results; The target matching cost is generated based on the intersection-union ratio, center point distance and similarity of personnel target features in adjacent frames. Personnel targets with a target matching cost less than the matching threshold are identified as the same personnel target. The same trajectory identifier is assigned to the same person target, and the coordinates of the center point of the person target box, the coordinates of the attitude key points, the features of related objects and the scene boundary features are recorded in ascending order of frame number to form the target behavior trajectory; When a person reappears after the number of consecutive lost frames is less than the loss threshold, the reappearing person will be matched with the person at the end of the target's behavior trajectory, and the trajectory identifier will be continued if the match is successful. When the number of consecutive lost frames reaches the loss threshold, the corresponding target behavior trajectory is terminated, and a new trajectory identifier is assigned to the newly appearing and unmatched personnel target. The target behavior trajectory, trajectory identifier, camera identifier, acquisition time, and number of short-term missing frames are bound together to obtain the trajectory-bound behavior unit; In the specific implementation process, cross-frame association adopts a bidirectional matching method between adjacent frames; the intersection-union ratio of the personnel target boxes, the distance between the center points, and the similarity of the personnel target features are used to calculate the cost of target matching, with weights set to 0.45, 0.35, and 0.20 respectively; the distance between the center points is first divided by the diagonal length of the standardized video frame, and the similarity of the personnel target features is based on cosine similarity; The target matching cost is obtained by weighted summation of the intersection-union ratio (I-minus), normalized center distance, and cosine similarity (I-minus). The matching threshold is set to 0.42. When the target matching cost is less than 0.42 and the displacement of the center point of the target box of the person in the adjacent frame is less than 1.5 times the diagonal length of the target box, the person in the adjacent frame is determined to be the same person. When there are multiple matching objects, the matching object with the smallest target matching cost is selected. The trajectory identifier adopts the encoding format of "camera identifier-date-sequence number"; the target behavior trajectory records the coordinates of the center point of the target frame, the coordinates of the posture key points, the features of the associated objects, the scene boundary features and the acquisition time in ascending order of frame number; when the acquisition time difference between adjacent trajectory points exceeds one second, a trajectory discontinuity marker is written. The threshold for consecutive lost frames is set to five frames. When a person or target reappears after a short period of loss and the target matching cost is less than 0.50, the original trajectory identifier is continued and the number of missing frames is written. When the number of consecutive lost frames reaches five frames, the current target behavior trajectory is terminated, and a new trajectory identifier is assigned to newly appearing and unmatched person or target. Through the above processing, the single-frame recognition result is transformed into a continuous behavior trajectory, providing target support for attitude difference, trajectory encoding, interaction distance, and regional constraint encoding.

[0029] In this embodiment, step four specifically includes: Read the target behavior trajectory, trajectory identifier, coordinates of the center point of the person target box, coordinates of the posture key points, associated object features and scene boundary features from the trajectory binding behavior unit; The attitude keypoint coordinates of adjacent frames under the same trajectory identifier are subtracted by the keypoint number and the absolute value is taken to obtain the attitude difference feature. Subtract the center point coordinates of the personnel target box in adjacent frames to obtain the trajectory displacement vector. Divide the trajectory displacement vector by the acquisition time difference between adjacent frames to obtain the trajectory velocity feature. Generate trajectory direction feature based on the horizontal and vertical components of the trajectory displacement vector. The Euclidean distance is calculated based on the coordinates of the center point of the person's bounding box and the coordinates of the center point of the object in the associated object features. The Euclidean distance is then divided by the length of the diagonal of the person's bounding box to obtain the interaction distance feature. The coordinates of the center point of the personnel target box are compared with the monitoring area boundary, the restricted area boundary, and the permitted stay area boundary in the scene boundary features to generate area constraint features, and a scene grid index is generated based on the grid position of the center point of the personnel target box. The attitude difference features, trajectory velocity features, trajectory direction features, interaction distance features, region constraint features, and scene grid index are concatenated according to trajectory identifier and acquisition time to construct a monitoring behavior relationship tensor; In the specific implementation process, the trajectory binding behavior units are grouped according to trajectory identifiers and arranged in ascending order of acquisition time; the attitude difference takes the same key point number in two adjacent frames as the calculation object, subtracts the key point coordinates of the previous frame from the key point coordinates of the current frame and takes the absolute value to obtain the attitude difference feature; when the key point has a missing marker, the corresponding difference value is set to 0. The trajectory encoding uses the center point coordinates of the personnel target box as the calculation object. The trajectory displacement vector is obtained by subtracting the center point coordinates of the previous frame from the center point coordinates of the current frame. The trajectory displacement vector is divided by the acquisition time difference to obtain the trajectory velocity feature. The trajectory displacement vector is divided by its own magnitude to obtain the trajectory direction feature. When the magnitude is 0, it is set to a zero vector. The interaction distance is calculated using the Euclidean distance between the center point coordinates of the person's bounding box and the center point coordinates of the bounding box of the associated object, and then divided by the diagonal length of the person's bounding box. When there are no associated object features in the same frame, the interaction distance feature is set to 1. The area constraint coding uses the ray casting method to determine whether the center point coordinates of the personnel target box are within the boundaries of the monitored area, the restricted area, and the permitted stay area. If they are within the boundaries, they are recorded as 1; otherwise, they are recorded as 0. The scene grid index uses an 8×8 grid, and the horizontal and vertical grid numbers are determined based on the center point coordinates of the personnel target box. Through the above processing, the trajectory binding behavior unit is converted into a continuous tensor.

[0030] In this embodiment, step five specifically includes: The scene anchoring video block embedding module reads the monitoring behavior relationship tensor according to the preset time window, takes the sixteen consecutive tensor segments under the same trajectory identifier as the model input window, divides the model input window into time blocks in four frames, and performs grid aggregation according to the scene grid index to obtain continuous spatiotemporal video blocks. The scene-anchored video block embedding module flattens continuous spatiotemporal video blocks into one-dimensional block vectors, multiplies the one-dimensional block vectors with the projection weight matrix and superimposes the projection bias to obtain video block embedding features, encodes camera identifiers, region identifiers and acquisition time into scene-anchored embedding features, and adds them to the video block embedding features and location encoding features to obtain scene-anchored video block features. The hierarchical spatiotemporal shift window coding module divides the scene anchored video block features into a three-dimensional local spatiotemporal window, generates a query vector, a key vector, and a value vector within the window, multiplies the query vector by the transpose of the key vector and divides it by the square root of the key vector dimension to obtain the window attention weight, and then multiplies it by the value vector to obtain the window coding features. The hierarchical spatiotemporal shift window encoding module moves the starting position of the window along the time, height, and width dimensions in adjacent encoding layers, and performs residual summation and layer normalization on the window encoding features to obtain hierarchical spatiotemporal behavior features; The behavior relationship enhancement aggregation module aligns the hierarchical spatiotemporal behavior features with the monitoring behavior relationship tensor according to trajectory identifier, acquisition time, and scene grid index, and concatenates them by channel. The concatenated features are multiplied by the aggregation weight matrix and processed by the GELU activation function to obtain the mapped behavior features. Then, mean-variance normalization and average pooling are performed to obtain the behavior relationship enhancement features. The counter-evidence gated anomaly decoding module inputs the gated anomaly features generated by the scene-anchored counter-evidence gated mechanism into the anomaly decoding layer, performs global pooling and decoding mapping on the gated anomaly features to obtain target-level anomaly semantic features, and concatenates them with trajectory identifier embedding and acquisition time embedding to obtain anomaly decoding features; The counter-evidence gated anomaly decoding module calculates the mean and standard deviation of the anomaly decoding features in the channel dimension, performs layer normalization based on the mean, standard deviation and stability coefficient, and uses the GELU activation function for nonlinear mapping to obtain the anomaly behavior representation vector. In the specific implementation process, the model input window length is set to sixteen frames and the sliding step size is set to eight frames; the monitoring behavior relationship tensor is grouped according to the trajectory identifier and tensor fragments are extracted. When the fragment is less than sixteen frames, zeros are padded in the time dimension and written into the zero-padding mask; each time block is aggregated according to the 8×8 scene grid index. When there are multiple targets in the same grid, average pooling is used, and empty grids are filled with zero vectors. After the continuous spatiotemporal video blocks are flattened into one-dimensional block vectors, linear projection is performed using a 128-dimensional projection space. Camera identifiers, monitoring scene area identifiers, and acquisition times are mapped to 128-dimensional scene anchoring embedding features through an embedding table. The position encoding uses three-dimensional position encoding with time block number, horizontal grid number, and vertical grid number. The three types of features are added element by element to form scene anchoring video block features. The hierarchical spatiotemporal shift window coding module sets up a three-layer coding structure, with the size of the three-dimensional local spatiotemporal window set to 2×4×4. Within the window, the scene anchored video block features are linearly mapped into query vectors, key vectors, and value vectors. The query vector is multiplied by the transpose of the key vector and then divided by the square root of the key vector dimension. After Softmax normalization, it is multiplied by the value vector to obtain the window coding features. Adjacent coding layers are shifted by half the window size along the time, height, and width dimensions. The window coding features are then subjected to residual summation, layer normalization, and feedforward mapping to obtain the hierarchical spatiotemporal behavior features. The behavior relationship enhancement aggregation module aligns the hierarchical spatiotemporal behavior features with the monitoring behavior relationship tensor according to trajectory identifiers, acquisition time, and scene grid indexes, and concatenates them along the channel dimension. The aggregation mapping layer uses 256-dimensional hidden channels, multiplies the concatenated features with the aggregation weight matrix, and superimposes the aggregation bias. After processing by the GELU activation function, the mapped behavior features are obtained. The mean and standard deviation of the mapped behavior features are calculated along the channel dimension, and normalization is performed according to the stability coefficient. The normalized features of the same person target within the model input window are averaged to obtain the behavior relationship enhancement features. The counter-evidence gated anomaly decoding module performs global average pooling on the gated anomaly features, multiplies the pooling result with the decoding weight matrix and superimposes the decoding bias to obtain the target-level anomaly semantic features; the trajectory identifier embedding features and the acquisition time embedding features are both set to 128 dimensions and concatenated with the target-level anomaly semantic feature channels; the anomaly decoding features are normalized according to the channel mean, standard deviation and stability coefficient, and then passed through the GELU activation function and a 256-dimensional output mapping layer to obtain the anomaly behavior representation vector; Through the above processing, the monitoring behavior relationship tensor is transformed into an abnormal behavior representation vector that integrates spatiotemporal window association, behavior relationship enhancement, and counter-evidence gating information, so that pose changes, trajectory changes, and scene constraints in continuous video can be uniformly used for abnormal behavior recognition.

[0031] In this embodiment, the scene-anchored counter-evidence gating mechanism is specifically as follows: The abnormal evidence branch reads and the behavior relationship enhancement features are aligned with the posture difference features, trajectory velocity features, trajectory direction features, interaction distance features and region constraint features in the monitoring behavior relationship tensor. The read features are multiplied by the abnormal evidence weight matrix and processed by the Sigmoid function to obtain the abnormal evidence weight. The counter-evidence branch determines the allowed stay matching value based on the regional constraint features, the passage consistency value based on the trajectory direction features and the passage direction identifier, the occlusion counter-evidence value based on the number of short-term missing frames of the personnel target box under the same trajectory identifier, and the illumination counter-evidence value based on the grayscale mean difference of adjacent consecutive video frames. The evidence of dissent branch concatenates the allowed stay matching value, passability consistency value, occlusion dissent value, and illumination dissent value. The concatenation result is multiplied by the evidence of dissent weight matrix and processed by the Sigmoid function to obtain the evidence of dissent weight. The scenario-anchored counter-evidence gating mechanism multiplies the result of subtracting the counter-evidence weight from the abnormal evidence weight to obtain the gating abnormal weight, and then multiplies the gating abnormal weight with the behavior relationship enhancement feature to obtain the gating abnormal feature. The counter-evidence gated anomaly decoding module inputs the gated anomaly features into the decoding layer to obtain the anomaly behavior representation vector; In the specific implementation process, the abnormal evidence branch reading and behavior relationship enhancement features are aligned with the monitoring behavior relationship tensor on trajectory identification, acquisition time and scene grid index; the attitude difference feature is averaged according to the difference of effective key points to obtain the attitude change value; the trajectory velocity feature is subtracted from the average velocity of the previous four frames of the same trajectory identification and the absolute value is taken and then normalized to obtain the velocity change value; the trajectory direction feature is calculated with the cosine distance of the passage direction identification to obtain the direction deviation value; the interaction distance feature is one minus the normalized interaction distance to obtain the interaction proximity value; when the prohibited area value is 1 or the monitoring area value is 0, the area anomaly value is recorded as 1. The attitude change value, velocity change value, direction deviation value, interaction proximity value, and regional anomaly value are concatenated to form the anomaly evidence input vector. The anomaly evidence input vector is multiplied by the anomaly evidence weight matrix and a bias is added. The weights of the five components are set to 0.25, 0.20, 0.20, 0.15, and 0.20, respectively. The matrix operation result is input into the Sigmoid function to obtain the anomaly evidence weights in the range of 0 to 1. The evidence branch for rebuttal reads the region constraint features, trajectory direction features, number of short-term missing frames, and average grayscale values ​​of adjacent consecutive video frames; the allowable stay matching value is recorded as 1 when the center point of the person target box is within the boundary of the allowable stay area, otherwise it is recorded as 0; the cosine similarity between the trajectory direction feature and the travel direction identifier is greater than 0.8, and the travel consistency value is recorded as 1; the occlusion rebuttal value is obtained by dividing the number of short-term missing frames by the loss threshold and cropping it to the range of 0 to 1, and the illumination rebuttal value is obtained by dividing the absolute value of the difference between the average grayscale values ​​of adjacent frames by 255 and cropping it to the range of 0 to 1; The allowed stay matching value, passage consistency value, occlusion counter-evidence value, and illumination counter-evidence value are concatenated to form the counter-evidence input vector; the counter-evidence input vector is multiplied by the counter-evidence weight matrix and a bias is added, with the weights of the four components set to 0.30, 0.25, 0.25, and 0.20 respectively; the result is input into the Sigmoid function to obtain the counter-evidence weights; Subtracting the weight of the counter-evidence from the value 1 yields the counter-evidence suppression coefficient. Multiplying the counter-evidence suppression coefficient by the weight of the abnormal evidence yields the gating anomaly weight. Multiplying the gating anomaly weight by the behavioral relationship enhancement feature element by element yields the gating anomaly feature. Through the above processing, the abnormal evidence and the counter-evidence constraint are combined in the same gating weight, suppressing the risk of misidentification corresponding to allowed stay, consistent passage, short-term occlusion, and changes in illumination.

[0032] In this embodiment, step six specifically includes: Read the abnormal behavior representation vector, input the abnormal behavior representation vector into the classification mapping branch, generate the class confidence through linear mapping and Softmax normalization, and take the abnormal class with the highest class confidence as the candidate abnormal behavior class; The abnormal behavior representation vector is input into the target localization branch, and the target center offset and target box scale offset are output. These are then superimposed on the personnel target boxes under the corresponding trajectory labels to obtain the candidate abnormal target locations. The abnormal behavior representation vector is input into the time regression branch, and the abnormal start offset and abnormal end offset are output. These are then added to the continuous acquisition time range corresponding to the generated abnormal behavior representation vector to obtain the candidate abnormal occurrence time period. The anomaly evidence score is calculated by weighting the category confidence, the overlap between the candidate anomaly target location and the target behavior trajectory, and the number of frames during the candidate anomaly occurrence time period. The result that the anomaly evidence score reaches the candidate screening threshold is taken as the candidate result of the anomaly behavior. In the specific implementation process, the dimension of the abnormal behavior representation vector is set to 256. The classification mapping branch multiplies the abnormal behavior representation vector with the category mapping weight matrix and adds the category bias to obtain the category score vector. The category score vector is input into the Softmax function to generate the category confidence scores corresponding to falling, fighting, loitering, going against the flow, abnormal clustering, and entering the restricted area. When the maximum category confidence score is less than 0.50, the corresponding identification object is marked as a low-confidence candidate object. The target localization branch outputs the horizontal offset of the target center, the vertical offset of the target center, the width offset of the target box, and the height offset of the target box through the linear regression layer. The four offsets are added to the center point coordinates, target box width, and target box height of the personnel target box under the corresponding trajectory marker to obtain the position of the candidate abnormal target. When the position of the candidate abnormal target exceeds the boundary of the monitoring area, the coordinates outside the boundary are clipped to the boundary of the area. The time regression branch outputs the anomaly start offset and the anomaly end offset, with the unit of the two offsets set to the number of frames. The anomaly start offset and the anomaly end offset are added to the continuous acquisition time range corresponding to the anomaly behavior representation vector to obtain the candidate anomaly occurrence time period. When the candidate anomaly occurrence time period is shorter than 3 frames, the number of continuous frames is corrected to 3 frames. The anomaly evidence score is calculated by combining the category confidence, the overlap between the candidate anomaly target location and the target behavior trajectory, and the number of frames in the time period of the candidate anomaly occurrence, with weights set to 0.50, 0.30, and 0.20 respectively. The number of frames in the time period is divided by 16 and cropped to the range of 0 to 1. When the weighted result reaches 0.60, anomaly behavior candidate results are generated. Through the above processing, the anomaly behavior representation vector is converted into a candidate identification object that can be located and scored.

[0033] In this embodiment, step seven specifically includes: Read the candidate results of abnormal behavior, and extract the abnormal evidence scores from continuous video frames according to the candidate abnormal behavior category, trajectory identifier and candidate abnormal occurrence time period; The evidence tracking window is formed by extending forward and backward a preset number of frames from the time period in which the candidate anomaly occurs, and the average score of the abnormal evidence and the number of consecutive frames that meet the standard are calculated within the evidence tracking window. Divide the number of consecutive compliant frames by the total number of frames in the evidence tracking window to obtain the evidence persistence coefficient, and multiply the evidence persistence coefficient by the mean score of abnormal evidence to obtain the continuous evidence score; The region counter-evidence value is calculated based on the positional relationship between the candidate abnormal target location and the boundary of the allowed stay area; the passage counter-evidence value is calculated based on the consistency between the trajectory direction features and the passage direction markers; the illumination counter-evidence value is calculated based on the grayscale mean difference between adjacent consecutive video frames within the evidence tracking window; and the region counter-evidence value, the passage counter-evidence value, and the illumination counter-evidence value are weighted and summed to obtain the counter-evidence score. When the continuous evidence score reaches the evidence retention threshold and the rebuttal score is lower than the rebuttal threshold, a retention mark is written for the candidate result of abnormal behavior; when the rebuttal score reaches the rebuttal threshold, the abnormal evidence score is reduced according to the rebuttal score, and the reduced abnormal evidence score is compared with the candidate screening threshold again, and a retention mark or a rejection mark is written; when the continuous evidence score is lower than the evidence rejection threshold, a rejection mark is written for the candidate result of abnormal behavior, and the abnormal behavior evidence verification result is obtained. In the specific implementation process, the evidence tracking window is established with the candidate anomaly occurrence time period as the center, extending forward by 8 frames and backward by 8 frames; the frame with an anomaly evidence score of 0.60 is recorded as the qualified frame, and the average anomaly evidence score and the number of consecutive qualified frames within the evidence tracking window are counted; the number of consecutive qualified frames is divided by the total number of frames in the evidence tracking window to obtain the evidence persistence coefficient. The continuous evidence score is obtained by multiplying the mean score of abnormal evidence by the evidence persistence coefficient; when there are time correction markers or trajectory discontinuity markers in the evidence tracking window, the continuous evidence score is multiplied by 0.90; the evidence retention threshold is set to 0.55, and the evidence removal threshold is set to 0.35. The region counter-evidence value is determined based on the positional relationship between the candidate abnormal target location and the boundary of the allowed stay area. When the candidate abnormal target location is within the boundary of the allowed stay area, it is recorded as 1. The passage counter-evidence value is determined based on the cosine similarity between the trajectory direction features and the passage direction marker. When the cosine similarity reaches 0.80, it is recorded as 1. The illumination counter-evidence value is obtained by dividing the absolute value of the difference between the mean grayscale values ​​of adjacent consecutive video frames within the evidence tracking window by 255, and then cropping it to the range of 0 to 1. The rebuttal score is obtained by weighted summation of the regional rebuttal value, the passage rebuttal value, and the illumination rebuttal value, with weights set to 0.40, 0.35, and 0.25 respectively, and the rebuttal threshold set to 0.65. When the continuous evidence score reaches 0.55 and the rebuttal score is lower than 0.65, a retention flag is written. When the rebuttal score reaches 0.65, the abnormal evidence score is multiplied by one minus the rebuttal score, and then compared with 0.60 again. When the continuous evidence score is lower than 0.35, a removal flag is written. Through the above processing, the abnormal behavior candidate results are verified by persistence and rebuttal information to form the abnormal behavior evidence verification results.

[0034] In this embodiment, step eight specifically includes: Read the abnormal behavior evidence verification results, convert the abnormal behavior candidate results with reserved tags into abnormal behavior recognition results, and generate abnormal behavior records according to camera identifiers, trajectory identifiers and collection time; Write abnormal behavior records into the scene behavior sample library, and write the abnormal behavior candidate results with the elimination mark into the misidentified sample set; Based on the normal passage trajectory, normal stay duration and normal population density within a preset period according to the monitored scene area, a normal behavior benchmark is generated; Based on the retention and removal markers in the abnormal behavior evidence verification results, the abnormal evidence score distribution of abnormal behavior records and misidentified sample sets are statistically analyzed respectively. The lowest valid evidence score in the abnormal behavior record is compared with the highest misidentified evidence score in the misidentified sample set to update the abnormal behavior discrimination threshold, and the normal behavior benchmark and abnormal behavior discrimination threshold are written into the scene behavior sample library. In the specific implementation process, the candidate results of abnormal behavior with reserved tags are converted into abnormal behavior identification results; the abnormal behavior record adopts the encoding format of "camera identifier-trajectory identifier-collection date-sequence number", and the fields include abnormal behavior category, abnormal target location, abnormal occurrence time period, abnormal evidence score, continuous evidence score and rebuttal evidence score; The scene behavior sample library is stored in partitions according to the monitored scene area. Abnormal behavior records are written to the abnormal sample table, and abnormal behavior candidate results with removal marks are written to the misidentified sample set. When multiple similar abnormal behavior records appear within 30 seconds for the same trajectory identifier, the record with the highest abnormal evidence score is retained, and the remaining records are merged into duplicate record marks. The normal behavior baseline is updated on a 24-hour statistical cycle; for each monitored scene area, the number of normal passage trajectories, the average normal dwell time, the standard deviation of the normal dwell time, and the average normal personnel density are counted; the normal personnel density is obtained by dividing the number of personnel targets in a unit area by the area of ​​the area; when the number of samples in a statistical cycle is less than 100, the normal behavior baseline of the previous statistical cycle is used. The abnormal behavior discrimination threshold is updated based on the abnormal sample table and the misidentified sample set. When the lowest valid evidence score in the abnormal behavior record is higher than the highest misidentified evidence score in the misidentified sample set, the average of the two is used as the new abnormal behavior discrimination threshold. When the lowest valid evidence score is less than or equal to the highest misidentified evidence score, the original abnormal behavior discrimination threshold is weighted with the current average of abnormal evidence scores using weights of 0.80 and 0.20, respectively, to obtain a new abnormal behavior discrimination threshold. Through the above processing, the scene behavior sample library, normal behavior benchmark, and abnormal behavior discrimination threshold are continuously updated as the monitoring scene changes.

[0035] Example 1: To verify the feasibility of the present invention in practice, it was applied to an abnormal behavior recognition scenario in the video surveillance of entrances and exits, waiting areas, and restricted passages of a comprehensive transportation hub. Twelve fixed cameras were deployed in the monitoring area, with a video resolution of 1920×1080 pixels, an original frame rate of 25 frames per second, and a sampling frame rate of 5 frames per second. The monitoring scene area included a normal passage area, a permitted stopping area, a restricted area, and a one-way passage. The abnormal behavior recognition targets included falling, fighting, loitering, going against the flow of traffic, abnormal gathering, and entering restricted areas.

[0036] Video surveillance data from 30 consecutive days was collected as a validation sample. Real abnormal events were labeled according to the results of manual review, including 82 fall incidents, 65 fight incidents, 108 loitering incidents, 94 reverse walking incidents, 76 abnormal gathering incidents, and 89 restricted area intrusion incidents. An additional 1800 video clips, including normal passage, queuing, short-term occlusion, sudden changes in lighting, and overlapping multiple people, were selected as false alarm interference samples. The comparison methods used were a standard 3D convolutional video classification method and a standard Video SwinTransformer video recognition method. The anomaly recognition accuracy, anomaly recall, false alarm rate, location accuracy, and average alarm latency of each method on the same test sample were statistically analyzed.

[0037] During implementation, the video surveillance stream first generates a scene-anchored video frame sequence according to camera identification, monitoring scene area configuration, and acquisition time; then, it extracts personnel target boxes, posture key points, associated object target boxes, and scene boundary features through image recognition, and forms trajectory-bound behavior units through cross-frame association; a monitoring behavior relationship tensor is constructed based on posture difference, trajectory speed, trajectory direction, interaction distance, and area constraints; the monitoring behavior relationship tensor is input into the improved Video SwinTransformer abnormal behavior recognition model to generate abnormal behavior representation vectors; after the abnormal behavior candidate results are verified by continuous evidence scoring tracking and scene anchoring counter-evidence gating mechanism, the abnormal behavior recognition results are output, as shown in Table 1 below.

[0038] Table 1. Comparison of the performance of different methods in identifying abnormal behavior in video surveillance.

[0039] As shown in Table 1, the anomaly recognition accuracy of the method of the present invention reaches 96.7%, which is 8.1 percentage points higher than that of ordinary 3D convolutional video classification methods and 4.3 percentage points higher than that of ordinary Video Swin Transformer video recognition methods. The anomaly recall rate reaches 94.8%, indicating that the method of the present invention has a higher detection capability for continuous abnormal behaviors such as falling, wandering, going against the flow, abnormal gathering, and entering restricted areas. The false alarm rate is reduced to 3.9%, which is 4.7 percentage points lower than that of ordinary Video Swin Transformer video recognition methods, indicating that the scene anchoring counter-evidence gating mechanism can effectively suppress false recognition caused by allowed stopping, normal passage, short-term occlusion, and sudden changes in lighting.

[0040] Anomaly recognition accuracy represents the proportion of correctly identified abnormal behaviors out of all recognition results; anomaly recall represents the proportion of real abnormal behaviors that are successfully identified; false alarm rate represents the proportion of normal behaviors or interfering segments that are incorrectly identified as abnormal behaviors; localization accuracy represents the proportion of candidate abnormal target locations that overlap with manually labeled abnormal target locations to a preset standard; average alarm delay represents the average time interval between the occurrence of an abnormal behavior and the system outputting the abnormal behavior recognition result.

[0041] The above results show that scene-anchored video frame sequences provide a unified spatiotemporal reference for video frames from different cameras and in different areas. The monitoring behavior relationship tensor centrally expresses changes in personnel posture, target movement trajectory, interactions of related objects, and regional constraints. The improved Video Swin Transformer abnormal behavior recognition model further utilizes hierarchical spatiotemporal shift window encoding and behavior relationship enhancement aggregation to extract continuous behavior features. The scene-anchored counter-evidence gating mechanism verifies abnormal candidate results through allowed stop matching, consistent passage, occlusion counter-evidence, and illumination counter-evidence, enabling the method of this invention to achieve high recognition accuracy, low false alarm rate, and short alarm delay in complex monitoring scenarios such as transportation hubs.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying abnormal behavior in video surveillance based on image recognition, characterized in that, Includes the following steps: Step 1: Acquire the video surveillance stream and generate a scene-anchored video frame sequence based on the camera identifier, monitored scene area, and acquisition time; Step 2: Perform image recognition on the scene-anchored video frame sequence to extract features of personnel targets, related objects, and scene boundaries to obtain the monitoring target recognition results; Step 3: Perform cross-frame association and trajectory binding processing on the monitoring target identification results to obtain trajectory binding behavior units; Step 4: Perform pose differentiation, trajectory encoding, interaction distance calculation, and region constraint encoding on the trajectory binding behavior unit to construct the monitoring behavior relationship tensor; Step 5: Input the monitoring behavior relationship tensor into the improved Video Swin Transformer abnormal behavior recognition model to obtain the abnormal behavior representation vector. The improved Video Swin Transformer abnormal behavior recognition model includes a scene anchoring video block embedding module, a hierarchical spatiotemporal shift window encoding module, a behavior relationship enhancement aggregation module, and a counter-evidence gating abnormal decoding module. The counter-evidence gating abnormal decoding module embeds a scene anchoring counter-evidence gating mechanism. Step 6: Perform classification mapping, target localization, temporal regression, and evidence scoring on the abnormal behavior representation vector to generate candidate results for abnormal behavior; Step 7: Perform continuous evidence scoring and tracking and rebuttal correction on the candidate results of abnormal behavior to obtain the evidence verification results of abnormal behavior; Step 8: Output the abnormal behavior identification result based on the abnormal behavior evidence verification result, write the abnormal behavior identification result into the scene behavior sample library, and update the normal behavior benchmark and abnormal behavior discrimination threshold in the scene behavior sample library.

2. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step one specifically involves: Read the video access address, camera identifier, and monitoring scene area configuration of the surveillance camera. The monitoring scene area configuration includes the boundary coordinates of the monitoring area, the boundary coordinates of the restricted area, the boundary coordinates of the permitted stay area, and the direction of passage. Configure the effective time and expiration time of the configuration for the monitoring scene area. When a new configuration for the monitoring scene area is written, update the expiration time of the configuration of the previous monitoring scene area to the new effective time. Continuous video frames are extracted from the video surveillance stream according to the sampling frame rate. One second is divided by the sampling frame rate to obtain the sampling interval. The acquisition time is generated based on the sampling interval and frame number, with the starting acquisition time as the reference. The data collection time is compared with the configuration effective time and the configuration expiration time. The monitoring scene area configuration is selected when the data collection time is greater than or equal to the configuration effective time and less than the configuration expiration time. When the data collection time falls into multiple valid time ranges, the monitoring scene area configuration is selected when the configuration effective time is closest to the data collection time. The continuous video frames, camera identifiers, acquisition time, and selected monitoring scene area configurations are bound according to the frame sequence number to generate a scene-anchored video frame sequence.

3. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step two specifically involves: Read the continuous video frames, camera identifier, acquisition time and monitoring scene area configuration in the scene anchored video frame sequence, scale the continuous video frames to the preset input resolution, and record the horizontal scaling factor and vertical scaling factor; Calculate the mean and standard deviation of grayscale values ​​for consecutive scaled video frames. Standardize the pixel grayscale values ​​based on the mean and standard deviation and crop them to a preset grayscale range to obtain standardized video frames. Standardized video frames are input into the target detection network to identify personnel and associated object bounding boxes. Low-confidence bounding boxes are removed according to the detection confidence threshold, and non-maximum suppression is performed on overlapping bounding boxes. Perform pose key point detection on the retained personnel target bounding boxes, extract the coordinates of key points corresponding to the head, shoulders, elbows, hands, hips, knees and feet of the human body, subtract the x-coordinate of the upper left corner of the personnel target bounding box from the x-coordinate of the key points and divide by the width of the personnel target bounding box, and subtract the y-coordinate of the upper left corner of the personnel target bounding box from the y-coordinate of the key points and divide by the height of the personnel target bounding box to obtain the personnel target features; The association relationship is determined based on the center distance and overlap area between the personnel target box and the associated object target box, and the center point coordinates of the associated object target box, the scale of the associated object target box and the association relationship are bound as associated object features; Based on the horizontal and vertical scaling factors, the area boundary coordinates in the monitoring scene area configuration are mapped to the standardized video frame coordinate system, and the passage direction identifier is bound to the mapped area boundary coordinates to generate scene boundary features. By binding the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifiers, and acquisition time according to the frame sequence number, the monitoring target recognition result is obtained.

4. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step three specifically involves: Read the personnel target bounding box, personnel target features, associated object features, scene boundary features, camera identifier, acquisition time and frame number from the monitoring target recognition results; The target matching cost is generated based on the intersection-union ratio, center point distance and similarity of personnel target features in adjacent frames. Personnel targets with a target matching cost less than the matching threshold are identified as the same personnel target. The same trajectory identifier is assigned to the same person target, and the coordinates of the center point of the person target box, the coordinates of the attitude key points, the features of related objects and the scene boundary features are recorded in ascending order of frame number to form the target behavior trajectory; When a person reappears after the number of consecutive lost frames is less than the loss threshold, the reappearing person will be matched with the person at the end of the target's behavior trajectory, and the trajectory identifier will be continued if the match is successful. When the number of consecutive lost frames reaches the loss threshold, the corresponding target behavior trajectory is terminated, and a new trajectory identifier is assigned to the newly appearing and unmatched personnel target. By binding the target behavior trajectory, trajectory identifier, camera identifier, acquisition time, and number of short-term missing frames, a trajectory-bound behavior unit is obtained.

5. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step four specifically involves: Read the target behavior trajectory, trajectory identifier, coordinates of the center point of the person target box, coordinates of the posture key points, associated object features and scene boundary features from the trajectory binding behavior unit; The attitude keypoint coordinates of adjacent frames under the same trajectory identifier are subtracted by the keypoint number and the absolute value is taken to obtain the attitude difference feature. Subtract the center point coordinates of the personnel target box in adjacent frames to obtain the trajectory displacement vector. Divide the trajectory displacement vector by the acquisition time difference between adjacent frames to obtain the trajectory velocity feature. Generate trajectory direction feature based on the horizontal and vertical components of the trajectory displacement vector. The Euclidean distance is calculated based on the coordinates of the center point of the person's bounding box and the coordinates of the center point of the object in the associated object features. The Euclidean distance is then divided by the length of the diagonal of the person's bounding box to obtain the interaction distance feature. The coordinates of the center point of the personnel target box are compared with the monitoring area boundary, the restricted area boundary, and the permitted stay area boundary in the scene boundary features to generate area constraint features, and a scene grid index is generated based on the grid position of the center point of the personnel target box. The attitude difference features, trajectory velocity features, trajectory direction features, interaction distance features, regional constraint features, and scene grid index are concatenated according to trajectory identifiers and acquisition time to construct a monitoring behavior relationship tensor.

6. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step five specifically involves: The scene anchoring video block embedding module reads the monitoring behavior relationship tensor according to the preset time window, takes the sixteen consecutive tensor segments under the same trajectory identifier as the model input window, divides the model input window into time blocks in four frames, and performs grid aggregation according to the scene grid index to obtain continuous spatiotemporal video blocks. The scene-anchored video block embedding module flattens continuous spatiotemporal video blocks into one-dimensional block vectors, multiplies the one-dimensional block vectors with the projection weight matrix and superimposes the projection bias to obtain video block embedding features, encodes camera identifiers, region identifiers and acquisition time into scene-anchored embedding features, and adds them to the video block embedding features and location encoding features to obtain scene-anchored video block features. The hierarchical spatiotemporal shift window coding module divides the scene anchored video block features into a three-dimensional local spatiotemporal window, generates a query vector, a key vector, and a value vector within the window, multiplies the query vector by the transpose of the key vector and divides it by the square root of the key vector dimension to obtain the window attention weight, and then multiplies it by the value vector to obtain the window coding features. The hierarchical spatiotemporal shift window encoding module moves the starting position of the window along the time, height, and width dimensions in adjacent encoding layers, and performs residual summation and layer normalization on the window encoding features to obtain hierarchical spatiotemporal behavior features; The behavior relationship enhancement aggregation module aligns the hierarchical spatiotemporal behavior features with the monitoring behavior relationship tensor according to trajectory identifier, acquisition time, and scene grid index, and concatenates them by channel. The concatenated features are multiplied by the aggregation weight matrix and processed by the GELU activation function to obtain the mapped behavior features. Then, mean-variance normalization and average pooling are performed to obtain the behavior relationship enhancement features. The counter-evidence gated anomaly decoding module inputs the gated anomaly features generated by the scene-anchored counter-evidence gated mechanism into the anomaly decoding layer, performs global pooling and decoding mapping on the gated anomaly features to obtain target-level anomaly semantic features, and concatenates them with trajectory identifier embedding and acquisition time embedding to obtain anomaly decoding features; The counter-evidence gated anomaly decoding module calculates the mean and standard deviation of the anomaly decoding features in the channel dimension, performs layer normalization based on the mean, standard deviation and stability coefficient, and uses the GELU activation function for nonlinear mapping to obtain the anomaly behavior representation vector.

7. The video surveillance abnormal behavior identification method based on image recognition according to claim 6, characterized in that, The scenario-anchored counter-evidence gating mechanism is specifically as follows: The abnormal evidence branch reads and the behavior relationship enhancement features are aligned with the posture difference features, trajectory velocity features, trajectory direction features, interaction distance features and region constraint features in the monitoring behavior relationship tensor. The read features are multiplied by the abnormal evidence weight matrix and processed by the Sigmoid function to obtain the abnormal evidence weight. The counter-evidence branch determines the allowed stay matching value based on the regional constraint features, the passage consistency value based on the trajectory direction features and the passage direction identifier, the occlusion counter-evidence value based on the number of short-term missing frames of the personnel target box under the same trajectory identifier, and the illumination counter-evidence value based on the grayscale mean difference of adjacent consecutive video frames. The evidence of dissent branch concatenates the allowed stay matching value, passability consistency value, occlusion dissent value, and illumination dissent value. The concatenation result is multiplied by the evidence of dissent weight matrix and processed by the Sigmoid function to obtain the evidence of dissent weight. The scenario-anchored counter-evidence gating mechanism multiplies the result of subtracting the counter-evidence weight from the abnormal evidence weight to obtain the gating abnormal weight, and then multiplies the gating abnormal weight with the behavior relationship enhancement feature to obtain the gating abnormal feature. The counter-evidence gated anomaly decoding module inputs the gated anomaly features into the decoding layer to obtain the anomaly behavior representation vector.

8. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step six specifically involves: Read the abnormal behavior representation vector, input the abnormal behavior representation vector into the classification mapping branch, generate the class confidence through linear mapping and Softmax normalization, and take the abnormal class with the highest class confidence as the candidate abnormal behavior class; The abnormal behavior representation vector is input into the target localization branch, and the target center offset and target box scale offset are output. These are then superimposed on the personnel target boxes under the corresponding trajectory labels to obtain the candidate abnormal target locations. The abnormal behavior representation vector is input into the time regression branch, and the abnormal start offset and abnormal end offset are output. These are then added to the continuous acquisition time range corresponding to the generated abnormal behavior representation vector to obtain the candidate abnormal occurrence time period. The anomaly evidence score is calculated by weighting the category confidence, the overlap between the candidate anomaly target location and the target behavior trajectory, and the number of frames during the candidate anomaly occurrence time period. The result that reaches the candidate screening threshold is taken as the candidate result of the anomaly behavior.

9. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step seven specifically involves: Read the candidate results of abnormal behavior, and extract the abnormal evidence scores from continuous video frames according to the candidate abnormal behavior category, trajectory identifier and candidate abnormal occurrence time period; The evidence tracking window is formed by extending forward and backward a preset number of frames from the time period in which the candidate anomaly occurs, and the average score of the abnormal evidence and the number of consecutive frames that meet the standard are calculated within the evidence tracking window. Divide the number of consecutive compliant frames by the total number of frames in the evidence tracking window to obtain the evidence persistence coefficient, and multiply the evidence persistence coefficient by the mean score of abnormal evidence to obtain the continuous evidence score; The region counter-evidence value is calculated based on the positional relationship between the candidate abnormal target location and the boundary of the allowed stay area; the passage counter-evidence value is calculated based on the consistency between the trajectory direction features and the passage direction markers; the illumination counter-evidence value is calculated based on the grayscale mean difference between adjacent consecutive video frames within the evidence tracking window; and the region counter-evidence value, the passage counter-evidence value, and the illumination counter-evidence value are weighted and summed to obtain the counter-evidence score. When the continuous evidence score reaches the evidence retention threshold and the rebuttal score is lower than the rebuttal threshold, a retention mark is written for the candidate result of abnormal behavior; when the rebuttal score reaches the rebuttal threshold, the abnormal evidence score is reduced according to the rebuttal score, and the reduced abnormal evidence score is recompared with the candidate screening threshold, and a retention mark or a removal mark is written. When the continuous evidence score is lower than the evidence removal threshold, a removal flag is written to the candidate results of abnormal behavior, and the abnormal behavior evidence verification result is obtained.

10. The video surveillance abnormal behavior identification method based on image recognition according to claim 1, characterized in that, Step eight specifically involves: Read the abnormal behavior evidence verification results, convert the abnormal behavior candidate results with reserved tags into abnormal behavior recognition results, and generate abnormal behavior records according to camera identifiers, trajectory identifiers and collection time; Write abnormal behavior records into the scene behavior sample library, and write the abnormal behavior candidate results with the elimination mark into the misidentified sample set; Based on the normal passage trajectory, normal stay duration and normal population density within a preset period according to the monitored scene area, a normal behavior benchmark is generated; Based on the retention and removal markers in the abnormal behavior evidence verification results, the abnormal evidence score distribution of abnormal behavior records and misidentified sample sets are statistically analyzed respectively. The lowest valid evidence score in the abnormal behavior record is compared with the highest false identification evidence score in the false identification sample set to update the abnormal behavior discrimination threshold. The normal behavior benchmark and the abnormal behavior discrimination threshold are written into the scene behavior sample library.

Images