Intelligent camera target automatic identification and tracking method and system

Through multi-scale hierarchical attention network and Kalman filter tracking algorithm, combined with behavior analysis algorithm, the problems of multi-objective tracking and abnormal behavior recognition in school scenarios are solved, and the automatic target recognition and tracking with high accuracy and stability are achieved in a timely manner. Handle emergencies and ensure students' safety.

CN119007106AInactive Publication Date: 2024-11-22HEBEI BODE CULTURE COMMUNICATION CO LTD
View PDF 0 Cites 10 Cited by

Patent Information

Application Number
CN202411076644.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing target detection and tracking technologies have poor accuracy and stability in school scenarios, and they cannot effectively identify and deal with abnormal behaviors, such as falling, chasing, fighting, etc.

Method used

Multi-scale hierarchical attention network is used to extract and enhance video frame features, combine Kalman filter tracking algorithm to optimize target trajectory information, and use behavior analysis algorithm to identify abnormal behaviors and generate alarm information.

Benefits of technology

It improves the accuracy of target detection and identification, enhances the stability of multi-target tracking, can identify and handle emergencies in a timely manner, and ensures the safety of students.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007106A_ABST
    Figure CN119007106A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent camera target automatic identification and tracking method and system which are applied to automatic identification and tracking with students as targets in a school scene. According to the method, video streams of student activity scenes are collected in real time in a monitoring area, and dynamic changes of the monitoring scenes are captured at a preset frame rate; performing feature extraction and enhancement on the video frame by using a multi-scale hierarchical attention network to generate a final feature map containing multi-scale information; tracking the salient target, constructing a salient target tracking loss calculation model, predicting and updating the position of the salient target through a Kalman filter tracking algorithm, and optimizing the trajectory information of the salient target; on the basis of the trajectory information of the significant target and in combination with school scene requirements, behaviors of students are analyzed by using a behavior analysis algorithm, abnormal behaviors are identified, and corresponding alarm information is generated; according to the method, the accuracy and stability of target detection and tracking can be improved, abnormal behaviors are identified and processed in time, and the safety of students is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an intelligent camera target automatic recognition and tracking method and system. Background Art

[0002] With the rapid development of artificial intelligence and computer vision technology, intelligent monitoring systems have been widely used in various fields, especially in public places such as schools. Traditional monitoring systems mainly rely on manual monitoring, which not only has a large workload, but is also prone to omissions and cannot detect and handle emergencies in a timely manner. In recent years, target detection and tracking technology based on deep learning has gradually become a research hotspot, which can automatically identify and track targets in monitoring videos and improve monitoring efficiency and accuracy.

[0003] However, the existing target detection and tracking technologies still have some shortcomings when applied to school scenarios: in school scenarios, there are a large number of students and they are active frequently, so targets are easily blocked from each other, resulting in poor accuracy and stability in multi-target tracking; traditional target detection methods often only consider features of a single scale during feature extraction and cannot make full use of multi-scale information, resulting in unsatisfactory target recognition and tracking effects; most existing monitoring systems can only perform simple target detection and tracking, and cannot conduct in-depth analysis of the target's behavior, making it difficult to promptly identify and handle abnormal behaviors such as falling, chasing, and fighting.

[0004] Based on the above problems, the present invention proposes an intelligent camera target automatic recognition and tracking method and system. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent camera target automatic identification and tracking method and system, which is applied in school scenarios, automatically identifies and tracks students as targets, uses behavior analysis algorithms to analyze students' behaviors, identifies abnormal behaviors and generates corresponding alarm information.

[0006] The specific technical solutions are as follows:

[0007] In a first aspect, the present invention provides an intelligent camera target automatic recognition and tracking method, the method is applied to automatic recognition and tracking of students as targets in a school scene, the method comprising the following steps:

[0008] Step S1, collect the video stream of the student activity scene in real time in the monitoring area, capture the dynamic changes of the monitoring scene at a predetermined frame rate, and divide the video stream into initial samples consisting of video frame sequences.

[0009] Step S2, using a multi-scale hierarchical attention network to extract and enhance features of the video frames in the initial sample; performing multi-scale feature extraction on the video frames to generate feature maps of different scale levels, and applying an attention mechanism at each scale level to enhance the feature representation of salient targets in the feature map by element-by-element multiplication, suppressing interference from non-salient areas, and processing the feature maps of different scale levels by weighted calculation to form a final feature map containing multi-scale information.

[0010] Step S3, tracking the salient target, building a salient target tracking loss calculation model, calculating the position change of the salient target in consecutive video frames, using the Kalman filter tracking algorithm, predicting and updating the position of the salient target, and optimizing the trajectory information of the salient target.

[0011] Step S4, based on the trajectory information of the significant target and combined with the school scenario requirements, the behavior of the students is analyzed using the behavior analysis algorithm to identify abnormal behavior and generate corresponding alarm information.

[0012] Furthermore, in step S2, multi-scale feature extraction is performed on the video frame to generate feature maps of different scale levels. Specifically, feature extraction is performed on the input video frame x through a convolutional neural network to generate feature maps of different scale levels:

[0013] x i =f i (x);

[0014] x={I t |t=1,2,…,T};

[0015] Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, and T is the total number of video frames.

[0016] Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ):

[0017] Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ;

[0018] Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, bi is the bias, and ⊙ represents element-by-element multiplication.

[0019] The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information:

[0020]

[0021] Among them, w i is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

[0022] Furthermore, in the step S3, in the significant target tracking loss calculation model, the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W k is the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q.

[0023] The covariance matrix P of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R.

[0024] Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

[0025] Furthermore, the salient target tracking loss calculation model performs the following steps:

[0026] Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k .

[0027] Δz k =z k -Hx k -1;

[0028] Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k- 1+K k Δz k , get the predicted position of the salient target in the current frame

[0029] Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

[0030] Furthermore, in step S4, in combination with school scenario requirements, a behavior analysis algorithm is used to analyze the student's behavior and identify abnormal behavior. The school scenario requirement is the identification of abnormal student behavior; the abnormal behavior includes at least: falling, chasing, and fighting.

[0031] The identification of abnormal student behavior includes: extracting student motion features and posture features, extracting motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing pixel changes between consecutive frames.

[0032] u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 );

[0033] Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 They represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function.

[0034] Furthermore, the posture estimation model is used to extract posture features, P = PoseEstimation (I t ), where PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees.

[0035] The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories.

[0036] Furthermore, combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

[0037] In a second aspect, the present invention provides an intelligent camera target automatic recognition and tracking system, which is applied to automatic recognition and tracking of students in school scenarios. The system includes: a video acquisition module, a HAN module, a target trajectory optimization module and an abnormal behavior analysis module connected in sequence.

[0038] Furthermore, the video acquisition module is used to collect video streams of student activity scenes in real time within the monitoring area, capture dynamic changes of the monitoring scene at a predetermined frame rate, and divide the video streams into initial samples consisting of video frame sequences.

[0039] Furthermore, the HAN module is used to extract and enhance features of the video frames in the initial sample using a multi-scale hierarchical attention network; perform multi-scale feature extraction on the video frames to generate feature maps of different scale levels, and at each scale level, apply an attention mechanism to enhance the feature representation of salient targets in the feature map by element-by-element multiplication, suppress interference from non-salient areas, and process the feature maps of different scale levels through weighted processing to form a final feature map containing multi-scale information.

[0040] Furthermore, the target trajectory optimization module is used to track the significant target, construct a significant target tracking loss calculation model, calculate the position change of the significant target in continuous video frames, use the Kalman filter tracking algorithm to predict and update the position of the significant target, and optimize the trajectory information of the significant target.

[0041] Furthermore, the abnormal behavior analysis module is used to analyze the student's behavior based on the trajectory information of the significant target and in combination with the school scenario requirements, using a behavior analysis algorithm to identify abnormal behavior and generate corresponding alarm information.

[0042] Furthermore, the HAN module performs multi-scale feature extraction on the video frame to generate feature maps of different scale levels. Specifically, the HAN module performs feature extraction on the input video frame x through a convolutional neural network to generate feature maps of different scale levels:

[0043] x i=f i (x);

[0044] x={I t |t=1,2,…,T};

[0045] Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, and T is the total number of video frames.

[0046] Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ):

[0047] Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ;

[0048] Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, b i is the bias, and ⊙ represents element-by-element multiplication.

[0049] The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information:

[0050]

[0051] Among them, w i is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

[0052] Furthermore, the target trajectory optimization module includes a significant target tracking loss calculation model, in which the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W kis the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q.

[0053] The covariance matrix P of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R.

[0054] Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

[0055] Furthermore, the salient target tracking loss calculation model performs the following steps:

[0056] Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k ;

[0057] Δz k =z k -Hx k -1;

[0058] Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k -1+K k Δz k , get the predicted position of the salient target in the current frame

[0059] Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

[0060] Furthermore, the abnormal behavior analysis module analyzes the student's behavior using a behavior analysis algorithm in combination with school scenario requirements to identify abnormal behaviors. The school scenario requirements are the identification of abnormal student behaviors, and the abnormal behaviors include at least falling, chasing, and fighting.

[0061] The identification of abnormal behavior of students includes: extracting the motion features and posture features of students, extracting the motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing the pixel changes between consecutive frames:

[0062] u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 );

[0063] Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 They represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function.

[0064] Use the pose estimation model to extract pose features, P = PoseEstimation (I t ); PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees;

[0065] The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories.

[0066] Furthermore, combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] 1. The present invention extracts and enhances features of video frames through a multi-scale hierarchical attention network, generates a final feature map containing multi-scale information, and improves the accuracy of target detection and recognition; uses a Kalman filter tracking algorithm to predict and update the positions of significant targets, optimizes the trajectory information of significant targets, and improves the stability and accuracy of multi-target tracking.

[0069] 2. The present invention combines school scenario requirements and uses a behavioral analysis algorithm to analyze student behaviors. It can promptly identify abnormal behaviors, such as falling, chasing, fighting, etc., and generate corresponding alarm information, thereby improving the practicality and safety of the monitoring system. The method of the present invention can process monitoring videos in real time, promptly discover and handle emergencies, and ensure the safety of students. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The present invention is a flow chart of an intelligent camera target automatic recognition and tracking method.

[0071] Figure 2 The present invention is a schematic diagram of the composition of an intelligent camera target automatic recognition and tracking system. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Example 1

[0074] like Figure 1 As shown, an intelligent camera target automatic recognition and tracking method of the present invention is applied to automatic recognition and tracking of students in school scenes. The method aims to realize automatic recognition and tracking of students in school scenes through a multi-scale hierarchical attention network and a Kalman filter tracking algorithm, and to identify abnormal behaviors through a behavior analysis algorithm to generate alarm information.

[0075] The method comprises the following steps:

[0076] Step S1, collect the video stream of the student activity scene in real time in the monitoring area, capture the dynamic changes of the monitoring scene at a predetermined frame rate, and divide the video stream into initial samples consisting of video frame sequences.

[0077] It should be noted that the predetermined frame rate can be adjusted according to the needs of the specific scene, such as 30 frames per second (FPS); in order to ensure that the dynamic changes in the student activity scene can be fully and accurately captured, the system first needs to deploy high-definition cameras in the monitoring area and configure them to collect video streams in real time at a predetermined frame rate; the selection of frame rate is based on the specific needs of the monitoring scene, such as when it is necessary to capture the details of fast-moving objects, a higher frame rate may be selected; the collected video stream is then segmented into a continuous sequence of video frames to form an initial sample set. These video frames not only contain the visual information in the scene, but also imply temporal continuity, providing basic data for subsequent feature extraction and salient target tracking.

[0078] In addition, in order to improve the efficiency and accuracy of subsequent processing, the video frames can also be preprocessed, including denoising, image enhancement (such as contrast adjustment, brightness equalization), and possible image size standardization. These preprocessing steps help reduce noise interference, enhance image quality, and make the feature extraction process more robust.

[0079] Step S2, using a multi-scale hierarchical attention network to extract and enhance features of the video frames in the initial sample; performing multi-scale feature extraction on the video frames to generate feature maps of different scale levels, and applying an attention mechanism at each scale level to enhance the feature representation of salient targets in the feature map by element-by-element multiplication, suppressing interference from non-salient areas, and processing the feature maps of different scale levels by weighted calculation to form a final feature map containing multi-scale information.

[0080] In step S2, multi-scale feature extraction is performed on the video frame to generate feature maps of different scale levels. Specifically, feature extraction is performed on the input video frame x through a convolutional neural network to generate feature maps of different scale levels:

[0081] x i =f i (x);

[0082] x={I t |t=1,2,…,T};

[0083] Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, and T is the total number of video frames.

[0084] Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ):

[0085] Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ;

[0086] Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, b i is the bias, and ⊙ represents element-by-element multiplication.

[0087] The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information:

[0088]

[0089] Among them, wi is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

[0090] Through the hierarchical structure of convolutional neural networks (CNN), feature maps at multiple scale levels are extracted from the input video frames. These feature maps not only contain the underlying information of the image, such as texture and edges, but also gradually abstract higher-level semantic information. The feature maps at each scale level correspond to different receptive fields and abstraction levels, which helps to fully understand the image content; at each scale level, an attention mechanism is introduced to enhance the feature representation of salient targets. The attention mechanism focuses more attention on salient targets by calculating the weight of each position in the feature map, thereby enhancing the feature representation of these areas and suppressing the interference of non-salient areas.

[0091] Step S3, tracking the salient target, building a salient target tracking loss calculation model, calculating the position change of the salient target in consecutive video frames, using the Kalman filter tracking algorithm, predicting and updating the position of the salient target, and optimizing the trajectory information of the salient target.

[0092] In the step S3, in the salient target tracking loss calculation model, the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W kis the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q.

[0093] It should be noted that before starting tracking, the state vector (usually including position, velocity, etc.) and covariance matrix of the Kalman filter need to be initialized. These initial values ​​can be determined based on prior knowledge or system settings.

[0094] The covariance matrix P of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R.

[0095] Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

[0096] The salient target tracking loss calculation model performs the following steps:

[0097] Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k .

[0098] Δz k =z k- Hx k -1;

[0099] Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k -1+K k Δz k , get the predicted position of the salient target in the current frame

[0100] Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

[0101] Step S4, based on the trajectory information of the significant target and combined with the school scenario requirements, the behavior of the students is analyzed using the behavior analysis algorithm to identify abnormal behavior and generate corresponding alarm information.

[0102] In step S4, the behavior of students is analyzed using a behavior analysis algorithm in combination with school scenario requirements to identify abnormal behaviors. The school scenario requirements are identification of abnormal behaviors of students; the abnormal behaviors include at least falling, chasing, and fighting.

[0103] It should be noted that when students suddenly fall to the ground while walking or running, the posture characteristics show that key points such as the head, shoulders and knees quickly approach the ground, and the motion characteristics show a large vertical speed change; when students move quickly, the motion characteristics show that the motion vectors of multiple significant targets are in the same direction and at a fast speed, and the posture characteristics show that the students are running; when physical conflicts occur between students, the posture characteristics show rapid and large-scale movements of the arms and legs, and the motion characteristics show that the distance between significant targets changes dramatically.

[0104] The identification of abnormal student behavior includes: extracting student motion features and posture features, extracting motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing pixel changes between consecutive frames.

[0105] u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 );

[0106] Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 They represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function.

[0107] Use the pose estimation model to extract pose features, P = PoseEstimation (I t ), where PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees.

[0108] The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories.

[0109] Combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

[0110] The trajectory information and behavior analysis results of significant targets are stored in the database for subsequent query and analysis. At the same time, the alarm information is transmitted through the network to the monitoring center or the terminal equipment of relevant personnel for timely processing.

[0111] Example 2

[0112] like Figure 2 As shown, it is a schematic diagram of the composition of an intelligent camera target automatic recognition and tracking system of the present invention. The system is applied to automatic recognition and tracking of students in school scenes. The system includes: a video acquisition module, a HAN module, a target trajectory optimization module and an abnormal behavior analysis module connected in sequence.

[0113] The video acquisition module is used to collect video streams of student activity scenes in real time within the monitoring area, capture dynamic changes of the monitoring scene at a predetermined frame rate, and divide the video streams into initial samples consisting of video frame sequences.

[0114] The HAN module is used to extract and enhance the features of the video frames in the initial sample using a multi-scale hierarchical attention network; perform multi-scale feature extraction on the video frames to generate feature maps at different scale levels, and at each scale level, apply an attention mechanism to enhance the feature representation of salient targets in the feature map by element-by-element multiplication, suppress interference from non-salient areas, and process the feature maps at different scale levels through weighted processing to form a final feature map containing multi-scale information.

[0115] The target trajectory optimization module is used to track the salient target, build a salient target tracking loss calculation model, calculate the position change of the salient target in continuous video frames, use the Kalman filter tracking algorithm to predict and update the position of the salient target, and optimize the trajectory information of the salient target.

[0116] The abnormal behavior analysis module is used to analyze students' behaviors based on the trajectory information of significant targets and in combination with school scenario requirements, using a behavior analysis algorithm to identify abnormal behaviors and generate corresponding alarm information.

[0117] The HAN module performs multi-scale feature extraction on the video frame to generate feature maps at different scale levels. Specifically, the HAN module performs feature extraction on the input video frame x through a convolutional neural network to generate feature maps at different scale levels:

[0118] x i =f i (x);

[0119] x={I t |t=1,2,…,T};

[0120] Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, and T is the total number of video frames.

[0121] Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ):

[0122] Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ;

[0123] Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, b i is the bias, and ⊙ represents element-by-element multiplication.

[0124] The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information:

[0125]

[0126] Among them, w i is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

[0127] The target trajectory optimization module includes a significant target tracking loss calculation model. In the significant target tracking loss calculation model, the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W kis the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q.

[0128] The covariance matrix P of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R.

[0129] Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

[0130] The salient target tracking loss calculation model performs the following steps:

[0131] Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k ;

[0132] Δz k =z k -Hx k- 1;

[0133] Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k -1+K k Δz k , get the predicted position of the salient target in the current frame

[0134] Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

[0135] The abnormal behavior analysis module analyzes the student's behavior using a behavior analysis algorithm in combination with school scenario requirements to identify abnormal behaviors. The school scenario requirements are the identification of abnormal student behaviors, and the abnormal behaviors include at least falling, chasing, and fighting.

[0136] The identification of abnormal behavior of students includes: extracting the motion features and posture features of students, extracting the motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing the pixel changes between consecutive frames:

[0137] u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 );

[0138] Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 They represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function.

[0139] Use the pose estimation model to extract pose features, P = PoseEstimation (I t ); PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees;

[0140] The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories.

[0141] Combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

[0142] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent camera target automatic recognition and tracking method, the method is applied to the automatic recognition and tracking of students in school scenes, characterized in that: The method comprises the following steps: Step S1, collecting video streams of student activity scenes in real time in the monitoring area, capturing dynamic changes of the monitoring scene at a predetermined frame rate, and dividing the video streams into initial samples consisting of video frame sequences; Step S2, using a multi-scale hierarchical attention network to extract and enhance features of the video frames in the initial sample; performing multi-scale feature extraction on the video frames to generate feature maps at different scale levels, applying an attention mechanism at each scale level, enhancing the feature representation of salient targets in the feature map by element-by-element multiplication, suppressing interference from non-salient areas, and processing the feature maps at different scale levels by weighted calculation to form a final feature map containing multi-scale information; Step S3, tracking the salient target, building a salient target tracking loss calculation model, calculating the position change of the salient target in consecutive video frames, using the Kalman filter tracking algorithm, predicting and updating the position of the salient target, and optimizing the trajectory information of the salient target; Step S4, based on the trajectory information of the significant target and combined with the school scenario requirements, the behavior of the students is analyzed using the behavior analysis algorithm to identify abnormal behavior and generate corresponding alarm information.

2. The method for automatic target recognition and tracking by an intelligent camera according to claim 1, characterized in that: In step S2, multi-scale feature extraction is performed on the video frame to generate feature maps of different scale levels. Specifically, feature extraction is performed on the input video frame x through a convolutional neural network to generate feature maps of different scale levels: x i =f i (x); x={I t ∣t=1,2,…,T}; Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, T is the total number of video frames; Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ): Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ; Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, b i is the bias, ⊙ represents element-by-element multiplication; The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information: Among them, w i is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

3. The method for automatic target recognition and tracking by an intelligent camera according to claim 2, characterized in that: In the step S3, in the salient target tracking loss calculation model, the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W k is the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q; The covariance matrix O of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R; Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

4. The method for automatic target recognition and tracking by an intelligent camera according to claim 3, characterized in that: The salient target tracking loss calculation model performs the following steps: Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k ; Δz k =z k -Hx k -1; Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k -1+K k Δz k , get the predicted position of the salient target in the current frame Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

5. The method for automatic target recognition and tracking by an intelligent camera according to claim 4, characterized in that: In step S4, the behavior of students is analyzed using a behavior analysis algorithm in combination with school scene requirements to identify abnormal behaviors, wherein the school scene requirements are identification of abnormal behaviors of students; the abnormal behaviors at least include: falling, chasing, and fighting; The identification of abnormal behavior of students includes: extracting the motion features and posture features of students, extracting the motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing the pixel changes between consecutive frames: u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 ); Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 Represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function; Use the pose estimation model to extract pose features, P = PoseEstimation (I t ); PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees; The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories; Combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

6. An intelligent camera target automatic recognition and tracking system, the system is applied to the automatic recognition and tracking of students in school scenes, characterized in that: The system comprises: a video acquisition module, a HAN module, a target trajectory optimization module and an abnormal behavior analysis module connected in sequence; The video acquisition module is used to collect video streams of student activity scenes in real time within the monitoring area, capture dynamic changes of the monitoring scene at a predetermined frame rate, and divide the video streams into initial samples consisting of video frame sequences; The HAN module is used to extract and enhance the features of the video frames in the initial sample using a multi-scale hierarchical attention network; perform multi-scale feature extraction on the video frames to generate feature maps at different scale levels, and at each scale level, apply an attention mechanism to enhance the feature representation of the salient targets in the feature map by element-by-element multiplication, suppress the interference of non-salient areas, and process the feature maps at different scale levels through weighted processing to form a final feature map containing multi-scale information; The target trajectory optimization module is used to track the significant target, build a significant target tracking loss calculation model, calculate the position change of the significant target in continuous video frames, use the Kalman filter tracking algorithm to predict and update the position of the significant target, and optimize the trajectory information of the significant target; The abnormal behavior analysis module is used to analyze students' behaviors based on the trajectory information of significant targets and in combination with school scenario requirements, using a behavior analysis algorithm to identify abnormal behaviors and generate corresponding alarm information.

7. The intelligent camera target automatic recognition and tracking system according to claim 6 is characterized in that: The HAN module performs multi-scale feature extraction on the video frame to generate feature maps at different scale levels. Specifically, the HAN module performs feature extraction on the input video frame x through a convolutional neural network to generate feature maps at different scale levels: x i =f i (x); x={I t ∣t=1,2,…,T}; Among them, f i represents the feature extraction function of the convolutional layer corresponding to the i-th scale level, x i Represents the feature map of the i-th scale level, I t represents the t-th frame image, T is the total number of video frames; Apply attention mechanism to each scale level to enhance the feature representation of salient objects i (x i ): Attention i (x i )=Sigmoid(a i ·x i +b i )⊙x i ; Among them, Sigmoid is the activation function, a i is the weight of the feature map at the i-th scale level, b i is the bias, ⊙ represents element-by-element multiplication; The feature maps of all scale levels are processed by weighted calculation to form the final feature map HAN(x) containing multi-scale information: Among them, w i is the weight of the corresponding scale level, Attention i is the attention mechanism function at the i-th scale level, responsible for extracting the salient features at that scale.

8. The intelligent camera target automatic recognition and tracking system according to claim 7 is characterized in that: The target trajectory optimization module includes a significant target tracking loss calculation model. In the significant target tracking loss calculation model, the state transfer equation in the Kalman filter tracking algorithm is: k =FX k-1 +BU k +W k , where X k is the state vector of the current frame, F is the state transfer matrix, B is the control input matrix, U k is the control vector, W k is the process noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the process noise vector is Q; The covariance matrix O of the predicted state k =FP k-1 F T +Q+W k ; Observation equation: z k =Hx k +v k , where z k is the actual observation vector, H is the observation matrix, v k is the observation noise vector, which satisfies Gaussian distribution and has a mean of zero. The covariance matrix of the observation noise vector is R; Then the Kalman gain K k =K k =P k H T (HP k H T +R) -1 ; Update the state estimation expression: x k =x k +K k (z k -Hx k ).

9. The intelligent camera target automatic recognition and tracking system according to claim 8, characterized in that: The salient target tracking loss calculation model performs the following steps: Step S31, calculate the position difference of the salient target in the current frame and the previous frame, and generate a position error Δz k ; Δz k =z k -Hx k -1; Step S32, using the position error to update the state estimate x of the Kalman filter k '=Fx k -1+K k Δz k , get the predicted position of the salient target in the current frame Step S33, calculate the error between the predicted position and the actual detected position as the tracking loss The trajectory information of salient objects is optimized by minimizing the tracking loss.

10. The intelligent camera target automatic recognition and tracking system according to claim 9, characterized in that: The abnormal behavior analysis module analyzes the student's behavior using a behavior analysis algorithm in combination with school scenario requirements to identify abnormal behaviors. The school scenario requirements are the identification of abnormal behaviors of students, and the abnormal behaviors at least include: falling, chasing, and fighting; The identification of abnormal behavior of students includes: extracting the motion features and posture features of students, extracting the motion features of significant targets by optical flow method, and calculating the motion vector of the target by analyzing the pixel changes between consecutive frames: u(x,y,t),v(x,y,t)=OpticalFlow(I t ,I t-1 ); Among them, u(x,y,t) and v(x,y,t) represent the optical flow vector at the image coordinate (x,y) at the current time t, representing the movement speed in the x direction and y direction respectively; t and I t-1 Represent the image data of the current frame and the previous frame respectively, and OpticalFlow is the optical flow function; Use the pose estimation model to extract pose features, P = PoseEstimation (I t ); PoseEstimation is a posture extraction algorithm, and P is a coordinate set of key points of the human body, including the head, shoulders, elbows, and knees; The extracted motion features and posture features are input into the pre-trained behavior classification model for behavior classification to obtain the classification results: Represents the classification result, which is the probability distribution of various behavior categories; Combined with the school scenario requirements, the threshold θ of the behavior classification model is set to identify abnormal behaviors; if If it is greater than the threshold θ of the set behavior classification model, it is determined that the student currently has abnormal behavior, and the alarm information is transmitted to the monitoring center or the terminal device of relevant personnel through the network.

Citation Information

Cited By

  • AI video low-altitude target identification and real-time tracking method based on deep learning

    CN119723421A

  • Method and system for monitoring position of fire extinguisher in hotel environment based on monitoring video

    CN119850741A

  • Multi-level analysis and monitoring system capable of integrating multiple video streams in real time

    CN119963823A

  • Method for realizing human body acupuncture point tracking by overcoming shielding of mechanical arm

    CN120148771A

  • Target tracking method based on spatial conversion network and convolutional network

    CN120219442A