A video intelligent monitoring algorithm architecture system based on behavior feature recognition

Abstract

The application discloses a video intelligent monitoring algorithm architecture system based on behavior feature recognition, comprising: a video acquisition processing module, which is used for acquiring and preprocessing monitoring videos; a time sequence unit construction module, which is used for analyzing displacement vectors according to position changes of target behaviors, and constructing behavior time sequence units; a state configuration generation module, which is used for jointly judging the displacement vectors, marking transition segmentation points, and giving state identification to the displacement vectors; an evolution trend modeling module, which is used for modeling state transition characteristics along a time dimension, judging behavior evolution trends, and generating corresponding state evaluation results; a behavior semantic judgment module, which is used for classifying and judging state feature vectors, and determining behavior semantic categories; and a monitoring result output module, which is used for generating and outputting behavior judgment results. The application can realize continuous time sequence analysis of monitoring videos, and improve the accuracy of complex behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of video recognition technology, and in particular to a video intelligent monitoring algorithm architecture system based on behavioral feature recognition. Background Technology

[0002] With the widespread deployment of video surveillance equipment in scenarios such as public safety, traffic management, and production operation supervision, intelligent analysis technology based on video data has gradually become an important direction for the development of monitoring systems.

[0003] Existing video behavior recognition technologies typically process video frames or short video clips, acquiring behavioral features of the target through methods such as object detection, trajectory extraction, or motion feature calculation, and then determining the behavior type based on a classification model. While these methods can recognize static actions or short-term behaviors to some extent, they often rely solely on local features or instantaneous judgments when faced with complex behaviors that exhibit continuity, stages, and evolutionary characteristics, making it difficult to depict the evolutionary process of the behavior over time.

[0004] In addition, although some existing technologies introduce temporal modeling methods to analyze behavioral sequences, they usually focus on directly modeling the original video features or low-level action features, lacking a structured expression of the internal state changes and state transition relationships of the behavior, resulting in the model's limited ability to characterize the evolution of behavior.

[0005] In the behavior determination stage, existing technologies mostly use single-shot or frame-by-frame discrimination to output behavior categories. The determination results are easily affected by local noise, occlusion or instantaneous anomalies, and lack consistency constraints on the determination results in the time dimension, resulting in jitter in the output results, making it difficult to use directly as system-level behavior determination results.

[0006] Therefore, how to provide a video intelligent surveillance algorithm architecture system based on behavioral feature recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a video intelligent monitoring algorithm architecture system based on behavioral feature recognition. This invention fully utilizes technical means such as video temporal analysis and interval discrimination. By constructing temporal units, generating state configurations, modeling behavioral evolution, and processing consistency in target behaviors in monitoring videos, it describes in detail a structured analysis and judgment method oriented towards behavioral evolution processes. This method has the advantages of high stability in behavioral judgment, strong anti-noise capability, and clear system architecture hierarchy.

[0008] A video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to an embodiment of the present invention includes: The video acquisition and processing module is used to acquire surveillance video and preprocess it to generate standard video sequences. The temporal unit construction module is used to perform temporal correlation analysis on the displacement vectors between adjacent frames based on the positional changes of the target behavior in consecutive frames, and to segment the standard video sequence based on the continuity constraint of the displacement vector in the time dimension to construct behavior temporal units. The state configuration generation module is used to jointly determine the displacement vector for each behavior time-series unit through the behavior change determination function, mark the transition segmentation point, and assign the corresponding state label to the displacement vector based on the transition segmentation point to construct the behavior state configuration. The evolutionary situation modeling module is used to model the state transition characteristics along the time dimension based on the change pattern of state identifiers, and to determine the behavioral evolution situation based on the modeling results, generating the corresponding state evaluation results. The behavior semantic determination module is used to map the behavior evolution situation into a situation feature vector based on the state assessment results, and to use the support vector machine algorithm to classify and determine the behavior semantic category corresponding to the behavior state configuration. The monitoring result output module is used to map and combine fields of the classification judgment results, generate and output the behavior judgment results.

[0009] Optionally, modules can be integrated using the following methods: S1. Acquire surveillance video and preprocess it to generate a standard video sequence; S2. In the standard video sequence, based on the positional changes of the target behavior in consecutive frames, the displacement vectors between adjacent frames are temporally correlated and analyzed. Based on the continuity constraint of the displacement vectors in the time dimension, the standard video sequence is segmented to construct behavior temporal units. S3. For each behavior time-series unit, the displacement vector is jointly determined by the behavior change determination function, the transition segmentation point is marked, and the corresponding state label is assigned to the displacement vector based on the transition segmentation point to construct the behavior state configuration. S4. Based on the changing form of state identifiers in the behavioral state configuration, a state modeling and situation determination algorithm of hidden state sequence is adopted. The state identifiers are used as observable state inputs. The state transition characteristics corresponding to the behavioral state configuration are modeled along the time dimension. The behavioral evolution situation is determined according to the modeling results, and the corresponding state evaluation results are generated. S5. Based on the state assessment results, the behavioral evolution situation is mapped into a situation feature vector, and the support vector machine algorithm is used for classification and judgment to determine the behavioral semantic category corresponding to the behavioral state configuration. S6. Map and combine the fields of the classification results to generate and output the behavior judgment results.

[0010] Optionally, the preprocessing specifically includes: performing time synchronization correction and frame sequence consistency verification on the acquired monitoring video, performing resolution unification and aspect ratio constraint processing on the video frames in the monitoring video, performing brightness and contrast normalization adjustment on the video frames, and suppressing noise components in the video frames.

[0011] Optionally, S2 specifically includes: S21. In a standard video sequence, calculate the displacement vector between adjacent frames based on the position coordinates of the target behavior in consecutive frames; S22. Using displacement vectors as input, a dynamic time warping algorithm is employed to match and calculate the time alignment relationship and change trend between adjacent displacement vectors, obtaining the temporal correlation degree of the displacement vectors, specifically including: Based on the displacement vectors corresponding to adjacent frames, the difference between the displacement vectors in the direction component and the magnitude component is calculated as the local matching cost of the displacement change. Introducing dynamic time warping constraints in the time dimension, performing nonlinear time alignment search on the local matching cost, and determining the optimal matching path that satisfies the time monotonicity and continuity constraints; Based on the cumulative matching cost corresponding to the optimal matching path, the temporal consistency of the displacement change process is normalized and mapped to generate temporal correlation degree. S23. Based on temporal correlation, a displacement continuity constraint rule is introduced. When the temporal correlation of adjacent time windows is lower than a preset threshold, the behavior boundary position is marked. S24. The standard video sequence is segmented according to the behavior boundary position, and the segmented video segments are bound to the corresponding displacement vectors to generate behavior time units.

[0012] Optionally, S3 specifically includes: S31. For each line of time sequence unit, select the displacement vector corresponding to the consecutive frames on the time axis, and perform difference operation and direction change operation on the displacement vector according to the preset time span to obtain the displacement change sequence. S32. Based on the displacement change sequence, a behavior change judgment function is introduced to jointly determine the displacement vector in terms of amplitude change rate, direction offset angle, and consistency of change trend. Specifically, this includes: Perform amplitude difference operation on the displacement changes corresponding to adjacent time positions in the displacement change sequence to obtain the amplitude change rate at continuous time positions; The direction angle is calculated for the displacement vectors corresponding to adjacent time positions in the displacement change sequence, and the difference operation is performed on the adjacent direction angles to obtain the direction offset angle at continuous time positions; Within a preset time span, synchronous constraint detection is performed on the rate of change of amplitude and the angle of directional offset, eliminating change items that do not meet the time continuity requirement and retaining change components that meet the consistency condition. The retained amplitude change rate and direction offset angle are normalized and then weighted and fused according to preset weights to generate a sequence of behavioral change intensity at the corresponding time position. S33. Using the sequence of behavioral change intensity as input, the CUSUM algorithm is used to update the cumulative deviation of the temporal evolution of behavioral change intensity. When the cumulative deviation exceeds the preset mutation judgment threshold, the corresponding time position is determined as the transition split point. S34. Using the transition point as the state boundary, the displacement vector is divided into state intervals within the behavior time unit. Based on the stability of the behavior change intensity within each state interval, a consistent state identifier is assigned to the displacement vector within the corresponding interval to construct the behavior state configuration.

[0013] Optionally, S4 specifically includes: S41. Based on the state identifiers arranged in chronological order in the behavioral state configuration, construct a discrete state identifier observation sequence. The identifier observation sequence serves as the observation input of the hidden state sequence model. Each observable state corresponds to at least one potential hidden state. S42. In the hidden state sequence model, set the hidden state space and initial distribution, and construct the probability mapping relationship from the observable state to the hidden state based on the identified observation sequence. S43. Along the time dimension, a state estimation process combining forward and backward recursion is performed on the hidden state sequence model, specifically including: In the hidden state sequence model, a hidden state transition probability is set, which is used to describe the conditional probability relationship of a transition from a first hidden state to a second hidden state at adjacent time positions. At the initial time position, based on the initial state distribution of the hidden states and the observation probability at the corresponding time position, the initial state probability of each hidden state is weighted and initialized to form the initial forward state probability. Proceeding forward in time, at each subsequent time position, the forward state probability of the previous time position is propagated through the hidden state transition probability, and combined with the observation probability of the current time position, the forward state probability of each hidden state is recursively updated. At the termination time position, the backward state probabilities of each hidden state are initialized; Proceeding backwards in time, at each preceding time position, the backward state probability of the next time position is backpropagated through the hidden state transition probability, and combined with the observation probability of the next time position, the backward state probability of each hidden state is recursively updated. At each time point, the corresponding forward state probability and backward state probability are jointly calculated state by state, and the posterior probability distribution of each hidden state is obtained under the probability normalization constraint. Based on the posterior probability distribution of the hidden states at adjacent time positions, the transition contribution between hidden state pairs is accumulated and statistically analyzed, and the transition probability of the hidden states is iteratively re-estimated while maintaining the transition probability and satisfying the preset constraints. S44. Based on the re-estimated hidden state transition probability and the hidden state posterior probability distribution, calculate the optimal state evolution path of the hidden state in the time dimension, and perform joint statistics on the transition frequency, consistency of transition direction and dwell time distribution between adjacent hidden states. S45. Based on statistical results, perform situational quantitative assessment of the evolution process of behavioral state configuration within a complete behavioral time series unit, and generate state assessment results.

[0014] Optionally, the iterative re-estimation of the hidden state transition probability specifically includes: Based on the posterior probability distribution of the hidden states at adjacent time positions, the joint occurrence probability of the hidden states between adjacent time positions is calculated to form the transition contribution set of the hidden state pairs. The cumulative statistics of the transfer contribution set are performed along the time dimension, and the transfer contributions of the same hidden state pair within the complete behavioral time unit are aggregated. Based on the aggregation results, the transition probability parameters in the hidden state transition probability structure are normalized and updated until the outward transition probability corresponding to each hidden state satisfies the preset probability constraint conditions. During the normalization update process, the update magnitude of hidden state transition paths below a preset contribution threshold is limited; Using the updated hidden state transition probability structure as the current iteration result, the steps of transition contribution calculation, cumulative statistics, and probability update are repeated until the preset iteration termination condition is met.

[0015] Optionally, S5 specifically includes: S51. Based on the state assessment results, extract statistics reflecting the distribution of hidden state transition frequency, state dwell time distribution, and consistency of state transition direction, and combine them according to the preset feature splicing rules to form a situation feature vector. S52. The principal component analysis method is used to decompose the situation feature vector into covariance, extract the principal component subspace of the behavior change direction, project the situation feature vector into the principal component subspace, and perform scale uniformity processing. S53. Input the normalized situation feature vector into the support vector machine discriminant function, and calculate the discriminant response value of the situation feature vector relative to the preset behavioral semantic discriminant boundary under the maximum margin constraint, specifically including: Within the discriminant space of the support vector machine, a set of support vectors consisting of several support vectors is determined. Each support vector corresponds to a set of discriminant coefficients, which, together with the discriminant bias term, constitute the discriminant function of the support vector machine. Based on the set of support vectors, an inner product operation is performed between the normalized situation feature vector and each support vector, and the inner product results are weighted and accumulated according to the corresponding discriminant coefficients to form a linear combination result of the discriminant function; A discriminant bias term is introduced into the linear combination result, and the weighted accumulation result is shifted and corrected to obtain the discriminant response value of the situation feature vector in the discriminant space. Under the maximum margin constraint, the margin consistency constraint is applied to the discrimination response value. The sign of the discrimination response value determines the corresponding side of the behavior semantic discrimination boundary of the situation feature vector, and the margin measure relative to the behavior semantic discrimination boundary is determined based on the absolute value of the discrimination response value. S54. Based on the distribution of the discrimination response values ​​on both sides of the discrimination boundary, determine the behavioral semantic category corresponding to the behavioral state configuration.

[0016] Optionally, S6 specifically includes: S61. Based on the behavioral semantic category, according to the preset field mapping rules, the classification judgment result is mapped to a set of behavioral judgment fields that include the behavioral type field, the occurrence time field, and the associated object field. S62. Construct a sliding time window along the time dimension, and collect a set of behavior judgment fields corresponding to multiple consecutive moments within the sliding time window; S63. Based on the collected set of behavior judgment fields, perform weighted consistency judgment on the behavior semantic categories, and calculate the weighted score according to the frequency of occurrence of each behavior semantic category in the sliding time window and the size of the corresponding judgment response value. S64. Based on the calculated weighted score, suppress the semantic categories of behaviors that are below the preset consistency threshold, and retain the semantic categories of behaviors whose weighted scores meet the consistency condition. S65. Based on the behavior semantic category after consistency determination, combine the corresponding behavior determination fields to generate the final behavior determination result and output it.

[0017] The beneficial effects of this invention are: First, this invention introduces a behavior temporal unit construction and behavior state configuration generation mechanism in video surveillance scenarios, which expands the behavior recognition process based on single frames or short time segments into a time-continuous analysis process, so that the changes of target behavior in the time dimension are decomposed into a state sequence with clear boundaries and internal correlations.

[0018] Secondly, this invention explicitly characterizes the transition relationships and evolutionary patterns between internal states of behavior by performing latent state sequence modeling and evolutionary situation assessment on behavioral state configurations. This enables behavior recognition to no longer rely on a single feature or static discrimination, but to conduct a comprehensive analysis based on state transition characteristics and evolutionary trends.

[0019] Finally, this invention introduces consistency constraints and weighted judgment mechanisms in the behavior judgment result output stage to stabilize the classification results in the time dimension, so that the final output behavior judgment results have continuity and reliability, and can directly serve the application needs of actual monitoring and management scenarios. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a module structure diagram of a video intelligent surveillance algorithm architecture system based on behavioral feature recognition proposed in this invention; Figure 2 This is a flowchart of a video intelligent surveillance algorithm architecture system based on behavioral feature recognition proposed in this invention. Figure 3 This is a schematic diagram illustrating the behavioral evolution of a video intelligent surveillance algorithm architecture system based on behavioral feature recognition proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figure 1 A video intelligent surveillance algorithm architecture system based on behavioral feature recognition includes: The video acquisition and processing module is used to acquire surveillance video and preprocess it to generate standard video sequences. The temporal unit construction module is used to perform temporal correlation analysis on the displacement vectors between adjacent frames based on the positional changes of the target behavior in consecutive frames, and to segment the standard video sequence based on the continuity constraint of the displacement vector in the time dimension to construct behavior temporal units. The state configuration generation module is used to jointly determine the displacement vector for each behavior time-series unit through the behavior change determination function, mark the transition segmentation point, and assign the corresponding state label to the displacement vector based on the transition segmentation point to construct the behavior state configuration. The evolutionary situation modeling module is used to model the state transition characteristics along the time dimension based on the change pattern of state identifiers, and to determine the behavioral evolution situation based on the modeling results, generating the corresponding state evaluation results. The behavior semantic determination module is used to map the behavior evolution situation into a situation feature vector based on the state assessment results, and to use the support vector machine algorithm to classify and determine the behavior semantic category corresponding to the behavior state configuration. The monitoring result output module is used to map and combine fields of the classification judgment results, generate and output the behavior judgment results.

[0023] refer to Figure 2-3 In this embodiment, the modules are interconnected using the following method: S1. Acquire surveillance video and preprocess it to generate a standard video sequence; S2. In the standard video sequence, based on the positional changes of the target behavior in consecutive frames, the displacement vectors between adjacent frames are temporally correlated and analyzed. Based on the continuity constraint of the displacement vectors in the time dimension, the standard video sequence is segmented to construct behavior temporal units. S3. For each behavior time-series unit, the displacement vector is jointly determined by the behavior change determination function, the transition segmentation point is marked, and the corresponding state label is assigned to the displacement vector based on the transition segmentation point to construct the behavior state configuration. S4. Based on the changing form of state identifiers in the behavioral state configuration, a state modeling and situation determination algorithm of hidden state sequence is adopted. The state identifiers are used as observable state inputs. The state transition characteristics corresponding to the behavioral state configuration are modeled along the time dimension. The behavioral evolution situation is determined according to the modeling results, and the corresponding state evaluation results are generated. S5. Based on the state assessment results, the behavioral evolution situation is mapped into a situation feature vector, and the support vector machine algorithm is used for classification and judgment to determine the behavioral semantic category corresponding to the behavioral state configuration. S6. Map and combine the fields of the classification results to generate and output the behavior judgment results.

[0024] In this embodiment, the preprocessing specifically includes: performing time synchronization correction and frame sequence consistency verification on the acquired monitoring video, performing resolution unification and proportion constraint processing on the video frames in the monitoring video, performing brightness and contrast normalization adjustment on the video frames, and suppressing noise components in the video frames.

[0025] In this embodiment, S2 specifically includes: S21. In a standard video sequence, calculate the displacement vector between adjacent frames based on the position coordinates of the target behavior in consecutive frames; S22. Using displacement vectors as input, a dynamic time warping algorithm is employed to match and calculate the time alignment relationship and change trend between adjacent displacement vectors, obtaining the temporal correlation degree of the displacement vectors, specifically including: Based on the displacement vectors corresponding to adjacent frames, the difference between the displacement vectors in the direction component and the magnitude component is calculated as the local matching cost of the displacement change. Introducing dynamic time warping constraints in the time dimension, performing nonlinear time alignment search on the local matching cost, and determining the optimal matching path that satisfies the time monotonicity and continuity constraints; Based on the cumulative matching cost corresponding to the optimal matching path, the temporal consistency of the displacement change process is normalized and mapped to generate temporal correlation degree. S23. Based on temporal correlation, a displacement continuity constraint rule is introduced. When the temporal correlation of adjacent time windows is lower than a preset threshold, the behavior boundary position is marked. S24. The standard video sequence is segmented according to the behavior boundary position, and the segmented video segments are bound to the corresponding displacement vectors to generate behavior time units.

[0026] In this embodiment, S3 specifically includes: S31. For each line of time sequence unit, select the displacement vector corresponding to the consecutive frames on the time axis, and perform difference operation and direction change operation on the displacement vector according to the preset time span to obtain the displacement change sequence. S32. Based on the displacement change sequence, a behavior change judgment function is introduced to jointly determine the displacement vector in terms of amplitude change rate, direction offset angle, and consistency of change trend. Specifically, this includes: Perform amplitude difference operation on the displacement changes corresponding to adjacent time positions in the displacement change sequence to obtain the amplitude change rate at continuous time positions; The direction angle is calculated for the displacement vectors corresponding to adjacent time positions in the displacement change sequence, and the difference operation is performed on the adjacent direction angles to obtain the direction offset angle at continuous time positions; Within a preset time span, synchronous constraint detection is performed on the rate of change of amplitude and the angle of directional offset, eliminating change items that do not meet the time continuity requirement and retaining change components that meet the consistency condition. The retained amplitude change rate and direction offset angle are normalized and then weighted and fused according to preset weights to generate a sequence of behavioral change intensity at the corresponding time position. S33. Using the behavior change intensity sequence as input, the CUSUM algorithm is used to update the cumulative deviation of the temporal evolution of the behavior change intensity. When the cumulative deviation exceeds the preset mutation judgment threshold, the corresponding time position is determined as the transition split point, specifically including: Based on the behavioral change intensity sequence, a preset reference level is selected, and the current change intensity at each time position in the behavioral change intensity sequence is compared with the reference level to obtain the instantaneous deviation value at the corresponding time position. The instantaneous deviation value is recursively accumulated and updated along the time sequence. At each time position, the current instantaneous deviation value is superimposed with the cumulative deviation result of the previous time position, and the superposition result is subjected to a lower limit constraint to obtain the cumulative deviation value at the current time position. During the cumulative deviation update process, an upper limit comparison operation is performed on the cumulative deviation value corresponding to each time position. When the cumulative deviation value exceeds the preset mutation judgment threshold for the first time, the corresponding time position is recorded as a candidate transition segmentation point. After recording the candidate transition segmentation points, the cumulative deviation value is reset, and the cumulative deviation is updated for the intensity of behavioral changes at subsequent time positions to identify multiple transition segmentation points within the behavioral time unit. S34. Using the transition point as the state boundary, the displacement vector is divided into state intervals within the behavior time unit. Based on the stability of the behavior change intensity within each state interval, a consistent state identifier is assigned to the displacement vector within the corresponding interval to construct the behavior state configuration.

[0027] In this embodiment, S4 specifically includes: S41. Based on the state identifiers arranged in chronological order in the behavioral state configuration, construct a discrete state identifier observation sequence. The identifier observation sequence serves as the observation input of the hidden state sequence model. Each observable state corresponds to at least one potential hidden state. S42. In the hidden state sequence model, set the hidden state space and initial distribution, and construct the probability mapping relationship from the observable state to the hidden state based on the identified observation sequence. S43. Along the time dimension, a state estimation process combining forward and backward recursion is performed on the hidden state sequence model, specifically including: In the hidden state sequence model, a hidden state transition probability is set, which is used to describe the conditional probability relationship of a transition from a first hidden state to a second hidden state at adjacent time positions. At the initial time position, based on the initial state distribution of the hidden states and the observation probability at the corresponding time position, the initial state probability of each hidden state is weighted and initialized to form the initial forward state probability. Proceeding forward in time, at each subsequent time position, the forward state probability of the previous time position is propagated through the hidden state transition probability, and combined with the observation probability of the current time position, the forward state probability of each hidden state is recursively updated. At the termination time position, the backward state probabilities of each hidden state are initialized; Proceeding backwards in time, at each preceding time position, the backward state probability of the next time position is backpropagated through the hidden state transition probability, and combined with the observation probability of the next time position, the backward state probability of each hidden state is recursively updated. At each time point, the corresponding forward state probability and backward state probability are jointly calculated state by state, and the posterior probability distribution of each hidden state is obtained under the probability normalization constraint. Based on the posterior probability distribution of the hidden states at adjacent time positions, the transition contribution between hidden state pairs is accumulated and statistically analyzed, and the transition probability of the hidden states is iteratively re-estimated while maintaining the transition probability and satisfying the preset constraints. S44. Based on the re-estimated hidden state transition probability and the hidden state posterior probability distribution, calculate the optimal state evolution path of the hidden state in the time dimension, and perform joint statistics on the transition frequency, consistency of transition direction and dwell time distribution between adjacent hidden states. S45. Based on statistical results, perform situational quantitative assessment of the evolution process of behavioral state configuration within a complete behavioral time series unit, and generate state assessment results.

[0028] In this embodiment, the iterative re-estimation of the hidden state transition probability specifically includes: Based on the posterior probability distribution of the hidden states at adjacent time positions, the joint occurrence probability of the hidden states between adjacent time positions is calculated to form the transition contribution set of the hidden state pairs. The cumulative statistics of the transfer contribution set are performed along the time dimension, and the transfer contributions of the same hidden state pair within the complete behavioral time unit are aggregated. Based on the aggregation results, the transition probability parameters in the hidden state transition probability structure are normalized and updated until the outward transition probability corresponding to each hidden state satisfies the preset probability constraint conditions. During the normalization update process, the update magnitude of hidden state transition paths below a preset contribution threshold is limited; Using the updated hidden state transition probability structure as the current iteration result, the steps of transition contribution calculation, cumulative statistics, and probability update are repeated until the preset iteration termination condition is met.

[0029] In this embodiment, S5 specifically includes: S51. Based on the state assessment results, extract statistics reflecting the distribution of hidden state transition frequency, state dwell time distribution, and consistency of state transition direction, and combine them according to the preset feature splicing rules to form a situation feature vector. S52. Principal component analysis is used to decompose the situation feature vector into covariance, extract the principal component subspace of the behavior change direction, project the situation feature vector onto the principal component subspace, and perform scale uniformity processing, specifically including: Within the situation feature vector space, each feature component is centered, and a covariance matrix is ​​constructed based on the correlation structure of the centered feature components. Perform eigenvalue decomposition on the covariance matrix and determine the set of orthogonal basis vectors for spanning the principal component subspace based on the energy contribution relationship corresponding to the eigenvalues; Under the condition of satisfying the preset cumulative contribution rate constraint, a low-dimensional principal component subspace is constructed with the orthogonal basis vector set, and the situation feature vector is orthogonally projected onto the principal component subspace; Apply scaling constraints to the projected situation feature vector along the principal component dimension; S53. Input the normalized situation feature vector into the support vector machine discriminant function, and calculate the discriminant response value of the situation feature vector relative to the preset behavioral semantic discriminant boundary under the maximum margin constraint, specifically including: Within the discriminant space of the support vector machine, a set of support vectors consisting of several support vectors is determined. Each support vector corresponds to a set of discriminant coefficients, which, together with the discriminant bias term, constitute the discriminant function of the support vector machine. Based on the set of support vectors, an inner product operation is performed between the normalized situation feature vector and each support vector, and the inner product results are weighted and accumulated according to the corresponding discriminant coefficients to form a linear combination result of the discriminant function; A discriminant bias term is introduced into the linear combination result, and the weighted accumulation result is shifted and corrected to obtain the discriminant response value of the situation feature vector in the discriminant space. Under the maximum margin constraint, the margin consistency constraint is applied to the discrimination response value. The sign of the discrimination response value determines the corresponding side of the behavior semantic discrimination boundary of the situation feature vector, and the margin measure relative to the behavior semantic discrimination boundary is determined based on the absolute value of the discrimination response value. S54. Based on the distribution of the discrimination response values ​​on both sides of the discrimination boundary, determine the behavioral semantic category corresponding to the behavioral state configuration.

[0030] In this embodiment, S6 specifically includes: S61. Based on the behavioral semantic category, according to the preset field mapping rules, the classification judgment result is mapped to a set of behavioral judgment fields that include the behavioral type field, the occurrence time field, and the associated object field. S62. Construct a sliding time window along the time dimension, and collect a set of behavior judgment fields corresponding to multiple consecutive moments within the sliding time window; S63. Based on the collected set of behavior judgment fields, perform weighted consistency judgment on the behavior semantic categories, and calculate the weighted score according to the frequency of occurrence of each behavior semantic category in the sliding time window and the size of the corresponding judgment response value. S64. Based on the calculated weighted score, suppress the semantic categories of behaviors that are below the preset consistency threshold, and retain the semantic categories of behaviors whose weighted scores meet the consistency condition. S65. Based on the behavior semantic category after consistency determination, combine the corresponding behavior determination fields to generate the final behavior determination result and output it.

[0031] Example 1: To verify the feasibility of this invention in practice, it was applied to a continuous video surveillance scenario. In this scenario, there is continuous activity of personnel or target objects within the monitored area, and the behavior process has obvious temporal continuity, stage changes, and state evolution characteristics. Traditional behavior recognition methods based on single frames or short segments are easily affected by occlusion, instantaneous action changes, and environmental interference in such scenarios, resulting in unstable behavior judgment results and a high misjudgment rate, making it difficult to meet the stability and reliability requirements of actual monitoring applications.

[0032] In this scenario, the monitoring system continuously collects video data and inputs the collected video into the video intelligent monitoring algorithm architecture system of this invention. The system first preprocesses the raw video, unifying the video resolution and frame sequence structure to obtain a standard video sequence. Within the standard video sequence, the system calculates the displacement vector between adjacent frames based on the target's positional changes in consecutive frames. Then, through temporal correlation analysis and continuity constraints, the system segments the video sequence into multiple behavioral temporal units.

[0033] After obtaining the behavioral time sequence units, the system further performs joint analysis on the displacement vectors within each behavioral time sequence unit, introduces a behavioral change judgment function, comprehensively judges the magnitude, direction and trend of displacement changes, identifies the locations where the behavioral state changes significantly, and constructs the behavioral state configuration accordingly.

[0034] Based on this, the system performs hidden state sequence modeling on the behavioral state configuration, recursively estimating and iteratively re-evaluating the state transition characteristics along the time dimension, forming a state transition probability structure that reflects the behavioral evolution law. Subsequently, the system constructs a situation feature vector based on the state evolution evaluation results, and uses principal component analysis to structurally compress the features, allowing key evolutionary features to be centrally expressed in a low-dimensional space.

[0035] During the judgment phase, the system inputs the dimensionality-reduced situation feature vector into the support vector machine discriminant function, calculates the discriminant response value through the maximum margin discriminant mechanism, and obtains the corresponding behavioral semantic category. To further improve the stability of the output results, the system introduces a sliding time window weighted consistency judgment method in the result generation phase. This method performs consistency constraint processing on multiple consecutive judgment results, suppresses the impact of short-term abnormal judgments, and finally generates stable and directly usable behavioral judgment results.

[0036] To verify the feasibility of this invention in practice, it was applied to a continuous video surveillance behavior recognition scenario. The performance of different methods in terms of behavior recognition accuracy, judgment stability, and misjudgment control was compared and analyzed, and the following experimental data were obtained.

[0037] Table 1 Performance Comparison of Different Behavior Recognition Methods

[0038] As can be seen from the data in Table 1, the present invention significantly improves the accuracy of behavior recognition compared with traditional methods. At the same time, it shows significant advantages in continuous judgment consistency rate and behavior stage recognition completeness, and the short-term misjudgment rate is significantly reduced. This fully demonstrates that the present invention can effectively solve the problems of unstable behavior judgment and susceptibility to instantaneous interference in the prior art, and has good practical application value.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A video intelligent surveillance algorithm architecture system based on behavioral feature recognition, characterized in that, include: The video acquisition and processing module is used to acquire surveillance video and preprocess it to generate standard video sequences. The temporal unit construction module is used to perform temporal correlation analysis on the displacement vectors between adjacent frames based on the positional changes of the target behavior in consecutive frames, and to segment the standard video sequence based on the continuity constraint of the displacement vector in the time dimension to construct behavior temporal units. The state configuration generation module is used to jointly determine the displacement vector for each behavior time-series unit through the behavior change determination function, mark the transition segmentation point, and assign the corresponding state label to the displacement vector based on the transition segmentation point to construct the behavior state configuration. The evolutionary situation modeling module is used to model the state transition characteristics along the time dimension based on the change pattern of state identifiers, and to determine the behavioral evolution situation based on the modeling results, generating the corresponding state evaluation results. The behavior semantic determination module is used to map the behavior evolution situation into a situation feature vector based on the state assessment results, and to use the support vector machine algorithm to classify and determine the behavior semantic category corresponding to the behavior state configuration. The monitoring result output module is used to map and combine fields of the classification judgment results, generate and output the behavior judgment results.

2. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 1, characterized in that, The modules are connected in the following way: S1. Acquire surveillance video and preprocess it to generate a standard video sequence; S2. In the standard video sequence, based on the positional changes of the target behavior in consecutive frames, the displacement vectors between adjacent frames are temporally correlated and analyzed. Based on the continuity constraint of the displacement vectors in the time dimension, the standard video sequence is segmented to construct behavior temporal units. S3. For each behavior time-series unit, the displacement vector is jointly determined by the behavior change determination function, the transition segmentation point is marked, and the corresponding state label is assigned to the displacement vector based on the transition segmentation point to construct the behavior state configuration. S4. Based on the changing form of state identifiers in the behavioral state configuration, a state modeling and situation determination algorithm of hidden state sequence is adopted. The state identifiers are used as observable state inputs. The state transition characteristics corresponding to the behavioral state configuration are modeled along the time dimension. The behavioral evolution situation is determined according to the modeling results, and the corresponding state evaluation results are generated. S5. Based on the state assessment results, the behavioral evolution situation is mapped into a situation feature vector, and the support vector machine algorithm is used for classification and judgment to determine the behavioral semantic category corresponding to the behavioral state configuration. S6. Map and combine the fields of the classification results to generate and output the behavior judgment results.

3. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, The preprocessing specifically includes: performing time synchronization correction and frame sequence consistency verification on the acquired monitoring video; performing resolution unification and aspect ratio constraint processing on the video frames in the monitoring video; performing brightness and contrast normalization adjustment on the video frames; and suppressing noise components in the video frames.

4. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, S2 specifically includes: S21. In a standard video sequence, calculate the displacement vector between adjacent frames based on the position coordinates of the target behavior in consecutive frames; S22. Using displacement vectors as input, a dynamic time warping algorithm is employed to match and calculate the time alignment relationship and change trend between adjacent displacement vectors, obtaining the temporal correlation degree of the displacement vectors, specifically including: Based on the displacement vectors corresponding to adjacent frames, the difference between the displacement vectors in the direction component and the magnitude component is calculated as the local matching cost of the displacement change. Introducing dynamic time warping constraints in the time dimension, performing nonlinear time alignment search on the local matching cost, and determining the optimal matching path that satisfies the time monotonicity and continuity constraints; Based on the cumulative matching cost corresponding to the optimal matching path, the temporal consistency of the displacement change process is normalized and mapped to generate temporal correlation degree. S23. Based on temporal correlation, a displacement continuity constraint rule is introduced. When the temporal correlation of adjacent time windows is lower than a preset threshold, the behavior boundary position is marked. S24. The standard video sequence is segmented according to the behavior boundary position, and the segmented video segments are bound to the corresponding displacement vectors to generate behavior time units.

5. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, S3 specifically includes: S31. For each line of time sequence unit, select the displacement vector corresponding to the consecutive frames on the time axis, and perform difference operation and direction change operation on the displacement vector according to the preset time span to obtain the displacement change sequence. S32. Based on the displacement change sequence, a behavior change judgment function is introduced to jointly determine the displacement vector in terms of amplitude change rate, direction offset angle, and consistency of change trend. Specifically, this includes: Perform amplitude difference operation on the displacement changes corresponding to adjacent time positions in the displacement change sequence to obtain the amplitude change rate at continuous time positions; The direction angle is calculated for the displacement vectors corresponding to adjacent time positions in the displacement change sequence, and the difference operation is performed on the adjacent direction angles to obtain the direction offset angle at continuous time positions; Within a preset time span, synchronous constraint detection is performed on the rate of change of amplitude and the angle of directional offset, eliminating change items that do not meet the time continuity requirement and retaining change components that meet the consistency condition. The retained amplitude change rate and direction offset angle are normalized and then weighted and fused according to preset weights to generate a sequence of behavioral change intensity at the corresponding time position. S33. Using the sequence of behavioral change intensity as input, the CUSUM algorithm is used to update the cumulative deviation of the temporal evolution of behavioral change intensity. When the cumulative deviation exceeds the preset mutation judgment threshold, the corresponding time position is determined as the transition split point. S34. Using the transition point as the state boundary, the displacement vector is divided into state intervals within the behavior time unit. Based on the stability of the behavior change intensity within each state interval, a consistent state identifier is assigned to the displacement vector within the corresponding interval to construct the behavior state configuration.

6. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, S4 specifically includes: S41. Based on the state identifiers arranged in chronological order in the behavioral state configuration, construct a discrete state identifier observation sequence. The identifier observation sequence serves as the observation input of the hidden state sequence model. Each observable state corresponds to at least one potential hidden state. S42. In the hidden state sequence model, set the hidden state space and initial distribution, and construct the probability mapping relationship from the observable state to the hidden state based on the identified observation sequence. S43. Along the time dimension, a state estimation process combining forward and backward recursion is performed on the hidden state sequence model, specifically including: In the hidden state sequence model, a hidden state transition probability is set, which is used to describe the conditional probability relationship of a transition from a first hidden state to a second hidden state at adjacent time positions. At the initial time position, based on the initial state distribution of the hidden states and the observation probability at the corresponding time position, the initial state probability of each hidden state is weighted and initialized to form the initial forward state probability. Proceeding forward in time, at each subsequent time position, the forward state probability of the previous time position is propagated through the hidden state transition probability, and combined with the observation probability of the current time position, the forward state probability of each hidden state is recursively updated. At the termination time position, the backward state probabilities of each hidden state are initialized; Proceeding backwards in time, at each preceding time position, the backward state probability of the next time position is backpropagated through the hidden state transition probability, and combined with the observation probability of the next time position, the backward state probability of each hidden state is recursively updated. At each time point, the corresponding forward state probability and backward state probability are jointly calculated state by state, and the posterior probability distribution of each hidden state is obtained under the probability normalization constraint. Based on the posterior probability distribution of the hidden states at adjacent time positions, the transition contribution between hidden state pairs is accumulated and statistically analyzed, and the transition probability of the hidden states is iteratively re-estimated while maintaining the transition probability and satisfying the preset constraints. S44. Based on the re-estimated hidden state transition probability and the hidden state posterior probability distribution, calculate the optimal state evolution path of the hidden state in the time dimension, and perform joint statistics on the transition frequency, consistency of transition direction and dwell time distribution between adjacent hidden states. S45. Based on statistical results, perform situational quantitative assessment of the evolution process of behavioral state configuration within a complete behavioral time series unit, and generate state assessment results.

7. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 6, characterized in that, The iterative re-estimation of the hidden state transition probability specifically includes: Based on the posterior probability distribution of the hidden states at adjacent time positions, the joint occurrence probability of the hidden states between adjacent time positions is calculated to form the transition contribution set of the hidden state pairs. The cumulative statistics of the transfer contribution set are performed along the time dimension, and the transfer contributions of the same hidden state pair within the complete behavioral time unit are aggregated. Based on the aggregation results, the transition probability parameters in the hidden state transition probability structure are normalized and updated until the outward transition probability corresponding to each hidden state satisfies the preset probability constraint conditions. During the normalization update process, the update magnitude of hidden state transition paths below a preset contribution threshold is limited; Using the updated hidden state transition probability structure as the current iteration result, the steps of transition contribution calculation, cumulative statistics, and probability update are repeated until the preset iteration termination condition is met.

8. The video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, S5 specifically includes: S51. Based on the state assessment results, extract statistics reflecting the distribution of hidden state transition frequency, state dwell time distribution, and consistency of state transition direction, and combine them according to the preset feature splicing rules to form a situation feature vector. S52. The principal component analysis method is used to decompose the situation feature vector into covariance, extract the principal component subspace of the behavior change direction, project the situation feature vector into the principal component subspace, and perform scale uniformity processing. S53. Input the normalized situation feature vector into the support vector machine discriminant function, and calculate the discriminant response value of the situation feature vector relative to the preset behavioral semantic discriminant boundary under the maximum margin constraint, specifically including: Within the discriminant space of the support vector machine, a set of support vectors consisting of several support vectors is determined. Each support vector corresponds to a set of discriminant coefficients, which, together with the discriminant bias term, constitute the discriminant function of the support vector machine. Based on the set of support vectors, an inner product operation is performed between the normalized situation feature vector and each support vector, and the inner product results are weighted and accumulated according to the corresponding discriminant coefficients to form a linear combination result of the discriminant function; A discriminant bias term is introduced into the linear combination result, and the weighted accumulation result is shifted and corrected to obtain the discriminant response value of the situation feature vector in the discriminant space. Under the maximum margin constraint, the margin consistency constraint is applied to the discrimination response value. The sign of the discrimination response value determines the corresponding side of the behavior semantic discrimination boundary of the situation feature vector, and the margin measure relative to the behavior semantic discrimination boundary is determined based on the absolute value of the discrimination response value. S54. Based on the distribution of the discrimination response values ​​on both sides of the discrimination boundary, determine the behavioral semantic category corresponding to the behavioral state configuration.

9. A video intelligent surveillance algorithm architecture system based on behavioral feature recognition according to claim 2, characterized in that, S6 specifically includes: S61. Based on the behavioral semantic category, according to the preset field mapping rules, the classification judgment result is mapped to a set of behavioral judgment fields that include the behavioral type field, the occurrence time field, and the associated object field. S62. Construct a sliding time window along the time dimension, and collect a set of behavior judgment fields corresponding to multiple consecutive moments within the sliding time window; S63. Based on the collected set of behavior judgment fields, perform weighted consistency judgment on the behavior semantic categories, and calculate the weighted score according to the frequency of occurrence of each behavior semantic category in the sliding time window and the size of the corresponding judgment response value. S64. Based on the calculated weighted score, suppress the semantic categories of behaviors that are below the preset consistency threshold, and retain the semantic categories of behaviors whose weighted scores meet the consistency condition. S65. Based on the behavior semantic category after consistency determination, combine the corresponding behavior determination fields to generate the final behavior determination result and output it.

Images