A smart behavior analysis system and method for surveillance cameras

Abstract

This invention relates to the field of surveillance behavior analysis technology, and discloses an intelligent behavior analysis system and method for surveillance cameras. The system acquires video stream data through a video data acquisition module, extracts video frame sequences and timestamps, identifies moving targets and their coordinates, associates behavior segments and calculates activity values, generating a set of active target behaviors. A behavior sequence modeling module extracts activity values ​​and coordinates, sorts adjacent behavior segments, and marks consistent and conflicting sections, obtaining a behavior consistency partitioning label set. An environment mapping module acquires target behaviors in consistent sections, extracts illumination and occlusion time series, and assesses the impact of environmental interference on behavior based on behavior type, generating overlay analysis results. An anomaly screening module identifies anomalies with response values ​​greater than the average baseline value and located in conflicting sections, forming a set of behavioral anomaly points. A risk output module acquires anomaly point information, marks points at risk of propagation, and generates behavior detection and risk warning results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of behavior analysis, in particular to an intelligent behavior analysis system and method for monitoring cameras. BACKGROUND

[0002] In the current security monitoring field, monitoring cameras have been widely used in urban traffic, residential areas, commercial places, industrial parks and other types of scenes. Its core function is to realize dynamic monitoring of personnel, vehicles and other targets in a specific area through video data collection, and to provide visual basis for security prevention, event tracing, management and scheduling. However, the functions of existing monitoring systems are often limited to basic video recording and real-time preview, and lack effective deep mining and intelligent analysis capabilities for the target behavior information contained in the video data, making it difficult to meet the needs of precise identification and early warning of potential risks in complex scenarios.

[0003] In the prior art, some monitoring systems with preliminary behavior analysis function have obvious defects in the video data processing link. Such systems can only simply identify the moving targets in the video frames and mark their positions, but cannot effectively associate the behavior segments of the targets in continuous time, and it is even more difficult to quantitatively evaluate the activity level of the target behavior. For example, in the flow monitoring of a commercial complex, the system can only determine that there is personnel flow in a certain area, but cannot distinguish between different active states of behavior such as normal walking, staying and gathering, or disordered shuttle, making it difficult for management personnel to quickly grasp the flow situation in the area.

[0004] In terms of behavior sequence processing, the prior art generally lacks fine analysis of the correlation between adjacent behavior segments. They mostly only arrange the behavior segments in chronological order, ignoring the consistency and conflict characteristics of behaviors in adjacent segments. Taking traffic intersection monitoring as an example, when a vehicle has consecutive behaviors such as lane changing, turning, parking, etc. at the intersection, existing systems cannot accurately mark the consistent section of the vehicle's normal driving and the conflict section such as illegal lane changing, making the identification of traffic violations rely on manual frame-by-frame viewing of the video, which is not only inefficient, but also prone to missed or mistaken judgments due to human negligence.

[0005] The interference of environmental factors on the behavior analysis result is also a bottleneck that the existing monitoring technology is difficult to break through. The change of the strength of natural light, the temporary appearance of the shielding object and other environmental conditions will directly affect the clear identification of the target behavior in the video frame. However, the existing system often does not combine and analyze the time series data of environmental factors such as light and shielding with the type of target behavior, cannot accurately evaluate the influence strength of environmental interference on behavior judgment, and thus leads to deviation of the behavior analysis result. For example, in night monitoring, due to insufficient light, the system may misjudge the normal behavior of a pedestrian bending down to pick up an object as an abnormal behavior, or due to strong light irradiation, the vehicle features are blurred, and the driving track cannot be accurately identified.

[0006] In the abnormal behavior screening and risk early warning link, the limitation of the existing technology is more prominent. Most systems only set a single behavior threshold to determine whether there is an abnormality, for example, define the running of a person as an abnormal behavior, without considering the scene and context association of the behavior, and without making a comprehensive judgment in combination with the consistency or conflict section of the behavior, resulting in extremely low accuracy of abnormality identification. Even if an abnormal behavior is identified, the existing system can only simply mark the abnormal point, and cannot further analyze whether the abnormality has a spread risk. For example, if a crowd gathering abnormality is not timely warned, it may cause a chain of risks such as crowding and stampede, and the existing system is difficult to output a warning result containing spread risk evaluation in advance, so that the proactive prevention ability of the monitoring system is greatly reduced, and the management party cannot gain effective emergency disposal time. SUMMARY

[0007] The purpose of the present application is to provide an intelligent behavior analysis system and method for a monitoring camera to solve the problems raised in the background art.

[0008] To achieve the above-mentioned purpose, the present application provides an intelligent behavior analysis system for a monitoring camera, which comprises:

[0009] A video data acquisition module acquires video stream data of a monitoring camera, extracts a video frame sequence and a timestamp, identifies a moving target and a position coordinate in a video frame, associates a target behavior segment and calculates a corresponding behavior activity value, and generates a target behavior activity set;

[0010] A behavior sequence modeling module extracts the behavior activity value and the corresponding coordinate in the target behavior activity set, sorts adjacent behavior segments in time sequence, marks consistent and conflict sections in adjacent behavior segments, and obtains a behavior consistency partition annotation set;

[0011] An environmental mapping module acquires a target behavior located in a behavior consistent section in the behavior consistency partition annotation set, extracts a light and shielding time series, compares the change amplitude in combination with the behavior type, evaluates the influence strength of behavior under environmental interference, and generates a behavior influence superposition analysis result;

[0012] The anomaly screening module identifies abnormal points in the target behavior that have a behavior response value greater than the average response reference value and are in the behavior conflict section in the behavior influence superposition analysis result, and forms a behavior abnormal point set;

[0013] The risk output module obtains all points in the behavior abnormal point set and corresponding point information, marks points with a spread risk, and generates a behavior detection and risk warning result.

[0014] Preferably, the target behavior active set includes behavior activity values, target space coordinates, and normalized environmental factors, the behavior consistency partition annotation set specifically includes behavior consistent section annotations, behavior conflict section annotations, and adjacent behavior activity value difference rates, the behavior influence superposition analysis result includes illumination change rate on behavior influence degree, shielding change rate on behavior influence degree, and behavior response comparison under each environmental interference condition, the behavior abnormal point set includes abnormal point space positions, abnormal point illumination and shielding amplitude characteristics, and abnormal point behavior types and area fluctuation ratios, and the behavior detection and risk warning result includes a detection abnormal point list and an abnormal point three-index joint determination label.

[0015] Preferably, the video data acquisition module includes:

[0016] The video information acquisition submodule obtains video stream data and timestamps of a monitoring camera, extracts motion target coordinates and behavior time data in a video frame, acquires illumination intensity and shielding degree data corresponding to the coordinate positions, records the acquisition results as two environmental factors of illumination factor and shielding factor, and obtains a target behavior environmental factor data group.

[0017] The environmental factor normalization submodule performs normalization processing on the illumination factor and shielding factor data in the target behavior environmental factor data group respectively, establishes a corresponding relationship between the normalized results and the target coordinate positions, calculates the average values of the normalized illumination values and the normalized shielding values as behavior activity values, and generates a target behavior active set.

[0018] Preferably, the behavior sequence modeling module includes:

[0019] The activity value extraction submodule obtains behavior activity values and corresponding coordinate data in the target behavior active set, identifies the order relationship of all target behaviors in time according to timestamp information, calls a target behavior coordinate set, takes adjacent time thresholds as a reference, measures and sorts the target behaviors in time at intervals, and generates an adjacent target behavior interval sorting sequence.

[0020] The difference rate calculation submodule calculates the behavior activity value difference rate between two adjacent target behaviors based on the adjacent target behavior interval sorting sequence, and integrates to generate a behavior activity value difference rate sequence.

[0021] The consistency identification submodule extracts the light change direction and the occlusion change direction between adjacent target behaviors based on the behavior activity value difference rate sequence, classifies and labels according to whether each pair of target behaviors is consistent in the two direction change trends, records and groups the direction consistent and direction conflict sections respectively, and obtains a behavior consistency partition label set.

[0022] Preferably, the environment mapping module comprises:

[0023] The environment sequence extraction submodule filters the sections marked as behavior consistent according to the behavior consistency partition label set, detects the light data and the occlusion data in each target behavior time period, arranges the light data and the occlusion data in time sequence to form a light time sequence and an occlusion time sequence, and generates an environment change time sequence set;

[0024] The influence superposition calculation submodule calculates the light change rate and the occlusion change rate between consecutive time nodes in each target behavior time sequence based on the environment change time sequence set, compares the light change rate and the occlusion change rate under the condition of the same behavior type, identifies the numerical relationship of the change amplitude of the behavior type through joint analysis of the two types of rate indicators, integrates the influence value sequence of each target behavior, and establishes a behavior influence superposition analysis result.

[0025] Preferably, the anomaly screening module comprises:

[0026] The record extraction submodule filters the target behaviors whose behavior response values are greater than the average response reference value and the target behaviors in the behavior conflict sections according to the behavior influence superposition analysis result, extracts the continuous behavior records of the target behaviors in time sequence, collects the behavior type and the behavior region data corresponding to each time node, and generates a continuous behavior record set;

[0027] The type region ratio calculation submodule calls the continuous behavior record set, extracts the type value and the region of the target behavior at two consecutive time nodes respectively, calculates the type change ratio and the behavior region change ratio respectively, integrates the type change ratio sequence and the behavior region change ratio sequence, and establishes a behavior fluctuation change data set;

[0028] The anomaly identification submodule extracts the light amplitude data and the occlusion fluctuation data corresponding to the time period based on the behavior fluctuation change data set, judges whether the type change ratio and the region change ratio exceed the set fluctuation recognition threshold value, judges whether the light amplitude and the occlusion fluctuation exceed the abnormality judgment threshold value at the same time, marks the time nodes that meet the conditions as abnormal points, and generates a behavior abnormal point set.

[0029] Preferably, the risk output module comprises:

[0030] The index joint determination submodule obtains all points in the behavior anomaly point set and corresponding coordinates and identification information, calculates a joint risk determination value, and establishes a joint risk determination value sequence;

[0031] The abnormal output arrangement submodule filters points with a behavior response value greater than a behavior response risk threshold, a behavior activity value lower than an activity reference value, and a behavior consistency label as a conflict section based on the joint risk determination value sequence, extracts corresponding point numbers, position identifiers, and belonging partitions, marks them as detection anomalies and with diffusion risks, outputs points meeting joint conditions in a structured format, and generates behavior detection and risk warning results.

[0032] Preferably, the system further comprises a behavior knowledge graph construction module that extracts behavior term text based on the target behavior active set, performs word segmentation to extract morpheme indexes, judges the first position and frequency of keywords, maps to generate a morpheme arrangement sequence, and constructs a behavior term original word sequence template.

[0033] The graph path generation module sorts morphemes based on the behavior term original word sequence template, establishes a directed path of morphemes from the top layer to the end layer according to the sorting result, collects the connection node numbers and adjacent node relationships of all edges in the path, and constructs a behavior graph path structure.

[0034] The path intersection analysis module extracts behavior term path node sequences according to the behavior graph path structure, compares intersection nodes and counts end node frequencies, filters intersection paths, and obtains a behavior term intersection path set.

[0035] The semantic class determination module collects semantic labels of end nodes in the behavior term intersection path set, matches path end labels after sorting by occurrence frequency, and obtains a behavior term semantic attribution label group.

[0036] The classification structure output module counts graph classification nodes to which each label belongs according to the behavior term semantic attribution label group, divides behavior term paths to corresponding nodes, establishes a node and behavior term path classification attribution relationship structure, and generates a behavior classification structure table.

[0037] Preferably, the system further comprises a digital twin simulation module that constructs a digital twin model of the monitored scene, collects real-time data of the scene through sensors deployed in the monitored scene, and synchronizes the real-time data to the digital twin model.

[0038] In the digital twin model, the actual behavior state is simulated, and the behavior development trend and potential anomalies are predicted.

[0039] An intelligent control strategy is set, which combines the degree of mutual influence between real-time data and the execution conditions of control action priority, judges whether there is a logical conflict in the rule engine, and if so, optimizes the control strategy;

[0040] The tracking, alarm and recording operations are automatically executed according to the optimized control strategy, and the control behavior and feedback data are recorded in real time, and the digital twin model is continuously adjusted based on the recorded feedback data.

[0041] Preferably, the application also includes an intelligent behavior analysis method of a monitoring camera, which includes all the modules and method processes of the intelligent behavior analysis system of the monitoring camera.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] In the video data acquisition link, the system can not only obtain the video stream data of the monitoring camera and extract the video frame sequence and timestamp, but also actively identify the moving target and position coordinates in the video frame, generate a target behavior active set by associating the target behavior segments and calculating the behavior activity value. This process changes the current situation that the prior art can only simply identify the target position and cannot quantify the behavior activity, so that the target behavior changes from "fuzzy distinguishable" to "precise measurable". Whether it is people flow monitoring in commercial places or equipment operation monitoring in industrial parks, management personnel can intuitively grasp the behavior state of the target through the behavior activity value, such as the density of personnel gathering and the activity frequency of equipment operation, so as to quickly judge the basic situation in the region and provide a clear initial basis for subsequent management decision-making.

[0044] The behavior sequence modeling module solves the problem of insufficient correlation analysis of behavior sequence in the prior art by extracting the behavior activity value and corresponding coordinates, sorting adjacent behavior segments in time sequence and marking consistent and conflict sections to form a behavior consistency partition annotation set. In the traffic monitoring scene, the system can clearly distinguish the consistent section of normal vehicle driving and the conflict section of illegal lane changing, red light running, etc. Without manually checking the video frame by frame, potential traffic violation behavior segments can be quickly located, greatly improving the violation identification efficiency; in the security monitoring of residential areas, for the walking track of personnel in the park, the system can accurately mark the consistent section conforming to the conventional route and the conflict section deviating from the normal path and wandering, helping security personnel focus on suspicious behavior and reducing invalid monitoring time.

[0045] The environmental mapping module is aimed at the pain point of environmental interference affecting the accuracy of behavior analysis. By obtaining the target behavior of the behavior consistent section, extracting the light and occlusion time sequence, and comparing the change amplitude combined with the behavior type, the behavior influence strength under the environmental interference is evaluated, and the behavior influence superposition analysis result is generated. In the scene of insufficient light such as night or rainy weather, the system can identify the influence degree of light change on pedestrian and vehicle behavior recognition, avoiding misjudgment of normal behavior as abnormal; when temporary occlusion appears in the monitoring picture, the system can judge whether the occlusion leads to behavior recognition deviation by analyzing the association between the occlusion time sequence and the target behavior, and then correcting the analysis result. This makes the system still maintain stable and accurate behavior analysis ability in complex and variable environmental conditions, breaking the limitations of the prior art restricted by environmental factors.

[0046] The abnormality screening module forms an abnormal point set by identifying abnormal points in the behavior influence superposition analysis result whose behavior response value is greater than the average response reference value and is in the conflict section, significantly improving the accuracy of abnormal behavior recognition. The prior art often misjudges due to single threshold judgment, while the system performs double screening combined with behavior response value and section attribute, which not only excludes normal behavior in the consistent section with high response value caused by environmental interference, but also accurately captures the abnormal behavior in the conflict section. For example, in the monitoring of people flow in a shopping mall, the system will not misjudge the normal peak of people flow during festivals as abnormal, but can accurately identify abnormal behaviors such as disordered crowding of personnel in the conflict section during off-peak hours; in the monitoring of factory production workshops, it can exclude temporary activity caused by normal equipment maintenance and accurately identify abnormal operation of equipment, avoiding "missed judgment" and "misjudgment" of abnormality recognition.

[0047] The risk output module marks the point with diffusion risk by obtaining all points and information of the abnormal point set and generates behavior detection and risk warning result, giving the monitoring system the ability of active prevention. The prior art can only mark abnormal points and cannot evaluate the risk diffusion possibility, while the system not only tells the management party "where the abnormality exists", but also prompts "whether the abnormality will spread". In the crowd gathering scene, after the system marks the gathering point with diffusion risk, the management personnel can deploy personnel for guidance in advance to prevent overcrowding, stampede and other accidents; in an industrial park, if an abnormality occurs in a device and there is a diffusion risk, the system can timely warn and help maintenance personnel take disposal measures before the risk expands. This makes the monitoring system shift from "post-tracing" to "pre-warning", truly playing the active defense role of security monitoring, and providing more comprehensive protection for safety management in various scenes. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The timing diagram of the intelligent behavior analysis system of the monitoring camera described in the present application;

[0049] Figure 2 A flowchart illustrating the composition of the system's various sets and results;

[0050] Figure 3 A flowchart illustrating how the behavior sequence modeling module works. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 This invention provides an intelligent behavior analysis system and method for surveillance cameras. The system includes: a video data acquisition module, a behavior sequence modeling module, an environment mapping module, an anomaly screening module, and a risk output module.

[0053] The video data acquisition module acquires video stream data from surveillance cameras, extracts video frame sequences and timestamps, identifies moving targets and their coordinates in the video frames, associates target behavior segments and calculates corresponding behavior activity values, generating a target behavior activity set. The behavior sequence modeling module extracts the behavior activity values ​​and corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, marks consistent and conflicting segments in adjacent behavior segments, and obtains a behavior consistency partition annotation set. The environment mapping module acquires target behaviors located in the behavior consistency partition annotation set within the behavior consistency segment, extracts illumination and occlusion time series, compares the change amplitude with the behavior type, assesses the intensity of behavior influence under environmental interference, and generates behavior influence superposition analysis results. The anomaly screening module identifies anomalous points in the behavior influence superposition analysis results where the behavior response value is greater than the average response benchmark value and is located in the behavior conflict segment, forming a behavior anomaly point set. The risk output module acquires all points in the behavior anomaly point set and their corresponding point information, marks points with a risk of spread, and generates behavior detection and risk warning results.

[0054] Example 1: See Figure 2The construction of the target behavior active set involves the integration and processing of multiple data dimensions. The video data acquisition module obtains raw video stream data from the surveillance camera. This data contains continuous image frames and corresponding timestamp information. The moving object detection algorithm identifies moving objects in each frame and records their pixel coordinates. The system associates the detection results of the same target in consecutive frames to form behavior segments that change over time. Each behavior segment calculates a behavior activity value, which is obtained by averaging two environmental factors: the light intensity and the degree of occlusion in the target's area after normalization. The final target behavior active set is a structured data set, in which each record contains the timestamp of the behavior, spatial coordinates, normalized light value, normalized occlusion value, and the calculated behavior activity value. The generation of the behavior consistency partition annotation set relies on the comparative analysis of the change trends of adjacent behavior segments. The system extracts the behavior activity values ​​and their corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, and calculates the behavior activity value difference rate between each pair of adjacent segments. This difference rate reflects the magnitude of the change in activity level. At the same time, the system analyzes the consistency of the illumination change direction and occlusion change direction between adjacent segments. If the change trends in the two directions are the same, they are marked as behavior consistency segments; if the trends are opposite, they are marked as behavior conflict segments. This process generates a behavior consistency partition annotation set containing segment annotations and difference rate data.

[0055] The behavioral impact superposition analysis results focus on assessing the interference of environmental factors on behavioral patterns. The system selects segments marked as behaviorally consistent from the behavioral consistency partition label set, extracts the illumination and occlusion data for each target behavior within these segments, and arranges them in chronological order to form illumination time series and occlusion time series. By calculating the rate of change between consecutive time nodes in the series, the illumination change rate and occlusion change rate are obtained respectively. Under the constraint of the same behavior type, the system compares the two types of rate indicators side by side and analyzes their joint impact on the magnitude of behavioral change. The final behavioral impact superposition analysis results include comparative data of behavioral responses under each type of environmental interference condition. The identification of behavioral anomaly sets is based on a multi-threshold judgment logic. The system filters out two types of target behaviors from the behavioral impact superposition analysis results: records whose behavioral response values ​​exceed the average response benchmark value, and records that are in behavioral conflict zones. For these target behaviors, the system extracts behavioral type and regional data at continuous time nodes, calculates the type change ratio and regional change ratio of adjacent time nodes, and simultaneously acquires the illumination amplitude and occlusion fluctuation data for the corresponding time period. When the type change ratio and regional change ratio both exceed the preset fluctuation identification threshold, and the illumination amplitude and occlusion fluctuation also exceed the anomaly judgment threshold, the system marks the corresponding time node as an anomaly point. All these anomaly points together constitute a behavioral anomaly set, which records the spatial location, illumination occlusion amplitude characteristics, behavioral type, and area fluctuation ratio of each anomaly point.

[0056] The generation of behavior detection and risk warning results is the final output of the system. The system acquires information on all points in the set of abnormal behavior points, including coordinates and identification data, and calculates a joint risk judgment value for each point to form a sequence. Based on this sequence, points that simultaneously meet three conditions are further selected: the behavior response value is higher than the behavior response risk threshold, the behavior activity value is lower than the activity benchmark value, and the behavior consistency label is a conflict segment. These points are marked as detection anomalies with a risk of spread. The system extracts their numbers, location identifiers, and partition information, organizes them in a structured format, and outputs them. The final generated behavior detection and risk warning results include a detailed list of anomaly points and a joint judgment label for each point based on three indicators. In the process of converting the raw video stream into the final risk warning, the system gradually refines and identifies behavioral patterns with risk characteristics through a series of processing steps, such as environmental factor normalization, trend consistency analysis, environmental interference quantitative assessment, and multi-threshold anomaly detection. This progressive analysis method can effectively integrate spatiotemporal information and environmental factors.

[0057] Example 2: See Figure 3The submodule continuously receives real-time video streams from surveillance cameras. The video stream consists of a series of image frames with precise timestamps. Each frame is processed by a built-in motion detection algorithm to identify moving targets in the scene and record their pixel coordinates and the time of their actions. At the same time, the system collects environmental data corresponding to each coordinate point, mainly light intensity and occlusion degree. These two types of data are recorded as light factor and occlusion factor, respectively. Together with the target's spatial coordinates and time information, they constitute the target behavior environmental factor data set. This data set is transmitted to the subsequent processing unit as the basic dataset. After receiving the target behavior environmental factor data set, the environmental factor normalization submodule starts the processing flow. This module reads the raw values ​​of the illumination factor and occlusion factor respectively. These values ​​may come from different sensors and have different dimensions, so they need to be normalized to eliminate the difference in dimensions. The normalization process linearly transforms the raw data into the range of [0,1], making environmental data from different sources comparable. The normalized illumination value and normalized occlusion value are re-mapped to the target coordinate position. The system calculates the arithmetic mean of these two normalized values ​​as the behavior activity value. All data points processed in the above way are arranged in chronological order to finally form a structured target behavior activity set.

[0058] The activity value extraction submodule in the behavior sequence modeling module begins operation. It reads the activity values ​​and their corresponding coordinate data from the target behavior activity set, identifies the chronological order of all target behaviors on the timeline by combining timestamp information, calls the stored target behavior coordinate set, and calculates the interval between temporally adjacent target behaviors based on a preset adjacent time threshold. All target behaviors are then sorted according to chronological order, generating a temporally ordered sequence of adjacent target behavior intervals. The difference rate calculation submodule performs calculations based on the adjacent target behavior interval order sequence. This module iterates through each pair of adjacent target behavior records in the sequence, calculating the difference rate of their activity values. The difference rate is calculated by subtracting the previous activity value from the subsequent activity value and then dividing by the previous activity value, yielding a percentage value representing the relative degree of change. The calculation results for all adjacent behavior pairs are integrated into a complete activity value difference rate sequence, which quantitatively describes the fluctuation of behavior activity over time.

[0059] The consistency recognition submodule receives the behavior activity value difference rate sequence as input. This module further analyzes the environmental change direction characteristics between each pair of adjacent target behaviors, extracts the illumination change direction and occlusion change direction corresponding to adjacent behaviors from the raw data, and determines whether the change trend of each pair of behaviors in these two directions is consistent, i.e., simultaneously enhanced, simultaneously weakened, or opposite. Based on the judgment result, each pair of behaviors is marked as consistent in direction or conflicting in direction. The system groups and records all behavior pairs. Those with consistent directions are classified into the behavior consistency segment, and those with conflicting directions are classified into the behavior conflict segment. Finally, the system outputs a behavior consistency partition label set with classification labels.

[0060] Taking a large parking lot monitoring system as an example, the video data acquisition module starts operating, and the monitoring cameras continuously capture video stream data within the parking lot. The video information acquisition submodule receives these video streams, extracts video frame sequences with millisecond-level timestamps, identifies moving targets in the video frames using a moving target detection algorithm, including moving vehicles and pedestrians, and records the pixel coordinates of each target. Simultaneously, the system collects environmental data corresponding to each coordinate position: it uses a photosensor to acquire light intensity data and uses an image analysis algorithm to calculate occlusion data. These data are recorded as light factors and occlusion factors, which, together with the target coordinates and timestamps, constitute the target behavior environmental factor data set. The environmental factor normalization submodule processes the target behavior environmental factor data set. This module reads the original values ​​of the illumination factor (ranging from 800 to 1200 lux) and the original values ​​of the occlusion factor (ranging from 0 to 100%), and performs normalization processing to transform them into the numerical range of [0,1]. The normalized illumination and occlusion values ​​are mapped to the corresponding target coordinates. The system calculates the arithmetic mean of the normalized illumination value and the normalized occlusion value at each location point to obtain the behavioral activity value at that location. All processed data points are organized in chronological order to generate a complete target behavior activity set containing timestamps, coordinates, normalized environmental factors, and behavioral activity values.

[0061] The activity value extraction submodule in the behavior sequence modeling module begins operation. This module reads all data records from the target behavior activity set, identifies the temporal order of each target behavior based on timestamp information, and calculates and sorts the temporally adjacent target behaviors using 30-second intervals as the adjacent time threshold. This generates a temporally ordered sequence of adjacent target behavior intervals, which accurately reflects the temporal relationship of each target behavior within the parking lot. The difference rate calculation submodule performs calculations based on the adjacent target behavior interval order sequence. This module iterates through each pair of adjacent target behavior records in the sequence, calculating the relative change rate of the activity value of the subsequent behavior compared to the activity value of the previous one. For example, for two adjacent behaviors at timestamps T1 and T2, the activity values ​​are 0.65 and 0.72 respectively, and the difference rate is (0.72-0.65) / 0.65=10.77%. The difference rate calculation results for all adjacent behavior pairs are integrated into a complete behavior activity value difference rate sequence, which quantitatively describes the temporal change characteristics of the activity level within the parking lot. The consistency identification submodule analyzes the behavioral activity value difference rate sequence. This module extracts the direction of light change and the direction of occlusion change corresponding to each pair of adjacent target behaviors. For example, in a pair of adjacent behaviors, the light increases from 850 lux to 920 lux (direction is increase), and the occlusion decreases from 30% to 25% (direction is decrease). The system determines that the change directions of these two environmental factors are inconsistent and marks them as direction conflict segments. When the change directions of the two environmental factors are consistent, they are marked as direction consistent segments. The marking results of all segments finally form a behavioral consistency partition label set.

[0062] Example 3: The submodule receives the behavior consistency partition label set from the preceding module, filters out all data segments marked as behavior consistency, and for each target behavior within these consistent segments, the system extracts the light intensity data and occlusion degree data collected within its complete time period, arranges them in chronological order to form light time series and occlusion time series respectively. These time series data together constitute the environmental change time series set. The influence superposition calculation submodule conducts in-depth analysis based on the environmental change time series set. This module calculates the rate of change between consecutive time nodes within the time period of each target behavior. For the light time series, it calculates the rate of light change between adjacent time nodes; for the occlusion time series, it similarly calculates the rate of occlusion change between adjacent time nodes. Under the constraint of the same behavior type, the two types of rate indicators are compared and analyzed side by side. Through numerical relationship identification methods, the combined influence of the two types of environmental factors on the magnitude of behavior change is analyzed. The final behavior influence superposition analysis result includes detailed data on the degree of influence of light change rate, the degree of influence of occlusion change rate, and the comparison data of behavior response under various environmental interference conditions. The record extraction submodule in the anomaly filtering module begins to process the results of the behavior impact superposition analysis. Based on preset filtering conditions, this submodule extracts two types of specific target behaviors from the analysis results: records whose behavior response values ​​exceed the average response benchmark value calculated by the system, and records that are in the behavior conflict zone. For these target behaviors, continuous behavior records are extracted in chronological order, and the specific behavior type identifier and behavior area range data corresponding to each time node are collected. All these extracted data are organized into a structured continuous behavior record set.

[0063] The type-region ratio calculation submodule receives and processes a continuous set of behavior records. For each target behavior in the record set, this module extracts its type identifier value and region range data at two consecutive time points, and calculates the type change ratio, expressed as: ;in: Indicates the ratio of type changes. This indicates the type value of the current time node. This indicates the type value of the previous time node. The function takes the larger of the two values ​​and calculates the behavior region change ratio. A similar method is used to calculate the proportional relationship of regional changes between consecutive time nodes. All calculation results are integrated into a type change ratio sequence and a behavior region change ratio sequence, which together constitute the behavior fluctuation change dataset.

[0064] The anomaly identification submodule makes the final judgment based on the behavioral fluctuation change dataset. This module extracts the illumination amplitude data and occlusion fluctuation data corresponding to each time point from the dataset, sets two sets of threshold values: a fluctuation identification threshold and an anomaly judgment threshold. It judges whether the type change ratio and the area change ratio both exceed the fluctuation identification threshold, and simultaneously judges whether both illumination amplitude and occlusion fluctuation exceed the anomaly judgment threshold. When a time point meets both of these conditions, the system marks it as an anomaly. All marked anomalies are collected to form the final behavioral anomaly point set, which is the output of this module. Starting from the extraction of environmental time series data, through change rate calculation and impact degree analysis, to multi-condition record screening and change ratio calculation, the anomaly point identification is finally completed through multiple threshold judgments.

[0065] Example 4: The implementation process of the risk output module begins. The joint indicator judgment submodule receives a set of abnormal behavior points from the preceding module. This set contains multiple data points marked as abnormal. Each data point includes its spatial coordinates, unique identifier, and related technical parameters. The submodule reads detailed data from these points, including the behavioral response value, behavioral activity value, and behavioral consistency label for each point. Based on these parameters, a composite indicator—the joint risk judgment value—is calculated. The calculation of the joint risk judgment value adopts a multi-parameter weighted fusion method, where the weight coefficient for the behavioral response value is set to 0.5, the weight coefficient for the behavioral activity value is 0.3, and the weight coefficient for the behavioral consistency label is 0.2. The weight coefficients are pre-set according to the importance of each parameter in the actual application scenario. The calculated joint risk judgment values ​​are arranged in chronological order to form a joint risk judgment value sequence, which reflects the comprehensive risk level of each abnormal point.

[0066] The anomaly output processing submodule further processes the data based on the joint risk assessment value sequence. This module sets three filtering conditions: the behavior response value must be greater than the preset behavior response risk threshold of 0.7, the behavior activity value must be lower than the activity benchmark value of 0.4, and the behavior consistency label must be a conflict zone. The system traverses all points in the sequence and specially marks points that simultaneously meet these three conditions, identifying these points as anomalies with a risk of spreading. The system extracts complete information from the marked points, including the point number, spatial coordinates, the monitoring area partition identifier, and the specific values ​​of various technical parameters. All this information is organized according to a prescribed structured format to generate the final behavior detection and risk warning results, which are output in the form of a data table. See Table 1 for the detection results of anomalies in the monitoring area.

[0067] Table 1: Results of Anomaly Detection

[0068]

[0069] In a specific application scenario, taking parking lot vehicle monitoring as an example, the system detected multiple abnormal behavior points. Point P-001 is located in the northeast corner of the parking lot. This location recorded a high behavioral response value of 0.85, indicating that the target behavior at this location is highly sensitive to environmental changes. Simultaneously, its behavioral activity value of 0.32 is lower than the baseline value of 0.4, showing a low level of behavioral activity. Combined with its characteristic of being marked as a conflict zone, this point was determined to be a high-risk point. Point P-003 is located in the southwest area of ​​the parking lot. It detected a high behavioral response value of 0.92, and its behavioral activity value of 0.29 is significantly lower than the baseline value. The consistency label indicates a conflict zone. These characteristics indicate significant abnormal behavior at this location. The system calculated its combined risk value to be 0.81, classifying it as a high-risk level.

[0070] Location P-005, situated in the southeast corner of the parking lot, has the highest behavioral response value (0.95) and the lowest behavioral activity value (0.26) among all locations. The labeling of the conflict zone confirms the inconsistency in behavioral patterns. These characteristics result in a combined risk value of 0.83 for this location, making it a high-risk location requiring close monitoring. For location P-004, although its behavioral response value (0.68) is slightly below the risk threshold and its behavioral activity value (0.41) is close to the baseline, it is classified as a medium-risk location because it is located in a consistent zone and does not meet all risk assessment criteria. While the risk level of such locations is relatively low, monitoring is still necessary. The system's output of behavioral detection and risk warning results not only includes detailed technical parameters for each location but also provides a risk level assessment based on a comprehensive multi-indicator judgment. This structured output format facilitates monitoring personnel in quickly identifying high-risk areas and taking timely warning and response measures.

[0071] Example 5: The behavior knowledge graph construction module begins processing behavior terminology data from the active set of target behaviors. This module extracts all textual information describing behaviors from the set, including terminology descriptions such as behavior type and behavior characteristics. It performs word segmentation on these terminology texts, decomposing them into basic morpheme units. Each morpheme is assigned a unique index identifier. The system records the first occurrence position of each keyword in the text and its frequency throughout the text. Based on the morpheme index and position information, an ordered sequence of morphemes is generated, constructing the original word order sequence template for behavior terms. The graph path generation module receives the original word order sequence template as input. This module sorts all morpheme units in the template, arranging them from general to sequential. Based on the sorting result, a directed path is established from the top-level concept to the bottom-level specific concept. During path construction, the node number information of all connecting edges is collected, and the semantic relationship type between adjacent nodes is recorded, such as "belongs to," "includes," and "leads to." Finally, a complete behavior graph path structure is constructed, which clarifies the hierarchical and associative relationships between concepts.

[0072] The path cross-analysis module operates based on the path structure of the behavior graph. This module extracts the complete path node sequence of all behavior terms, compares the node intersections between different paths, and statistically analyzes the frequency and distribution of the endpoint nodes of each path. By setting a path cross-degree threshold, it filters out path combinations with significant cross-features. These cross-paths indicate strong semantic associations between different behavior terms. All filtered cross-paths are compiled into a behavior term cross-path set, providing data support for subsequent semantic analysis. The semantic classification determination module processes the endpoint node data in the behavior term cross-path set. This module collects corresponding semantic label information for each endpoint node. These labels describe the core semantic features of the node and are sorted according to the frequency of the node's appearance in the path. High-frequency nodes are matched to the path endpoints first. A label matching algorithm is used to assign appropriate semantic labels to each endpoint node, ultimately obtaining a semantic classification label group for each behavior term. This label group clarifies the semantic category of each behavior term.

[0073] The classification structure output module outputs a final structured result based on the semantic attribution label groups of behavioral terms. This module statistically analyzes the graph classification nodes to which each semantic label belongs, assigns behavioral term paths with the same semantic attribution to the corresponding classification nodes, establishes a mapping relationship between classification nodes and behavioral term paths, forming a complete classification attribution structure. The final output behavioral classification structure table displays the classification results of all behavioral terms according to semantic features. The digital twin simulation module constructs a virtual digital model of the monitoring scene. This module collects real-time environmental data, including parameters such as video stream, temperature, humidity, and light intensity, through various sensor devices deployed at the monitoring site. The collected real-time data is synchronized to the digital twin model through a data interface, reproducing the real-time state of the actual monitoring scene in the virtual model.

[0074] The digital twin model simulates and extrapolates actual behavioral states. Based on current behavioral data and historical behavioral patterns, this module uses predictive algorithms to calculate future trends in behavior. By establishing a behavioral evolution model, it predicts potential abnormal behavioral patterns, enabling the system to identify potential risky behaviors in advance and providing data support for risk warnings. The intelligent control strategy module is responsible for building the rule engine. This module analyzes the interrelationships between various real-time data, determines the priority execution conditions for different control actions, checks for potential logical conflicts in the rule engine, and initiates optimization algorithms when rule conflicts are detected to adjust and optimize the control strategy, ensuring the coordination and effectiveness of the rule system. Based on the optimized control strategy, the system automatically executes monitoring operations, automatically tracking target behavior, generating early warning and alarm information, and recording behavioral data. All control operations and system feedback are recorded in real time to form log data. Based on this feedback data, the parameter settings of the digital twin model are continuously adjusted and optimized, enabling the model to better reflect changes in the actual scenario.

[0075] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0076] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent behavior analysis system for monitoring a camera, characterized in that, The system comprises: The video data acquisition module acquires video stream data of a monitoring camera, extracts video frame sequences and time stamps, identifies moving targets and position coordinates in the video frames, associates target behavior segments and calculates corresponding behavior activity values, and generates a target behavior activity set; The behavior sequence modeling module extracts behavior activity values and corresponding coordinates in the target behavior activity set, sorts adjacent behavior segments in chronological order, marks consistent and conflicting sections in adjacent behavior segments, and obtains a behavior consistency partition annotation set; The environment mapping module acquires target behaviors located in the behavior consistent section in the behavior consistency partition annotation set, extracts light and occlusion time sequences, compares the change amplitudes combined with behavior types, evaluates the behavior impact strength under environmental interference, and generates a behavior impact superposition analysis result; The anomaly screening module identifies abnormal points in the target behaviors in the behavior impact superposition analysis result that have a behavior response value greater than an average response reference value and are in a behavior conflict section, forms a behavior anomaly point set, and generates a behavior detection and risk warning result; The video data acquisition module comprises: The video information acquisition sub-module acquires video stream data and time stamps of a monitoring camera, extracts moving target coordinates and behavior time data in the video frames, collects light intensity and occlusion degree data corresponding to the coordinate positions, and records the collection results as two environmental factors, i.e., light factor and occlusion factor, to obtain a target behavior environmental factor data group; The environment factor normalization sub-module performs normalization processing on the light factor and occlusion factor data in the target behavior environmental factor data group, respectively, establishes a corresponding relationship between the normalized results and the target coordinate positions, calculates the average values of the normalized light values and the normalized occlusion values as behavior activity values, and generates a target behavior activity set. The target behavior activity set includes behavior activity values, target spatial coordinates, and normalized environmental factors. The behavior consistency partition annotation set specifically includes behavior consistent section annotations, behavior conflict section annotations, and adjacent behavior activity value difference rates. The behavior impact superposition analysis result includes light change rate on behavior impact degree, occlusion change rate on behavior impact degree, and behavior response comparison under each environmental interference condition. The behavior anomaly point set includes abnormal point spatial positions, abnormal point light and occlusion amplitude characteristics, abnormal point behavior types, and area fluctuation ratios. The behavior detection and risk warning result includes a list of detected abnormal points, and a joint determination label of three abnormal point indicators.

2. The intelligent behavior analysis system of claim 1, wherein, The behavior sequence modeling module comprises: 3.The intelligent behavior analysis system of surveillance camera of claim 1, wherein, The activity value extraction sub-module acquires behavior activity values and corresponding coordinate data in the target behavior activity set, identifies the chronological order relationship of all target behaviors according to the time stamp information, calls a target behavior coordinate set, takes an adjacent time threshold as a reference, measures and sorts the target behaviors in time at intervals, and generates an adjacent target behavior interval sorting sequence; ​ The difference rate calculation sub-module calculates the behavior activity value difference rate between two adjacent target behaviors based on the adjacent target behavior interval ordering sequence, and integrates to generate a behavior activity value difference rate sequence; The consistency identification sub-module extracts the light change direction and the occlusion change direction between adjacent target behaviors based on the behavior activity value difference rate sequence, classifies and labels according to whether the change trend of each pair of target behaviors in the two directions is consistent, records and groups the sections with consistent and conflicting directions respectively, and obtains a behavior consistency partition labeling set.

4. The intelligent behavior analysis system of claim 3, wherein, The environment mapping module comprises: The environment sequence extraction sub-module filters the sections labeled as consistent behaviors according to the behavior consistency partition labeling set, detects the light data and occlusion data in each target behavior time period, arranges the light data and occlusion data in time sequence to form a light time sequence and an occlusion time sequence, and generates an environment change time sequence set; The influence superposition calculation sub-module calculates the light change rate and the occlusion change rate between consecutive time nodes in each target behavior time sequence based on the environment change time sequence set, compares the light change rate and the occlusion change rate under the condition of the same behavior type, identifies the numerical relationship of the change amplitude of the behavior type through joint analysis of the two types of rate indicators, integrates the influence value sequence of each target behavior, and establishes a behavior influence superposition analysis result.

5. The intelligent behavior analysis system of claim 4, wherein, The anomaly screening module comprises: The record extraction sub-module filters the target behaviors with behavior response values greater than the average response reference value and the target behaviors in the behavior conflict sections according to the behavior influence superposition analysis result, extracts the continuous behavior records of the target behaviors in time sequence, collects the behavior type and behavior region data corresponding to each time node, and generates a continuous behavior record set; The type region ratio calculation sub-module calls the continuous behavior record set, respectively extracts the type value and region of the target behavior at two consecutive time nodes, respectively calculates the type change ratio and the behavior region change ratio, integrates them into a type change ratio sequence and a behavior region change ratio sequence, and establishes a behavior fluctuation change data set; The anomaly identification sub-module extracts the light amplitude change data and the occlusion fluctuation data corresponding to the time period based on the behavior fluctuation change data set, judges whether the type change ratio and the region change ratio exceed the set fluctuation recognition threshold, judges whether the light amplitude change and the occlusion fluctuation exceed the abnormality judgment threshold at the same time, marks the time nodes that meet the conditions as abnormal points, and generates a behavior abnormal point set.

6. The intelligent behavior analysis system of claim 5, wherein, The risk output module comprises: The index joint determination sub-module obtains all points in the behavior abnormal point set and corresponding coordinate and identification information, calculates a joint risk determination value, and establishes a joint risk determination value sequence; The anomaly output arrangement sub-module filters the points with behavior response values greater than the behavior response risk threshold, behavior activity values lower than the activity reference value, and behavior consistency labels as conflict sections based on the joint risk determination value sequence, extracts the corresponding point number, position identifier and belonging partition, marks them as detection anomalies and diffusion risks, outputs the points meeting the joint conditions in a structured format, and generates a behavior detection and risk warning result.

7. The intelligent behavioral analysis system of a surveillance camera according to claim 1, wherein, The system further comprises a behavior knowledge graph construction module, which extracts behavior term text based on the target behavior active set, extracts morpheme index through word segmentation, judges the first position and frequency of keywords, maps to generate morpheme arrangement sequence, and constructs behavior term original language sequence template; The graph path generation module sorts morphemes based on the behavior term original language sequence template, establishes morpheme directed path from top layer to end layer according to the sorting result, collects connection node number and adjacent node relationship of all edges in the path, and constructs behavior graph path structure; The path intersection analysis module extracts behavior term path node sequence according to the behavior graph path structure, compares intersection nodes and counts end node frequency, filters intersection paths, and obtains behavior term intersection path set; The semantic class determination module collects semantic labels of end nodes based on the end nodes in the behavior term intersection path set, matches path end label after sorting according to appearance frequency, and obtains behavior term semantic attribution label group; The classification structure output module counts graph classification nodes to which each label belongs according to the behavior term semantic attribution label group, divides behavior term paths to corresponding nodes, establishes node and behavior term path classification attribution relationship structure, and generates behavior classification structure table.

8. The intelligent behavior analysis system of claim 7, wherein, The system further comprises a digital twin simulation module, which constructs a digital twin model of the monitoring scene, collects real-time data of the scene through sensors deployed in the monitoring scene, and synchronizes the real-time data to the digital twin model; Simulate the actual behavior state in the digital twin model, predict the behavior development trend and potential abnormalities; Set an intelligent control strategy, which combines the mutual influence degree between real-time data and the execution conditions of control action priority, judges whether there is logical conflict in the rule engine, and if there is, optimizes the control strategy; According to the optimized control strategy, automatically execute tracking, alarm and recording operations, and record control behavior and feedback data in real time, and continuously adjust the digital twin model based on the recorded feedback data.

9. A method for intelligent behavior analysis of surveillance cameras, characterized in that, All modules and method processes of the intelligent behavior analysis system including the monitoring camera of any one of claims 1 to 8.

Images