Intelligent behavior analysis system and method for monitoring camera

Abstract

The invention relates to the technical field of monitoring behavior analysis, and discloses an intelligent behavior analysis system and method for a monitoring camera. The system obtains video stream data through a video data acquisition module, extracts a video frame sequence and a timestamp, identifies a moving target and a position coordinate, associates behavior segments and calculates an active value, and generates a target behavior active set; a behavior sequence modeling module extracts active values and coordinates, sorts adjacent behavior segments and marks consistent and conflict sections to obtain a behavior consistency partition marking set; the environment mapping module obtains consistent section target behaviors, extracts illumination and shielding time sequences, evaluates the influence intensity of environment interference on the behaviors in combination with behavior types, and generates an overlay analysis result; the exception screening module recognizes exceptional points with response values larger than the average reference value and located in the conflict section, and a behavior exceptional point set is formed; and the risk output module obtains abnormal point information, marks diffusion risk point locations, and generates behavior detection and risk early warning results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of behavior analysis, in particular to an intelligent behavior analysis system and method for monitoring cameras. BACKGROUND

[0002] In the current security monitoring field, monitoring cameras have been widely used in urban traffic, residential areas, commercial places, industrial parks and other types of scenes. Its core function is to realize dynamic monitoring of personnel, vehicles and other targets in a specific area through video data collection, providing visual basis for security prevention, event tracing, management and scheduling. However, the functions of existing monitoring systems are often limited to basic video recording and real-time preview, lacking effective deep mining and intelligent analysis capabilities for the target behavior information contained in the video data, making it difficult to meet the needs of accurate identification and early warning of potential risks in complex scenarios.

[0003] In the prior art, some monitoring systems with preliminary behavior analysis function have obvious defects in video data processing. Such systems can only simply identify the moving targets in the video frames and mark their positions, but cannot effectively associate the behavior segments of the targets in continuous time, and it is even more difficult to quantitatively evaluate the activity level of the target behavior. For example, in the flow monitoring of a commercial complex, the system may only be able to determine that there is personnel flow in a certain area, but cannot distinguish between different active states of behavior such as normal walking, staying and gathering, or disordered passing, making it difficult for management personnel to quickly grasp the flow situation in the area. In terms of behavior sequence processing, the existing technology generally lacks fine analysis of the correlation between adjacent behavior segments. They mostly simply arrange the behavior segments in chronological order, ignoring the consistency and conflict characteristics of behaviors in adjacent segments. Taking traffic intersection monitoring as an example, when a vehicle exhibits consecutive behaviors such as lane changing, turning, parking, etc. at an intersection, existing systems cannot accurately mark the consistent sections of the vehicle's normal driving and the conflict sections such as illegal lane changing, making the identification of traffic violations dependent on manual frame-by-frame video viewing, which is not only inefficient but also prone to missed or incorrect judgments due to human negligence. The interference of environmental factors on behavior analysis results is also a bottleneck that existing monitoring technology cannot break through. Changes in the strength of natural light, the temporary appearance of obstructions and other environmental conditions will directly affect the clear identification of target behavior in video frames. However, existing systems often do not combine time series data of environmental factors such as light and obstruction with target behavior types for combined analysis, cannot accurately assess the influence strength of environmental interference on behavior judgment, and thus lead to deviations in behavior analysis results. For example, in night monitoring, due to insufficient light, the system may misjudge a pedestrian's normal bending to pick up an object as an abnormal behavior, or due to strong light radiation, the vehicle features may be blurred, making it impossible to accurately identify whether its driving trajectory is compliant. The limitations of existing technologies are even more pronounced in the screening and risk warning stages of abnormal behavior. Most systems determine the existence of anomalies by setting a single behavioral threshold, such as defining running as an abnormal behavior, without considering the context and relationship of the behavior, or combining it with the consistency or conflict areas in which the behavior occurs. This results in extremely low accuracy in anomaly identification. Even if abnormal behavior is identified, existing systems can only simply mark the abnormal location and cannot further analyze whether the anomaly poses a risk of spread. For example, if abnormal crowd gatherings are not warned in time, they may trigger chain risks such as overcrowding and stampedes. Existing systems cannot output warning results that include a risk assessment of spread in advance, which greatly reduces the proactive prevention capabilities of the monitoring system and fails to buy management effective emergency response time. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent behavior analysis system and method for surveillance cameras to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides an intelligent behavior analysis system for surveillance cameras, the system comprising: The video data acquisition module acquires video stream data from the surveillance camera, extracts video frame sequences and timestamps, identifies moving targets and their position coordinates in the video frames, associates target behavior segments and calculates the corresponding behavior activity values, and generates a target behavior activity set. The behavior sequence modeling module extracts the behavior activity value and corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, marks consistent and conflicting segments in adjacent behavior segments, and obtains a behavior consistency partition label set. The environment mapping module obtains the target behaviors located in the behavior consistency segment of the behavior consistency partition annotation set, extracts the illumination and occlusion time series, compares the change amplitude with the behavior type, evaluates the intensity of behavior influence under environmental interference, and generates behavior influence superposition analysis results. The anomaly screening module identifies anomalous points in the target behaviors whose behavior response values ​​are greater than the average response baseline value and are located in the behavior conflict zone in the behavior impact superposition analysis results, forming a set of behavior anomaly points. The risk output module obtains all points and corresponding point information in the set of abnormal behavior points, marks points with the risk of spread, and generates behavior detection and risk warning results.

[0006] Preferably, the target active behavior set includes active behavior value, target spatial coordinates, and normalized environmental factors; the behavior consistency partitioning label set specifically includes consistent behavior segment labels, conflict behavior segment labels, and adjacent active behavior value difference rates; the behavior influence superposition analysis results include the influence of illumination change rate on behavior, the influence of occlusion change rate on behavior, and a comparison of behavior responses under each environmental interference condition; the behavior anomaly point set includes the spatial location of anomaly points, illumination occlusion amplitude characteristics of anomaly points, and the behavior type and area fluctuation ratio of anomaly points; and the behavior detection and risk warning results include a list of detected anomaly points and a joint judgment label for the three indicators of anomaly points.

[0007] Preferably, the video data acquisition module includes: The video information acquisition submodule acquires video stream data and timestamps from surveillance cameras, extracts the coordinates and behavior time data of moving targets in video frames, collects the light intensity and occlusion data corresponding to the coordinate positions, and records the acquisition results as two environmental factors: light factor and occlusion factor, and obtains the target behavior environmental factor data set. The environmental factor normalization submodule performs normalization processing on the illumination factor and occlusion factor data in the target behavior environmental factor data group, establishes a correspondence between the normalized results and the target coordinate position, calculates the average of the normalized illumination value and the normalized occlusion value as the behavior activity value, and generates the target behavior activity set.

[0008] Preferably, the behavior sequence modeling module includes: The activity value extraction submodule obtains the activity value and corresponding coordinate data of the target behavior activity set, identifies the temporal order of all target behaviors based on the timestamp information, calls the target behavior coordinate set, calculates and sorts the temporal intervals of target behaviors based on the adjacent time threshold, and generates an adjacent target behavior interval sorting sequence. The difference rate calculation submodule calculates the difference rate of behavioral activity values ​​between two adjacent target behaviors based on the adjacent target behavior interval sorting sequence, and integrates them to generate a behavioral activity value difference rate sequence. The consistency recognition submodule extracts the direction of illumination change and the direction of occlusion change between adjacent target behaviors based on the behavior activity value difference rate sequence. It classifies and labels each pair of target behaviors according to whether the change trends in the two directions are consistent. It records and groups the segments with consistent directions and conflicting directions respectively to obtain the behavior consistency partition label set.

[0009] Preferably, the environment mapping module includes: The environmental sequence extraction submodule filters out segments marked as having consistent behavior based on the behavior consistency partition label set, detects the illumination data and occlusion data within each target behavior time period, arranges them in chronological order to form an illumination time series and an occlusion time series, and generates an environmental change time series set. The impact superposition calculation submodule calculates the rate of change of illumination and the rate of change of occlusion between consecutive time nodes in the time series of each target behavior based on the environmental change time series. The rate of change of illumination and the rate of change of occlusion are compared side by side under the same behavior type conditions. The numerical relationship between the change magnitude under the behavior type is identified by joint analysis of the two rate indicators. The impact value sequence of each target behavior is integrated to establish the behavior impact superposition analysis result.

[0010] Preferably, the anomaly screening module includes: The record extraction submodule, based on the results of the behavior impact superposition analysis, filters out target behaviors whose behavior response values ​​are greater than the average response benchmark value, as well as target behaviors in the behavior conflict zone, extracts continuous behavior records of target behaviors in chronological order, collects behavior type and behavior area data corresponding to each time node, and generates a continuous behavior record set. The type-region ratio calculation submodule calls the continuous behavior record set to extract the type value and region of the target behavior at two consecutive time nodes, calculates the type change ratio and behavior region change ratio respectively, and integrates them into a type change ratio sequence and a behavior region change ratio sequence to establish a behavior fluctuation change dataset. The anomaly identification submodule extracts illumination amplitude data and occlusion fluctuation data for the corresponding time period based on the behavior fluctuation change dataset, determines whether the type change ratio and the area change ratio both exceed the set fluctuation identification threshold, determines whether illumination amplitude and occlusion fluctuation both exceed the anomaly judgment threshold, marks the time nodes that meet the conditions as anomaly points, and generates a set of behavior anomaly points.

[0011] Preferably, the risk output module includes: The joint indicator judgment submodule obtains all points in the set of abnormal behavior points and their corresponding coordinates and identification information, calculates the joint risk judgment value, and establishes a joint risk judgment value sequence. The anomaly output processing submodule, based on the joint risk judgment value sequence, filters out points whose behavior response value is greater than the behavior response risk threshold, whose behavior activity value is lower than the activity benchmark value, and whose behavior consistency label is a conflict segment. It extracts the corresponding point number, location identifier, and partition, marks them as detected anomalies with a risk of spread, and outputs the points that meet the joint conditions in a structured format to generate behavior detection and risk warning results.

[0012] Preferably, the system further includes a behavior knowledge graph construction module, which extracts behavior terminology text based on the target behavior active set, extracts word morpheme indexes through word segmentation, determines the first position and frequency of keywords, maps and generates word morpheme arrangement sequences, and constructs a template for the original word order sequence of behavior terms. The graph path generation module sorts the morphemes based on the original word order sequence template of the behavioral terms, establishes a directed path of morphemes from the top layer to the bottom layer according to the sorting result, collects the connection node numbers and adjacent node relationships of all edges in the path, and constructs the behavioral graph path structure. The path intersection analysis module extracts the path node sequence of behavioral terms based on the path structure of the behavioral graph, compares the intersection nodes and counts the frequency of the endpoint nodes, filters the intersection paths, and obtains the set of behavioral term intersection paths. The semantic classification determination module collects semantic tags from the endpoint nodes in the cross-path set of behavioral terms, sorts them by frequency of occurrence, and matches the endpoint tags to obtain the semantic classification tag group of behavioral terms. The classification structure output module, based on the semantic affiliation tag group of the behavioral terms, counts the graph classification nodes to which each tag belongs, divides the behavioral term paths under the corresponding nodes, establishes the classification affiliation relationship structure between nodes and behavioral term paths, and generates a behavioral classification structure table.

[0013] Preferably, the system further includes a digital twin simulation module, which constructs a digital twin model of the monitoring scene, collects real-time scene data through sensors deployed at the monitoring site, and synchronizes the real-time data to the digital twin model; Simulate actual behavioral states in digital twin models to predict behavioral trends and potential anomalies; Set up an intelligent control strategy. The control strategy combines the degree of mutual influence between real-time data and the execution conditions of control action priority to determine whether there is a logical conflict in the rule engine. If there is, the control strategy is optimized. The system automatically performs tracking, alarm, and recording operations based on the optimized control strategy, and records control behavior and feedback data in real time, continuously adjusting the digital twin model based on the recorded feedback data.

[0014] Preferably, the present invention also includes an intelligent behavior analysis method for surveillance cameras, comprising all modules and method flows of the aforementioned intelligent behavior analysis system for surveillance cameras.

[0015] Compared with the prior art, the beneficial effects of the present invention are: In the video data acquisition phase, the system not only acquires video stream data from surveillance cameras and extracts video frame sequences and timestamps, but also actively identifies moving targets and their location coordinates within the video frames. By associating target behavior segments and calculating behavioral activity values, it generates a set of target behavioral activity. This process changes the current situation where existing technologies can only simply identify target locations but cannot quantify the degree of behavioral activity, transforming target behavior from "fuzzy and discernible" to "precisely measurable." Whether it's monitoring pedestrian flow in commercial venues or monitoring equipment operation in industrial parks, managers can intuitively grasp the behavioral status of targets through behavioral activity values, such as the density of people gathering and the frequency of equipment operation, thereby quickly judging the basic situation in the area and providing a clear initial basis for subsequent management decisions. The behavior sequence modeling module extracts behavior activity values ​​and corresponding coordinates, sorts adjacent behavior segments chronologically, and marks consistent and conflicting segments to form a behavior consistency partitioning annotation set. This solves the problem of insufficient behavior sequence correlation analysis in existing technologies. In traffic monitoring scenarios, the system can clearly distinguish consistent segments of normal vehicle driving from conflicting segments such as illegal lane changes and running red lights. It can quickly locate potential traffic violation segments without requiring manual frame-by-frame video review, significantly improving violation identification efficiency. In residential community security monitoring, the system can accurately mark consistent segments that conform to normal routes and conflicting segments that deviate from normal paths or linger, helping security personnel focus on suspicious behavior and reducing ineffective monitoring time. The environment mapping module addresses the pain point of environmental interference affecting the accuracy of behavior analysis. It acquires target behaviors within consistent behavioral segments, extracts time series data on illumination and occlusion, and compares the magnitude of changes in behavior type to assess the intensity of behavioral impact under environmental interference, generating a superimposed analysis of behavioral impacts. In low-light scenarios such as nighttime or rainy weather, the system can identify the degree to which changes in illumination affect pedestrian and vehicle behavior recognition, avoiding misclassification of normal behavior as abnormal. When temporary obstructions appear in the monitoring image, the system can analyze the correlation between the occlusion time series and the target behavior to determine whether the occlusion caused behavioral recognition deviations, thereby correcting the analysis results. This allows the system to maintain stable and accurate behavior analysis capabilities even under complex and changing environmental conditions, breaking the limitations of existing technologies constrained by environmental factors. The anomaly screening module identifies anomalous points in the behavioral impact overlay analysis results where the behavioral response value is greater than the average response benchmark value and is located in a conflict zone, forming a set of behavioral anomaly points. This significantly improves the accuracy of anomaly behavior identification. Existing technologies often lead to misjudgments of anomalies due to a single threshold judgment. However, this system combines behavioral response value and zone attributes for dual screening. It not only excludes normal behaviors within the same zone that have high response values ​​due to environmental interference, but also accurately captures truly abnormal behaviors within conflict zones. For example, in shopping mall pedestrian flow monitoring, the system will not misjudge normal peak pedestrian flow during holidays as anomalies, but can accurately identify abnormal behaviors such as disorderly crowding in conflict zones during off-peak hours. In factory production workshop monitoring, it can exclude brief activity caused by normal equipment maintenance and accurately identify abnormal equipment operation, avoiding "missed" and "false" anomaly identification. The risk output module acquires all locations and information of anomaly points, marks points with a risk of spreading, and generates behavior detection and risk warning results, giving the monitoring system proactive prevention capabilities. Existing technologies can only mark anomaly points and cannot assess the likelihood of risk spread, while this system not only informs management "where the anomaly is," but also indicates "whether the anomaly will spread." In crowded scenarios, after the system marks gathering points with a risk of spread, management personnel can allocate personnel in advance to guide the flow of people and prevent accidents such as crowding and stampedes. In industrial parks, if a piece of equipment malfunctions and poses a risk of spread, the system can issue a timely warning, helping maintenance personnel take measures before the risk escalates. This shifts the monitoring system from "post-event tracing" to "pre-event warning," truly leveraging the proactive defense role of security monitoring and providing more comprehensive protection for security management in various scenarios. Attached Figure Description

[0016] Figure 1 This is a timing diagram of the intelligent behavior analysis system for surveillance cameras described in this invention. Figure 2 A flowchart illustrating the composition of the system's various sets and results; Figure 3 A flowchart illustrating how the behavior sequence modeling module works. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1This invention provides an intelligent behavior analysis system and method for surveillance cameras. The system includes: a video data acquisition module, a behavior sequence modeling module, an environment mapping module, an anomaly screening module, and a risk output module.

[0019] The video data acquisition module acquires video stream data from surveillance cameras, extracts video frame sequences and timestamps, identifies moving targets and their coordinates in the video frames, associates target behavior segments and calculates corresponding behavior activity values, generating a target behavior activity set. The behavior sequence modeling module extracts the behavior activity values ​​and corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, marks consistent and conflicting segments in adjacent behavior segments, and obtains a behavior consistency partition annotation set. The environment mapping module acquires target behaviors located in the behavior consistency partition annotation set within the behavior consistency segment, extracts illumination and occlusion time series, compares the change amplitude with the behavior type, assesses the intensity of behavior influence under environmental interference, and generates behavior influence superposition analysis results. The anomaly screening module identifies anomalous points in the behavior influence superposition analysis results where the behavior response value is greater than the average response benchmark value and is located in the behavior conflict segment, forming a behavior anomaly point set. The risk output module acquires all points in the behavior anomaly point set and their corresponding point information, marks points with a risk of spread, and generates behavior detection and risk warning results.

[0020] Example 1: See Figure 2 The construction of the target behavior active set involves the integration and processing of multiple data dimensions. The video data acquisition module obtains raw video stream data from the surveillance camera. This data contains continuous image frames and corresponding timestamp information. The moving object detection algorithm identifies moving objects in each frame and records their pixel coordinates. The system associates the detection results of the same target in consecutive frames to form behavior segments that change over time. Each behavior segment calculates a behavior activity value, which is obtained by averaging two environmental factors: the light intensity and the degree of occlusion in the target's area after normalization. The final target behavior active set is a structured data set, in which each record contains the timestamp of the behavior, spatial coordinates, normalized light value, normalized occlusion value, and the calculated behavior activity value. The generation of the behavior consistency partition annotation set relies on the comparative analysis of the change trends of adjacent behavior segments. The system extracts the behavior activity values ​​and their corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, and calculates the behavior activity value difference rate between each pair of adjacent segments. This difference rate reflects the magnitude of the change in activity level. At the same time, the system analyzes the consistency of the illumination change direction and occlusion change direction between adjacent segments. If the change trends in the two directions are the same, they are marked as behavior consistency segments; if the trends are opposite, they are marked as behavior conflict segments. This process generates a behavior consistency partition annotation set containing segment annotations and difference rate data.

[0021] The behavioral impact superposition analysis results focus on assessing the interference of environmental factors on behavioral patterns. The system selects segments marked as behaviorally consistent from the behavioral consistency partition label set, extracts the illumination and occlusion data for each target behavior within these segments, and arranges them in chronological order to form illumination time series and occlusion time series. By calculating the rate of change between consecutive time nodes in the series, the illumination change rate and occlusion change rate are obtained respectively. Under the constraint of the same behavior type, the system compares the two types of rate indicators side by side and analyzes their joint impact on the magnitude of behavioral change. The final behavioral impact superposition analysis results include comparative data of behavioral responses under each type of environmental interference condition. The identification of behavioral anomaly sets is based on a multi-threshold judgment logic. The system filters out two types of target behaviors from the behavioral impact superposition analysis results: records whose behavioral response values ​​exceed the average response benchmark value, and records that are in behavioral conflict zones. For these target behaviors, the system extracts behavioral type and regional data at continuous time nodes, calculates the type change ratio and regional change ratio of adjacent time nodes, and simultaneously acquires the illumination amplitude and occlusion fluctuation data for the corresponding time period. When the type change ratio and regional change ratio both exceed the preset fluctuation identification threshold, and the illumination amplitude and occlusion fluctuation also exceed the anomaly judgment threshold, the system marks the corresponding time node as an anomaly point. All these anomaly points together constitute a behavioral anomaly set, which records the spatial location, illumination occlusion amplitude characteristics, behavioral type, and area fluctuation ratio of each anomaly point.

[0022] The generation of behavior detection and risk warning results is the final output of the system. The system acquires information on all points in the set of abnormal behavior points, including coordinates and identification data, and calculates a joint risk judgment value for each point to form a sequence. Based on this sequence, points that simultaneously meet three conditions are further selected: the behavior response value is higher than the behavior response risk threshold, the behavior activity value is lower than the activity benchmark value, and the behavior consistency label is a conflict segment. These points are marked as detection anomalies with a risk of spread. The system extracts their numbers, location identifiers, and partition information, organizes them in a structured format, and outputs them. The final generated behavior detection and risk warning results include a detailed list of anomaly points and a joint judgment label for each point based on three indicators. In the process of converting the raw video stream into the final risk warning, the system gradually refines and identifies behavioral patterns with risk characteristics through a series of processing steps, such as environmental factor normalization, trend consistency analysis, environmental interference quantitative assessment, and multi-threshold anomaly detection. This progressive analysis method can effectively integrate spatiotemporal information and environmental factors.

[0023] Example 2: See Figure 3The submodule continuously receives real-time video streams from surveillance cameras. The video stream consists of a series of image frames with precise timestamps. Each frame is processed by a built-in motion detection algorithm to identify moving targets in the scene and record their pixel coordinates and the time of their actions. At the same time, the system collects environmental data corresponding to each coordinate point, mainly light intensity and occlusion degree. These two types of data are recorded as light factor and occlusion factor, respectively. Together with the target's spatial coordinates and time information, they constitute the target behavior environmental factor data set. This data set is transmitted to the subsequent processing unit as the basic dataset. After receiving the target behavior environmental factor data set, the environmental factor normalization submodule starts the processing flow. This module reads the raw values ​​of the illumination factor and occlusion factor respectively. These values ​​may come from different sensors and have different dimensions, so they need to be normalized to eliminate the difference in dimensions. The normalization process linearly transforms the raw data into the range of [0,1], making environmental data from different sources comparable. The normalized illumination value and normalized occlusion value are re-mapped to the target coordinate position. The system calculates the arithmetic mean of these two normalized values ​​as the behavior activity value. All data points processed in the above way are arranged in chronological order to finally form a structured target behavior activity set.

[0024] The activity value extraction submodule in the behavior sequence modeling module begins operation. It reads the activity values ​​and their corresponding coordinate data from the target behavior activity set, identifies the chronological order of all target behaviors on the timeline by combining timestamp information, calls the stored target behavior coordinate set, and calculates the interval between temporally adjacent target behaviors based on a preset adjacent time threshold. All target behaviors are then sorted according to chronological order, generating a temporally ordered sequence of adjacent target behavior intervals. The difference rate calculation submodule performs calculations based on the adjacent target behavior interval order sequence. This module iterates through each pair of adjacent target behavior records in the sequence, calculating the difference rate of their activity values. The difference rate is calculated by subtracting the previous activity value from the subsequent activity value and then dividing by the previous activity value, yielding a percentage value representing the relative degree of change. The calculation results for all adjacent behavior pairs are integrated into a complete activity value difference rate sequence, which quantitatively describes the fluctuation of behavior activity over time.

[0025] The consistency recognition submodule receives the behavior activity value difference rate sequence as input. This module further analyzes the environmental change direction characteristics between each pair of adjacent target behaviors, extracts the illumination change direction and occlusion change direction corresponding to adjacent behaviors from the raw data, and determines whether the change trend of each pair of behaviors in these two directions is consistent, i.e., simultaneously enhanced, simultaneously weakened, or opposite. Based on the judgment result, each pair of behaviors is marked as consistent in direction or conflicting in direction. The system groups and records all behavior pairs. Those with consistent directions are classified into the behavior consistency segment, and those with conflicting directions are classified into the behavior conflict segment. Finally, the system outputs a behavior consistency partition label set with classification labels.

[0026] Taking a large parking lot monitoring system as an example, the video data acquisition module starts operating, and the monitoring cameras continuously capture video stream data within the parking lot. The video information acquisition submodule receives these video streams, extracts video frame sequences with millisecond-level timestamps, identifies moving targets in the video frames using a moving target detection algorithm, including moving vehicles and pedestrians, and records the pixel coordinates of each target. Simultaneously, the system collects environmental data corresponding to each coordinate position: it uses a photosensor to acquire light intensity data and uses an image analysis algorithm to calculate occlusion data. These data are recorded as light factors and occlusion factors, which, together with the target coordinates and timestamps, constitute the target behavior environmental factor data set. The environmental factor normalization submodule processes the target behavior environmental factor data set. This module reads the original values ​​of the illumination factor (ranging from 800 to 1200 lux) and the original values ​​of the occlusion factor (ranging from 0 to 100%), and performs normalization processing to transform them into the numerical range of [0,1]. The normalized illumination and occlusion values ​​are mapped to the corresponding target coordinates. The system calculates the arithmetic mean of the normalized illumination value and the normalized occlusion value at each location point to obtain the behavioral activity value at that location. All processed data points are organized in chronological order to generate a complete target behavior activity set containing timestamps, coordinates, normalized environmental factors, and behavioral activity values.

[0027] The activity value extraction submodule in the behavior sequence modeling module begins operation. This module reads all data records from the target behavior activity set, identifies the temporal order of each target behavior based on timestamp information, and calculates and sorts the temporally adjacent target behaviors using 30-second intervals as the adjacent time threshold. This generates a temporally ordered sequence of adjacent target behavior intervals, which accurately reflects the temporal relationship of each target behavior within the parking lot. The difference rate calculation submodule performs calculations based on the adjacent target behavior interval order sequence. This module iterates through each pair of adjacent target behavior records in the sequence, calculating the relative change rate of the activity value of the subsequent behavior compared to the activity value of the previous one. For example, for two adjacent behaviors at timestamps T1 and T2, the activity values ​​are 0.65 and 0.72 respectively, and the difference rate is (0.72-0.65) / 0.65=10.77%. The difference rate calculation results for all adjacent behavior pairs are integrated into a complete behavior activity value difference rate sequence, which quantitatively describes the temporal change characteristics of the activity level within the parking lot. The consistency identification submodule analyzes the behavioral activity value difference rate sequence. This module extracts the direction of light change and the direction of occlusion change corresponding to each pair of adjacent target behaviors. For example, in a pair of adjacent behaviors, the light increases from 850 lux to 920 lux (direction is increase), and the occlusion decreases from 30% to 25% (direction is decrease). The system determines that the change directions of these two environmental factors are inconsistent and marks them as direction conflict segments. When the change directions of the two environmental factors are consistent, they are marked as direction consistent segments. The marking results of all segments finally form a behavioral consistency partition label set.

[0028] Example 3: The submodule receives the behavior consistency partition label set from the preceding module, filters out all data segments marked as behavior consistency, and for each target behavior within these consistent segments, the system extracts the light intensity data and occlusion degree data collected within its complete time period, arranges them in chronological order to form light time series and occlusion time series respectively. These time series data together constitute the environmental change time series set. The influence superposition calculation submodule conducts in-depth analysis based on the environmental change time series set. This module calculates the rate of change between consecutive time nodes within the time period of each target behavior. For the light time series, it calculates the rate of light change between adjacent time nodes; for the occlusion time series, it similarly calculates the rate of occlusion change between adjacent time nodes. Under the constraint of the same behavior type, the two types of rate indicators are compared and analyzed side by side. Through numerical relationship identification methods, the combined influence of the two types of environmental factors on the magnitude of behavior change is analyzed. The final behavior influence superposition analysis result includes detailed data on the degree of influence of light change rate, the degree of influence of occlusion change rate, and the comparison data of behavior response under various environmental interference conditions. The record extraction submodule in the anomaly filtering module begins to process the results of the behavior impact superposition analysis. Based on preset filtering conditions, this submodule extracts two types of specific target behaviors from the analysis results: records whose behavior response values ​​exceed the average response benchmark value calculated by the system, and records that are in the behavior conflict zone. For these target behaviors, continuous behavior records are extracted in chronological order, and the specific behavior type identifier and behavior area range data corresponding to each time node are collected. All these extracted data are organized into a structured continuous behavior record set.

[0029] The type-region ratio calculation submodule receives and processes a continuous set of behavior records. For each target behavior in the record set, this module extracts its type identifier value and region range data at two consecutive time points, and calculates the type change ratio, expressed as: ;in: Indicates the ratio of type changes. This indicates the type value of the current time node. This indicates the type value of the previous time node. The function takes the larger of the two values ​​and calculates the behavior region change ratio. A similar method is used to calculate the proportional relationship of regional changes between consecutive time nodes. All calculation results are integrated into a type change ratio sequence and a behavior region change ratio sequence, which together constitute the behavior fluctuation change dataset.

[0030] The anomaly identification submodule makes the final judgment based on the behavioral fluctuation change dataset. This module extracts the illumination amplitude data and occlusion fluctuation data corresponding to each time point from the dataset, sets two sets of threshold values: a fluctuation identification threshold and an anomaly judgment threshold. It judges whether the type change ratio and the area change ratio both exceed the fluctuation identification threshold, and simultaneously judges whether both illumination amplitude and occlusion fluctuation exceed the anomaly judgment threshold. When a time point meets both of these conditions, the system marks it as an anomaly. All marked anomalies are collected to form the final behavioral anomaly point set, which is the output of this module. Starting from the extraction of environmental time series data, through change rate calculation and impact degree analysis, to multi-condition record screening and change ratio calculation, the anomaly point identification is finally completed through multiple threshold judgments.

[0031] Example 4: The implementation process of the risk output module begins. The joint indicator judgment submodule receives a set of abnormal behavior points from the preceding module. This set contains multiple data points marked as abnormal. Each data point includes its spatial coordinates, unique identifier, and related technical parameters. The submodule reads detailed data from these points, including the behavioral response value, behavioral activity value, and behavioral consistency label for each point. Based on these parameters, a composite indicator—the joint risk judgment value—is calculated. The calculation of the joint risk judgment value adopts a multi-parameter weighted fusion method, where the weight coefficient for the behavioral response value is set to 0.5, the weight coefficient for the behavioral activity value is 0.3, and the weight coefficient for the behavioral consistency label is 0.2. The weight coefficients are pre-set according to the importance of each parameter in the actual application scenario. The calculated joint risk judgment values ​​are arranged in chronological order to form a joint risk judgment value sequence, which reflects the comprehensive risk level of each abnormal point.

[0032] The anomaly output processing submodule further processes the data based on the joint risk assessment value sequence. This module sets three filtering conditions: the behavior response value must be greater than the preset behavior response risk threshold of 0.7, the behavior activity value must be lower than the activity benchmark value of 0.4, and the behavior consistency label must be a conflict zone. The system traverses all points in the sequence and specially marks points that simultaneously meet these three conditions, identifying these points as anomalies with a risk of spreading. The system extracts complete information from the marked points, including the point number, spatial coordinates, the monitoring area partition identifier, and the specific values ​​of various technical parameters. All this information is organized according to a prescribed structured format to generate the final behavior detection and risk warning results, which are output in the form of a data table. See Table 1 for the detection results of anomalies in the monitoring area.

[0033] Table 1: Results of Anomaly Detection

[0034] In a specific application scenario, taking parking lot vehicle monitoring as an example, the system detected multiple abnormal behavior points. Point P-001 is located in the northeast corner of the parking lot. This location recorded a high behavioral response value of 0.85, indicating that the target behavior at this location is highly sensitive to environmental changes. Simultaneously, its behavioral activity value of 0.32 is lower than the baseline value of 0.4, showing a low level of behavioral activity. Combined with its characteristic of being marked as a conflict zone, this point was determined to be a high-risk point. Point P-003 is located in the southwest area of ​​the parking lot. It detected a high behavioral response value of 0.92, and its behavioral activity value of 0.29 is significantly lower than the baseline value. The consistency label indicates a conflict zone. These characteristics indicate significant abnormal behavior at this location. The system calculated its combined risk value to be 0.81, classifying it as a high-risk level.

[0035] Location P-005, situated in the southeast corner of the parking lot, has the highest behavioral response value (0.95) and the lowest behavioral activity value (0.26) among all locations. The labeling of the conflict zone confirms the inconsistency in behavioral patterns. These characteristics result in a combined risk value of 0.83 for this location, making it a high-risk location requiring close monitoring. For location P-004, although its behavioral response value (0.68) is slightly below the risk threshold and its behavioral activity value (0.41) is close to the baseline, it is classified as a medium-risk location because it is located in a consistent zone and does not meet all risk assessment criteria. While the risk level of such locations is relatively low, monitoring is still necessary. The system's output of behavioral detection and risk warning results not only includes detailed technical parameters for each location but also provides a risk level assessment based on a comprehensive multi-indicator judgment. This structured output format facilitates monitoring personnel in quickly identifying high-risk areas and taking timely warning and response measures.

[0036] Example 5: The behavior knowledge graph construction module begins processing behavior terminology data from the active set of target behaviors. This module extracts all textual information describing behaviors from the set, including terminology descriptions such as behavior type and behavior characteristics. It performs word segmentation on these terminology texts, decomposing them into basic morpheme units. Each morpheme is assigned a unique index identifier. The system records the first occurrence position of each keyword in the text and its frequency throughout the text. Based on the morpheme index and position information, an ordered sequence of morphemes is generated, constructing the original word order sequence template for behavior terms. The graph path generation module receives the original word order sequence template as input. This module sorts all morpheme units in the template, arranging them from general to sequential. Based on the sorting result, a directed path is established from the top-level concept to the bottom-level specific concept. During path construction, the node number information of all connecting edges is collected, and the semantic relationship type between adjacent nodes is recorded, such as "belongs to," "includes," and "leads to." Finally, a complete behavior graph path structure is constructed, which clarifies the hierarchical and associative relationships between concepts.

[0037] The path cross-analysis module operates based on the path structure of the behavior graph. This module extracts the complete path node sequence of all behavior terms, compares the node intersections between different paths, and statistically analyzes the frequency and distribution of the endpoint nodes of each path. By setting a path cross-degree threshold, it filters out path combinations with significant cross-features. These cross-paths indicate strong semantic associations between different behavior terms. All filtered cross-paths are compiled into a behavior term cross-path set, providing data support for subsequent semantic analysis. The semantic classification determination module processes the endpoint node data in the behavior term cross-path set. This module collects corresponding semantic label information for each endpoint node. These labels describe the core semantic features of the node and are sorted according to the frequency of the node's appearance in the path. High-frequency nodes are matched to the path endpoints first. A label matching algorithm is used to assign appropriate semantic labels to each endpoint node, ultimately obtaining a semantic classification label group for each behavior term. This label group clarifies the semantic category of each behavior term.

[0038] The classification structure output module outputs a final structured result based on the semantic attribution label groups of behavioral terms. This module statistically analyzes the graph classification nodes to which each semantic label belongs, assigns behavioral term paths with the same semantic attribution to the corresponding classification nodes, establishes a mapping relationship between classification nodes and behavioral term paths, forming a complete classification attribution structure. The final output behavioral classification structure table displays the classification results of all behavioral terms according to semantic features. The digital twin simulation module constructs a virtual digital model of the monitoring scene. This module collects real-time environmental data, including parameters such as video stream, temperature, humidity, and light intensity, through various sensor devices deployed at the monitoring site. The collected real-time data is synchronized to the digital twin model through a data interface, reproducing the real-time state of the actual monitoring scene in the virtual model.

[0039] The digital twin model simulates and extrapolates actual behavioral states. Based on current behavioral data and historical behavioral patterns, this module uses predictive algorithms to calculate future trends in behavior. By establishing a behavioral evolution model, it predicts potential abnormal behavioral patterns, enabling the system to identify potential risky behaviors in advance and providing data support for risk warnings. The intelligent control strategy module is responsible for building the rule engine. This module analyzes the interrelationships between various real-time data, determines the priority execution conditions for different control actions, checks for potential logical conflicts in the rule engine, and initiates optimization algorithms when rule conflicts are detected to adjust and optimize the control strategy, ensuring the coordination and effectiveness of the rule system. Based on the optimized control strategy, the system automatically executes monitoring operations, automatically tracking target behavior, generating early warning and alarm information, and recording behavioral data. All control operations and system feedback are recorded in real time to form log data. Based on this feedback data, the parameter settings of the digital twin model are continuously adjusted and optimized, enabling the model to better reflect changes in the actual scenario.

[0040] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0041] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent behavior analysis system for surveillance cameras, characterized in that, The system includes: The video data acquisition module acquires video stream data from the surveillance camera, extracts video frame sequences and timestamps, identifies moving targets and their position coordinates in the video frames, associates target behavior segments and calculates the corresponding behavior activity values, and generates a target behavior activity set. The behavior sequence modeling module extracts the behavior activity value and corresponding coordinates from the target behavior activity set, sorts adjacent behavior segments in chronological order, marks consistent and conflicting segments in adjacent behavior segments, and obtains a behavior consistency partition label set. The environment mapping module obtains the target behaviors located in the behavior consistency segment of the behavior consistency partition annotation set, extracts the illumination and occlusion time series, compares the change amplitude with the behavior type, evaluates the intensity of behavior influence under environmental interference, and generates behavior influence superposition analysis results. The anomaly screening module identifies anomalous points in the target behaviors whose behavior response values ​​are greater than the average response baseline value and are located in the behavior conflict zone in the behavior impact superposition analysis results, forming a set of behavior anomaly points. The risk output module obtains all points and corresponding point information in the set of abnormal behavior points, marks points with the risk of spread, and generates behavior detection and risk warning results.

2. The intelligent behavior analysis system for surveillance cameras according to claim 1, characterized in that, The target active behavior set includes active behavior value, target spatial coordinates, and normalized environmental factors. The behavior consistency partitioning label set specifically includes consistent behavior segment labels, conflict behavior segment labels, and the difference rate of adjacent active behavior values. The behavior influence superposition analysis results include the influence of illumination change rate on behavior, the influence of occlusion change rate on behavior, and the comparison of behavior response under each environmental interference condition. The behavior anomaly point set includes the spatial location of anomaly points, illumination occlusion amplitude characteristics of anomaly points, and the behavior type and area fluctuation ratio of anomaly points. The behavior detection and risk warning results include a list of detected anomaly points and a joint judgment label of the three indicators of anomaly points.

3. The intelligent behavior analysis system for surveillance cameras according to claim 1, characterized in that, The video data acquisition module includes: The video information acquisition submodule acquires video stream data and timestamps from surveillance cameras, extracts the coordinates and behavior time data of moving targets in video frames, collects the light intensity and occlusion data corresponding to the coordinate positions, and records the acquisition results as two environmental factors: light factor and occlusion factor, and obtains the target behavior environmental factor data set. The environmental factor normalization submodule performs normalization processing on the illumination factor and occlusion factor data in the target behavior environmental factor data group, establishes a correspondence between the normalized results and the target coordinate position, calculates the average of the normalized illumination value and the normalized occlusion value as the behavior activity value, and generates the target behavior activity set.

4. The intelligent behavior analysis system for surveillance cameras according to claim 3, characterized in that, The behavior sequence modeling module includes: The activity value extraction submodule obtains the activity value and corresponding coordinate data of the target behavior activity set, identifies the temporal order of all target behaviors based on the timestamp information, calls the target behavior coordinate set, calculates and sorts the temporal intervals of target behaviors based on the adjacent time threshold, and generates an adjacent target behavior interval sorting sequence. The difference rate calculation submodule calculates the difference rate of behavioral activity values ​​between two adjacent target behaviors based on the adjacent target behavior interval sorting sequence, and integrates them to generate a behavioral activity value difference rate sequence. The consistency recognition submodule extracts the direction of illumination change and the direction of occlusion change between adjacent target behaviors based on the behavior activity value difference rate sequence. It classifies and labels each pair of target behaviors according to whether the change trends in the two directions are consistent. It records and groups the segments with consistent directions and conflicting directions respectively to obtain the behavior consistency partition label set.

5. The intelligent behavior analysis system for surveillance cameras according to claim 4, characterized in that, The environment mapping module includes: The environmental sequence extraction submodule filters out segments marked as having consistent behavior based on the behavior consistency partition label set, detects the illumination data and occlusion data within each target behavior time period, arranges them in chronological order to form an illumination time series and an occlusion time series, and generates an environmental change time series set. The impact superposition calculation submodule calculates the rate of change of illumination and the rate of change of occlusion between consecutive time nodes in the time series of each target behavior based on the environmental change time series. The rate of change of illumination and the rate of change of occlusion are compared side by side under the same behavior type conditions. The numerical relationship between the change magnitude under the behavior type is identified by joint analysis of the two rate indicators. The impact value sequence of each target behavior is integrated to establish the behavior impact superposition analysis result.

6. The intelligent behavior analysis system for surveillance cameras according to claim 5, characterized in that, The anomaly filtering module includes: The record extraction submodule, based on the results of the behavior impact superposition analysis, filters out target behaviors whose behavior response values ​​are greater than the average response benchmark value, as well as target behaviors in the behavior conflict zone, extracts continuous behavior records of target behaviors in chronological order, collects behavior type and behavior area data corresponding to each time node, and generates a continuous behavior record set. The type-region ratio calculation submodule calls the continuous behavior record set to extract the type value and region of the target behavior at two consecutive time nodes, calculates the type change ratio and behavior region change ratio respectively, and integrates them into a type change ratio sequence and a behavior region change ratio sequence to establish a behavior fluctuation change dataset. The anomaly identification submodule extracts illumination amplitude data and occlusion fluctuation data for the corresponding time period based on the behavior fluctuation change dataset, determines whether the type change ratio and the area change ratio both exceed the set fluctuation identification threshold, determines whether illumination amplitude and occlusion fluctuation both exceed the anomaly judgment threshold, marks the time nodes that meet the conditions as anomaly points, and generates a set of behavior anomaly points.

7. The intelligent behavior analysis system for surveillance cameras according to claim 6, characterized in that, The risk output module includes: The joint indicator judgment submodule obtains all points in the set of abnormal behavior points and their corresponding coordinates and identification information, calculates the joint risk judgment value, and establishes a joint risk judgment value sequence. The anomaly output processing submodule, based on the joint risk judgment value sequence, filters out points whose behavior response value is greater than the behavior response risk threshold, whose behavior activity value is lower than the activity benchmark value, and whose behavior consistency label is a conflict segment. It extracts the corresponding point number, location identifier, and partition, marks them as detected anomalies with a risk of spread, and outputs the points that meet the joint conditions in a structured format to generate behavior detection and risk warning results.

8. The intelligent behavior analysis system for surveillance cameras according to claim 1, characterized in that, The system also includes a behavior knowledge graph construction module, which extracts behavior term text based on the target behavior active set, extracts word morpheme index by word segmentation, determines the first position and frequency of keywords, maps and generates word morpheme arrangement sequence, and constructs a template for the original word order sequence of behavior terms. The graph path generation module sorts the morphemes based on the original word order sequence template of the behavioral terms, establishes a directed path of morphemes from the top layer to the bottom layer according to the sorting result, collects the connection node numbers and adjacent node relationships of all edges in the path, and constructs the behavioral graph path structure. The path intersection analysis module extracts the path node sequence of behavioral terms based on the path structure of the behavioral graph, compares the intersection nodes and counts the frequency of the endpoint nodes, filters the intersection paths, and obtains the set of behavioral term intersection paths. The semantic classification determination module collects semantic tags from the endpoint nodes in the cross-path set of behavioral terms, sorts them by frequency of occurrence, and matches the endpoint tags to obtain the semantic classification tag group of behavioral terms. The classification structure output module, based on the semantic affiliation tag group of the behavioral terms, counts the graph classification nodes to which each tag belongs, divides the behavioral term paths under the corresponding nodes, establishes the classification affiliation relationship structure between nodes and behavioral term paths, and generates a behavioral classification structure table.

9. The intelligent behavior analysis system for surveillance cameras according to claim 8, characterized in that, The system also includes a digital twin simulation module, which constructs a digital twin model of the monitoring scene, collects real-time scene data through sensors deployed at the monitoring site, and synchronizes the real-time data to the digital twin model; Simulate actual behavioral states in digital twin models to predict behavioral trends and potential anomalies; Set up an intelligent control strategy. The control strategy combines the degree of mutual influence between real-time data and the execution conditions of control action priority to determine whether there is a logical conflict in the rule engine. If there is, the control strategy is optimized. The system automatically performs tracking, alarm, and recording operations based on the optimized control strategy, and records control behavior and feedback data in real time, continuously adjusting the digital twin model based on the recorded feedback data.

10. A method for intelligent behavior analysis of surveillance cameras, characterized in that, The system includes all modules and method flows of the intelligent behavior analysis system for surveillance cameras as described in any one of claims 1 to 9.

Images