Video content analysis system based on AI

Abstract

The invention relates to the technical field of computer vision, in particular to an AI-based video content analysis system, which comprises a video stream acquisition and detection module, a background interference analysis and filtering module, a target movement track calculation module, a visual angle conversion and track compensation module and an abnormal behavior analysis module. According to the method, the pixel brightness values of continuous frames are extracted, the foreground region recognition accuracy is improved, the brightness change trend is screened to accurately lock a dynamic target, the background change rate is analyzed, similar regions are removed, interference is reduced, and the target edge feature and inter-frame position change are extracted to perform fine-grained trajectory tracking. And track prediction is optimized in combination with the view angle switching time, track changes and speed changes are recorded, the stability and accuracy of the moving target under multiple view angles are improved, abnormal change modes are classified and analyzed, abnormal events are effectively captured, background interference is reduced, and the target detection precision and the abnormal behavior analysis response speed are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an AI-based video content analysis system. Background Art

[0002] The field of computer vision technology involves the cross-application of computer science and image processing technology, and is committed to enabling computers to automatically acquire, analyze and understand information from images and videos. The core content of this field covers image recognition, target detection, posture estimation, video analysis, image generation and other aspects. The research focus of computer vision technology is on how to process and analyze visual data and extract meaningful information to achieve automated data understanding and decision support. With the development of artificial intelligence technology, computer vision has gradually been combined with deep learning, machine learning and other technologies to promote the progress of various visual tasks, such as video content analysis, real-time monitoring, autonomous driving and other fields.

[0003] Among them, the AI-based video content analysis system refers to a system that automatically analyzes and processes the content in the video through artificial intelligence technology. The patent subject focuses on how to efficiently and accurately extract key information from the video for analysis, and solves the problems of low efficiency and poor accuracy of traditional video content analysis methods. The system uses target detection, action recognition, scene analysis and other technologies to automatically identify objects, activities and time series relationships in the video, and perform intelligent classification and labeling. The system uses deep learning models to train and optimize video data, thereby improving the accuracy and real-time performance of the analysis results. The patent subject improves the automated analysis capabilities of video content by integrating multiple computer vision technologies and is suitable for large-scale video surveillance scenarios.

[0004] The existing technology has the problem of difficult to effectively eliminate background interference in video content analysis. The brightness changes of the background and the target in complex environments are similar, which can easily lead to target detection errors. The extraction and tracking of feature points in the moving target trajectory analysis lack precision, resulting in discontinuity or deviation of the trajectory in fast motion or complex trajectory scenes. The motion trajectory prediction under the condition of perspective switching is easily disturbed and the prediction direction is unstable, which affects the target tracking performance in a multi-camera environment. The classification of changes at different time points in abnormal behavior analysis lacks meticulousness and is prone to missing potential abnormal events. The analysis results are easily affected by background interference and trajectory instability, which reduces the reliability and real-time performance of the overall analysis and fails to fully meet the high-demand video surveillance and anomaly detection needs. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose an AI-based video content analysis system.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A video content analysis system based on AI includes: The video stream acquisition and detection module extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change rate in the time dimension, identifies the highlighted area as the foreground area, extracts the brightness change trend in the area, and obtains the video foreground detection result; The background interference analysis and filtering module analyzes the background pixel change rate based on the video foreground detection result, removes the areas with similar change rates, and obtains the target area map after background filtering; The target motion trajectory calculation module extracts target edge feature points based on the target area map after background filtering, records the inter-frame position changes of the feature points, analyzes the motion trajectory and smoothness, screens the trajectory that conforms to the global trend, and obtains the target motion trajectory data; The perspective conversion and trajectory compensation module records the changes in trajectory and speed according to the target motion trajectory data, identifies the moment of perspective switching, adjusts the trajectory prediction direction, and obtains the trajectory prediction result after perspective compensation; The abnormal behavior analysis module determines the trend change characteristics based on the trajectory prediction results after the perspective compensation, classifies and analyzes the deviation types and abnormal change patterns of the differentiated time points, and generates abnormal change analysis results of the video content.

[0007] As a further solution of the present invention, the video foreground detection result includes a highlight area, a brightness change trend of the foreground area, and pixel features of the foreground area; the target area map after background filtering includes a filtered target area boundary, background interference features, and a remaining target area; the target motion trajectory data includes trajectory feature points, inter-frame position information, and motion smoothness; the trajectory prediction result after perspective compensation includes a compensated trajectory path, trajectory speed information, and a perspective switching time point; and the video content abnormal change analysis result includes an abnormal trajectory deviation type, an abnormal change at a time point, and a behavior change pattern.

[0008] As a further solution of the present invention, the video stream acquisition and detection module includes a brightness value extraction submodule, a foreground area recognition submodule, and a foreground change trend screening submodule; The brightness value extraction submodule extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change of adjacent frames, records the brightness change rate, and generates a pixel brightness change rate data set; The foreground area recognition submodule determines whether the brightness change exceeds the brightness fluctuation threshold based on the pixel brightness change rate data set, selects pixels with prominent change rates, and generates a highlight area pixel set; The foreground change trend screening submodule calls the highlight area pixel set to analyze the pixel brightness change trend in the area using the formula: ; Calculate the brightness change trend value of the highlight area, perform trend classification, and generate video foreground detection results; in, Represents the brightness change trend value of the highlight area. Representative Frame pixel brightness value, represents the frame interval time, represents the weight coefficient of the rate of change, Represents the total number of pixels in the area. Indicates the pixel brightness value of the next frame.

[0009] As a further solution of the present invention, the background interference analysis and filtering module includes a background pixel change submodule, a similar target elimination submodule, and a target area construction submodule; The background pixel change submodule calculates the brightness change rate of background pixels in consecutive frames based on the video foreground detection result, calculates the mean value and the change amplitude range, and obtains the video background pixel change characteristics; The similar target elimination submodule analyzes the foreground pixel change rate according to the video background pixel change characteristics, using the formula: ; Calculate the difference between the foreground and background change rates, filter and remove areas with similar change rates, and obtain the background removal target area; in, Represents the difference between the foreground and background change rates. For background The rate at which pixel brightness changes, For the future The rate at which pixel brightness changes, is the mean value of background brightness variation, is the total number of pixels, is the foreground pixel index; The target region construction submodule calls the background-removed target region, reconstructs foreground pixels, marks the target region boundary, and generates a target region map after background filtering.

[0010] As a further solution of the present invention, the target motion trajectory calculation module includes a target area extraction submodule, a target edge extraction submodule, and a motion trajectory screening submodule; The target region extraction submodule extracts dynamic change information in the target region based on the target region map after background filtering, identifies and filters the position change between frames of the target region, and generates the coordinate value of the target region; The target edge extraction submodule extracts edge feature points according to the target area coordinate values, captures the target edge position, records the position change of the edge feature points, and obtains the target edge coordinate data; The motion trajectory screening submodule analyzes and calculates the smoothness value of the target motion trajectory according to the target edge coordinate data, and screens the trajectory that conforms to the global trend using the formula: ; Analyze and optimize the accuracy of the motion trajectory to obtain the target motion trajectory data; in, Represents the smoothness value of the target motion trajectory, and Representative The horizontal and vertical coordinates of the frame feature points, and Represents the horizontal and vertical coordinates of the previous frame. The frame sequence number representing the feature point, Represents the total number of feature points.

[0011] As a further solution of the present invention, the perspective conversion and trajectory compensation module includes a trajectory recording submodule, a perspective switching identification submodule, and a trajectory compensation calculation submodule; The trajectory recording submodule records the change in the target position and the change in speed according to the target motion trajectory data, stores the trajectory data, and obtains the trajectory speed change record; The perspective switching identification submodule calculates the target trajectory mutation point based on the trajectory speed change record, analyzes the trajectory direction adjustment trend, identifies the perspective switching moment, and obtains the perspective switching moment sequence; The trajectory compensation calculation submodule calls the view switching time sequence, analyzes the trajectory error before and after the view conversion, and adjusts the trajectory prediction direction using the formula: ; Calculate the perspective compensation distance, correct the trajectory based on the target trajectory trend, and obtain the trajectory prediction result after perspective compensation; in, Represents the viewing angle compensation distance, Representative moments The spatial location of the target, represents the spatial position of the target at the previous moment, Representative moments The target's movement speed, Indicates the target's moving speed at the previous moment. Represents the time series length of the trajectory record.

[0012] As a further solution of the present invention, the abnormal behavior analysis module includes a trend change calculation submodule, an abnormal deviation classification submodule, and an abnormal pattern recognition submodule; The trend change calculation submodule analyzes the target's motion trend change rate in the time series based on the trajectory prediction result after the perspective compensation, determines the mutation of the motion mode, and obtains the trend change characteristics; The abnormal deviation classification submodule calls the trend change characteristics to identify the deviation angle, position error and speed change of the target at the differentiated time points, using the formula: ; Calculate the abnormal deviation and obtain the deviation type at the differentiated time points; in, represents the degree of abnormal deviation, and Represents adjacent time points and The target location, and Represents adjacent time points and The target speed, and Represents adjacent time points and The target acceleration; The abnormal pattern recognition submodule analyzes the time distribution of the deviation based on the deviation type of the differentiated time point, calculates the time density of the abnormal change, screens the abnormal area, and obtains the analysis result of the abnormal change of the video content.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by extracting the brightness values ​​of pixels in consecutive frames and calculating the brightness change rate in the time dimension, the light intensity change between video frames can be obtained more efficiently, the recognition accuracy of the foreground area can be improved, the brightness change trend in the area can be extracted to realize the accurate locking of the target area in the dynamic scene, the background pixel change rate can be analyzed and the areas with similar change rates can be eliminated, so that the interference under the complex background is significantly reduced, the target edge feature points are extracted and the position change between frames is recorded, the fine-grained tracking of the moving target trajectory is realized, the motion trajectory and smoothness analysis and the global trend trajectory screening are carried out to ensure the continuity and consistency of the target motion path, the trajectory change and speed change are recorded in combination with the recognition of the perspective switching moment, the stability and accuracy of the motion trajectory prediction under the multi-perspective switching condition are significantly improved, the trajectory prediction direction is adjusted to optimize the target tracking performance under different shooting perspectives, the trend change characteristics and the classification analysis of the deviation type make the abnormal behavior recognition more targeted, the abnormal change pattern extraction realizes the accurate capture of abnormal events in complex video content, the overall processing logic reduces background interference, improves the target detection accuracy, enhances the continuity and stability of the motion trajectory through layer-by-layer analysis and screening, realizes efficient trajectory compensation under multi-perspective conditions, and finally improves the accuracy and response speed of abnormal behavior analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a system flow chart of the present invention; Figure 2 This is a flow chart of obtaining the video stream acquisition and detection module in the present invention; Figure 3 This is a flowchart of obtaining the background interference analysis and filtering module in the present invention; Figure 4 This is a flow chart of obtaining the target motion trajectory calculation module in the present invention; Figure 5 This is a flowchart of obtaining the viewing angle conversion and trajectory compensation module in the present invention; Figure 6 The figure is a flowchart for obtaining the abnormal behavior analysis module in the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0017] See also Figure 1 , an AI-based video content analysis system includes: The video stream acquisition and detection module extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change rate in the time dimension, identifies the highlighted area as the foreground area, extracts the brightness change trend in the area, and obtains the video foreground detection result; The background interference analysis and filtering module analyzes the background pixel change rate based on the video foreground detection results, removes areas with similar change rates, and obtains the target area map after background filtering; The target motion trajectory calculation module extracts the target edge feature points based on the target area map after background filtering, records the position changes of the feature points between frames, analyzes the motion trajectory and smoothness, screens the trajectory that conforms to the global trend, and obtains the target motion trajectory data; The perspective conversion and trajectory compensation module records the changes in trajectory and speed according to the target motion trajectory data, identifies the moment of perspective switching, adjusts the trajectory prediction direction, and obtains the trajectory prediction result after perspective compensation; The abnormal behavior analysis module determines the trend change characteristics based on the trajectory prediction results after perspective compensation, classifies and analyzes the deviation types and abnormal change patterns of differentiated time points, and generates abnormal change analysis results of the video content.

[0018] The video foreground detection results include the highlight area, the brightness change trend of the foreground area, and the pixel features of the foreground area. The target area map after background filtering includes the filtered target area boundary, background interference features, and the remaining target area. The target motion trajectory data includes trajectory feature points, inter-frame position information, and motion smoothness. The trajectory prediction results after perspective compensation include the compensated trajectory path, trajectory speed information, and perspective switching time point. The analysis results of abnormal changes in video content include abnormal trajectory deviation types, abnormal changes in time points, and behavior change patterns.

[0019] See also Figure 2 ,The video stream acquisition and detection module includes a brightness value extraction submodule, a foreground area recognition submodule, and a foreground change trend screening submodule; The brightness value extraction submodule extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change of adjacent frames, records the brightness change rate, and generates a pixel brightness change rate data set; First, the video stream parsing tool is called to perform frame operations on the video stream. The frequency of obtaining video frames is set according to the monitoring requirements. In real-time monitoring of public places, the frame rate is 30 frames per second. Each frame after parsing is a two-dimensional matrix composed of pixels, where the horizontal axis of the matrix represents the horizontal position of the image, and the vertical axis represents the vertical position of the image. Each pixel includes the color values ​​of the three channels of red, green, and blue. Next, the RGB value of each pixel needs to be extracted, and the corresponding brightness value is calculated through the brightness conversion calculation formula. The extracted brightness value will be stored in the corresponding matrix position. After completing the extraction of all pixel brightness values ​​of a frame, the brightness matrix of the frame will be compared pixel by pixel with the brightness matrix of the previous frame to calculate the brightness change of each pixel, that is, the difference in brightness values ​​of the same pixel in adjacent frames. After that, the ratio of the brightness change of each pixel to the inter-frame time needs to be recorded to obtain the brightness rate data. The rate data will be continuously recorded and stored as a pixel brightness change rate data set.

[0020] The foreground area recognition submodule determines whether the brightness change exceeds the brightness fluctuation threshold based on the pixel brightness change rate data set, selects pixels with prominent change rates, and generates a set of pixels in the highlight area; First, the entire data set is traversed to calculate the mean brightness change rate of all pixels during the monitoring period. Then, the standard deviation of the brightness change rate needs to be calculated to provide a reference for subsequent screening. Next, an adjustment coefficient is set according to the monitoring environment. The size of the coefficient is set according to the changes in the ambient light. In outdoor environments, the light changes frequently, and the adjustment coefficient is set high, while in indoor environments, it is relatively low. Finally, the brightness fluctuation threshold is calculated using the mean, standard deviation and adjustment coefficient of the brightness change rate. Next, each pixel in the data set is traversed to compare its brightness change rate with the threshold. If the brightness change rate exceeds the threshold, the pixel is marked as a highlight area pixel, and finally a highlight area pixel set is formed.

[0021] The foreground change trend screening submodule calls the highlight area pixel set and analyzes the pixel brightness change trend in the area using the formula: ; Calculate the brightness change trend value of the highlight area, perform trend classification, and generate video foreground detection results; in, Represents the brightness change trend value of the highlight area. Representative Frame pixel brightness value, represents the frame interval time, represents the weight coefficient of the rate of change, Represents the total number of pixels in the area. Indicates the brightness value of the pixel in the next frame; First, a time window is set for analysis. The length of the time window is set according to the specific monitoring requirements. For example, in 24-hour monitoring, every 30 minutes can be set as a time window. Then, all frames in each time window are traversed, and the brightness change between adjacent frames is calculated for each pixel. The brightness change rate is calculated based on the time between frames. Then, a change rate weight coefficient needs to be introduced to assign different weights to different frames. The weight of the recent time frame is larger to reflect the recent brightness changes. Assume that a highlight area contains 5 pixels, and the brightness change values ​​in 4 consecutive frames are [20, 25, 30, 15] respectively, and the inter-frame time , the weight coefficient is set to [1.2, 1.1, 1.3, 1.0], then the brightness change trend value is calculated as follows: ; ; Calculated brightness change trend value It is 3165, which is compared with the set trend threshold. If the trend threshold is set to 3000, the area is determined to be a foreground target, and finally a video foreground detection result is generated.

[0022] See also Figure 3 ,The background interference analysis and filtering module includes the background pixel change submodule, the same target elimination submodule, and the target area construction submodule; The background pixel change submodule calculates the brightness change rate of background pixels in consecutive frames based on the video foreground detection results, and calculates the mean and change range to obtain the video background pixel change characteristics; First, extract the pixel set of the background area. The entire video frame needs to be divided into foreground and background parts. The pixels in the background area have relatively stable brightness change characteristics. Therefore, by counting multiple consecutive frames, the pixels with a lower change rate can be screened as background pixels. The brightness change rate of each pixel in the background area in consecutive frames needs to be calculated, and the brightness difference between adjacent frames of the same pixel is obtained. It is divided by the time interval between frames to obtain the pixel change rate. Then, all background pixels are traversed, the mean brightness change is counted, and the range of the change amplitude is calculated. The range can be set by adding or subtracting the standard deviation of the mean value to measure the normal range of background changes. In practical applications, such as airport security monitoring scenes, the background change amplitude is small, while the brightness change rate in the pedestrian activity area is relatively high. Through this analysis, the background and dynamic targets can be distinguished, and the background pixel change characteristics can be finally obtained.

[0023] The similar target elimination submodule analyzes the foreground pixel change rate based on the video background pixel change characteristics and uses the formula: ; Calculate the difference between the foreground and background change rates, filter and remove areas with similar change rates, and obtain the background removal target area; in, Represents the difference between the foreground and background change rates. For background The rate at which pixel brightness changes, For the future The rate at which pixel brightness changes, is the mean value of background brightness variation, is the total number of pixels, is the foreground pixel index; First, traverse all foreground pixels, obtain their brightness change rate in consecutive frames, and compare it with the change rate of background pixels to calculate the rate difference between foreground pixels and background pixels; Assume that the brightness change rate of the 5 pixels in the background area is for ; but , the rate of 5 pixels in the foreground area for ; According to the formula: After calculating item by item, the final , if the rejection threshold is 0.1, the area is judged as the background and is rejected, and the background rejection target area is obtained.

[0024] The target region construction submodule calls to remove the background target region, reconstruct the foreground pixels, mark the target region boundary, and generate the target region map after background filtering; First, the discarded background area needs to be removed from the original foreground area, and the connected area analysis is performed on the remaining foreground pixels to ensure that the target area is complete and traceable. Next, the boundary of the target area needs to be marked. The distribution of foreground pixels can be calculated to identify areas with higher pixel density, and the target area can be framed in the form of a minimum circumscribed rectangle. In practical applications, such as intelligent traffic monitoring, this step can be used to detect moving vehicles or pedestrians and remove environmental background interference, and finally obtain a target area map after background filtering.

[0025] See also Figure 4 ,The target motion trajectory calculation module includes a target area extraction submodule, a target edge extraction submodule, and a motion trajectory screening submodule; The target region extraction submodule extracts the dynamic change information in the target region based on the target region map after background filtering, identifies the position change between frames of the target region and performs screening to generate the coordinate value of the target region; First, the inter-frame position change information of the target area is obtained based on the target area map after background filtering. The dynamic change information in the target area is extracted. The target area map needs to be binarized to distinguish the background from the foreground, retain the data of the target area, and identify the change of the target by detecting the significant area in the image. For example, if the color or texture of the target area changes significantly in a frame image, this part of the area will be marked as the target area. By comparing with the target area position of the previous frame, the change of the target between the current frame and the previous frame is obtained. For different target areas, the data of each frame is further screened and confirmed to ensure that the extracted target area position has high accuracy and validity. For example, assuming that the initial position of a target area is (30, 40) in frame 1 and (35, 45) in frame 2, the inter-frame change of the target area is 5 pixels to the upper right. After processing multiple frames, the coordinate value of the target area is generated.

[0026] The target edge extraction submodule extracts edge feature points according to the target area coordinate values, captures the target edge position, records the position changes of the edge feature points, and obtains the target edge coordinate data; First, the edge feature points of the target area are extracted based on the coordinate value of the target area. It is necessary to find the edge of the target in the image according to the coordinate value of the target area image, and use edge detection methods such as Canny operator, Sobel operator, etc. to process the image and identify the contour edge of the target. The edge feature point refers to all qualified points on the edge of the target, which are extracted by the edge detection algorithm. For example, in a target area, if the contour of the target has a sharp angle or a bend, then the position is the edge feature point. The coordinates of each feature point are recorded, and the changes between each frame are calculated. For example, in one frame, the coordinates of a feature point are (20, 30), and the position of the point in the next frame is (25, 35). Therefore, by calculating the changes between the feature points, the dynamic changes of the target edge can be obtained, and the changed data forms the target edge coordinate data.

[0027] The motion trajectory screening submodule analyzes and calculates the smoothness value of the target motion trajectory according to the target edge coordinate data, and screens the trajectory that conforms to the global trend using the formula: ; Analyze and optimize the accuracy of the motion trajectory to obtain the target motion trajectory data; in, Represents the smoothness value of the target motion trajectory, and Representative The horizontal and vertical coordinates of the frame feature points, and Represents the horizontal and vertical coordinates of the previous frame. The frame sequence number representing the feature point, Represents the total number of feature points; First, the target's motion trajectory is analyzed based on the target edge coordinate data generated by the first two submodules. By comparing the edge coordinates of each frame, the target's motion path is obtained. For example, assuming that the target position in frame 1 is (10, 20), the target position in frame 2 is (15, 25), and the target position in frame 3 is (20, 30), the target's motion trajectory can be obtained by continuously calculating the change in coordinates, and the trajectory that conforms to the global trend can be screened by analyzing the smoothness of the motion trajectory; Assume that the target's motion trajectory contains 3 frames (i.e. ), the corresponding coordinates are: Frame 1: ; Frame 2: ; Frame 3: ; Now, substitute the values ​​into the formula for calculation. First, calculate the change of the horizontal coordinate between each frame: ; ; Then calculate the change in the ordinate between each frame: ; ; Therefore, the smoothness for: ; ; The smoothness of the resulting motion trajectory ,The smoothness of the target trajectory will be compared with the preset threshold ,if the threshold is set to 10, it can be judged that the trajectory is relatively smooth and ,conforms to the target’s motion trend. The target’s motion trajectory is ,screened to obtain the target motion trajectory data, which can be ,determined to be valid and conforms to the global trend.

[0028] See also Figure 5 ,The view conversion and trajectory compensation module includes a trajectory recording submodule, a view switching recognition submodule, and a trajectory compensation calculation submodule; The trajectory recording submodule records the target position change and speed change according to the target motion trajectory data, stores the trajectory data, and obtains the trajectory speed change record; The target's motion trajectory data is obtained through the surveillance video analysis system. This data includes the target's spatial position at continuous time points. In order to realize practical application, it is assumed that in a shopping mall's surveillance video, the continuous frame position of the person's movement is first detected through image processing technology, and then the position data is marked as a time series. This can achieve accurate recording of the target position at each time point. The system calculates the speed change of the target in a continuous time period, for example, how the speed of a customer moving from the north gate to the south gate changes. This involves real-time calculation of speed, that is, the spatial difference between the current position and the previous position divided by the time difference. The calculation process is implemented through physical motion formulas and does not involve any preset models or methods. It can generate trajectory speed change records, which can be used for subsequent behavior analysis or security monitoring, such as evaluating the change in pedestrian speed within a certain period of time, detecting abnormal movement behavior, etc.

[0029] The perspective switching recognition submodule calculates the target trajectory mutation point based on the trajectory speed change record, analyzes the trajectory direction adjustment trend, identifies the perspective switching moment, and obtains the perspective switching moment sequence; First, based on the trajectory speed change record of the previous module, the target trajectory is analyzed. Taking a park monitoring as an example, when monitoring a child moving quickly from the playground to the lake, the system determines whether a perspective switch has occurred by detecting the mutation point of the speed. This occurs when the monitoring angle or monitoring range changes. The system analyzes the mutation point of the target trajectory and determines the direction adjustment trend of the target movement through the time series change of the point. For example, if a person suddenly changes from going straight to turning, it means that the perspective needs to be adjusted to continue to effectively track the target. The specific moment of the perspective switch can be determined and a perspective switch moment sequence can be generated. This sequence is crucial for the video content analysis system, ensuring that the monitoring system can adapt to environmental changes and effectively capture the target dynamics.

[0030] The trajectory compensation calculation submodule calls the view switching time sequence, analyzes the trajectory error before and after the view conversion, and adjusts the trajectory prediction direction using the formula: ; Calculate the perspective compensation distance, correct the trajectory based on the target trajectory trend, and obtain the trajectory prediction result after perspective compensation; in, Represents the viewing angle compensation distance, Representative moments The spatial location of the target, represents the spatial position of the target at the previous moment, Representative moments The target's movement speed, Indicates the target's moving speed at the previous moment. represents the time series length of the trajectory record; First, the view switching time sequence is called to process the impact of view conversion on trajectory prediction. For example, in urban traffic monitoring, when a car leaves monitoring area A and enters monitoring area B, the trajectory prediction will deviate due to the change in camera angle. In order to compensate for this deviation, the view compensation distance is calculated. Assume that the trajectory data of a vehicle at different time points are as follows: , , , , , , , , the total length of the time series ; Substituting into the formula: ; ; ; ; Calculated , which means that due to the perspective conversion, the trajectory of the target between different monitoring areas is offset by about 11.11 meters. This compensation distance is used to adjust the trajectory prediction so that the monitoring system can track the target more accurately. For urban traffic monitoring, such a compensation mechanism can reduce the trajectory deviation caused by the switching of monitoring perspectives, thereby more accurately predicting the actual driving path of the vehicle.

[0031] See also Figure 6 ,The abnormal behavior analysis module includes a trend change calculation submodule, an abnormal deviation classification submodule, and an abnormal pattern recognition submodule; The trend change calculation submodule analyzes the target's motion trend change rate in the time series based on the trajectory prediction results after view compensation, determines the mutation of the motion mode, and obtains the trend change characteristics; First, the target's trajectory data is time-segmented. The trajectory data is divided into different intervals according to a fixed time window. For example, the position change is calculated every 0.5 seconds, the position information of the target at each time point is extracted, and the distance increment between adjacent time points is calculated. Assuming that a target is The position at the moment is (10, 20), The position at the moment is (12, 24), then its displacement in this time period can be calculated as ; Calculate the target's motion trend change rate at each time point, that is, deduce its acceleration through the velocity changes at adjacent time points. The speed is 2.5m / s. The speed at that moment is 3.0 m / s, so the acceleration can be calculated as ; Based on the change in acceleration, determine whether there is a sudden change in the target's movement pattern. For example, if the target's speed suddenly changes from 3m / s to 6m / s in a short period of time, and the speed change was small at the previous moment, it can be considered that the target's movement pattern has undergone a sudden change at this moment. By combining the trend changes at all time points, the target's trend change characteristics are established.

[0032] The abnormal deviation classification submodule calls the trend change characteristics to identify the deviation angle, position error and speed change of the target at different time points, using the formula: ; Calculate the abnormal deviation and obtain the deviation type at the differentiated time points; in, represents the degree of abnormal deviation, and Represents adjacent time points and The target location, and Represents adjacent time points and The target speed, and Represents adjacent time points and The target acceleration; First, obtain the position information of the target at adjacent time points to calculate the deviation angle. For example, at time and time The target positions at are (15, 30) and (18, 36) respectively, then the deviation angle can be calculated by the inverse tangent function , calculate the target position information error, that is, the deviation between the expected position of the target on the predicted trajectory and the actual position. For example, if the predicted position is (17, 35) and the actual position is (18, 36), the position error is calculated as ; Calculate the speed change of the target at adjacent time points using the formula: ; in , , , , , ,but: ; According to the calculated abnormal deviation, different deviation situations are classified. For example, if the threshold is set to 5, , it is judged as a normal deviation, otherwise it is judged as an abnormal deviation, and finally the deviation type of the differentiated time point is obtained.

[0033] The abnormal pattern recognition submodule analyzes the time distribution of the deviation based on the deviation type at the differentiated time point, calculates the time density of the abnormal change, screens the abnormal area, and obtains the analysis results of the abnormal change of the video content; First, count the abnormal deviation points of the target in the time series and extract the specific time period when the deviation points occur. For example, if there are 3 abnormal deviations within a 10-second time window, the abnormal occurrence frequency is calculated as , calculate the time density of anomalies, and determine the concentration of anomalies by counting the number of anomalies per unit time. For example, if anomalies occur 3 times in the first time window (0-10 seconds) and 6 times in the second time window (10-20 seconds), the anomaly densities are 0.3 and 0.6 respectively. Filter the concentrated areas of abnormal changes. Assuming that the anomaly density threshold is set to 0.5, the time period with an anomaly density higher than 0.5 is determined as an abnormal concentrated area. For example, in the time period (10-20 seconds), the anomaly density reaches 0.6, and the time period is marked as an abnormal area. Combined with all abnormal area data, the abnormal change analysis results of the video content are established.

[0034] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An AI-based video content analysis system, characterized in that: The system comprises: The video stream acquisition and detection module extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change rate in the time dimension, identifies the highlighted area as the foreground area, extracts the brightness change trend in the area, and obtains the video foreground detection result; The background interference analysis and filtering module analyzes the background pixel change rate based on the video foreground detection result, removes the areas with similar change rates, and obtains the target area map after background filtering; The target motion trajectory calculation module extracts target edge feature points based on the target area map after background filtering, records the inter-frame position changes of the feature points, analyzes the motion trajectory and smoothness, screens the trajectory that conforms to the global trend, and obtains the target motion trajectory data; The perspective conversion and trajectory compensation module records the changes in trajectory and speed according to the target motion trajectory data, identifies the moment of perspective switching, adjusts the trajectory prediction direction, and obtains the trajectory prediction result after perspective compensation; The abnormal behavior analysis module determines the trend change characteristics based on the trajectory prediction results after the perspective compensation, classifies and analyzes the deviation types and abnormal change patterns of the differentiated time points, and generates abnormal change analysis results of the video content.

2. The AI-based video content analysis system according to claim 1, characterized in that: The video foreground detection result includes the highlight area, the brightness change trend of the foreground area, and the pixel features of the foreground area. The target area map after background filtering includes the filtered target area boundary, background interference features, and the remaining target area. The target motion trajectory data includes trajectory feature points, inter-frame position information, and motion smoothness. The trajectory prediction result after perspective compensation includes the compensated trajectory path, trajectory speed information, and perspective switching time point. The video content abnormal change analysis result includes the abnormal trajectory deviation type, abnormal change at the time point, and behavior change pattern.

3. The AI-based video content analysis system according to claim 1, characterized in that: The video stream acquisition and detection module includes a brightness value extraction submodule, a foreground area recognition submodule, and a foreground change trend screening submodule; The brightness value extraction submodule extracts the brightness values ​​of pixels in consecutive frames based on the surveillance video data, calculates the brightness change of adjacent frames, records the brightness change rate, and generates a pixel brightness change rate data set; The foreground area recognition submodule determines whether the brightness change exceeds the brightness fluctuation threshold based on the pixel brightness change rate data set, selects pixels with prominent change rates, and generates a highlight area pixel set; The foreground change trend screening submodule calls the highlight area pixel set to analyze the pixel brightness change trend in the area using the formula: ； Calculate the brightness change trend value of the highlight area, perform trend classification, and generate video foreground detection results; in, Represents the brightness change trend value of the highlight area. Representative Frame pixel brightness value, represents the frame interval time, represents the weight coefficient of the rate of change, Represents the total number of pixels in the area. Indicates the pixel brightness value of the next frame.

4. The AI-based video content analysis system according to claim 2, characterized in that: The background interference analysis and filtering module includes a background pixel change submodule, a similar target elimination submodule, and a target area construction submodule; The background pixel change submodule calculates the brightness change rate of background pixels in consecutive frames based on the video foreground detection result, calculates the mean value and the change amplitude range, and obtains the video background pixel change characteristics; The similar target elimination submodule analyzes the foreground pixel change rate according to the video background pixel change characteristics, using the formula: ； Calculate the difference between the foreground and background change rates, filter and remove areas with similar change rates, and obtain the background removal target area; in, Represents the difference between the foreground and background change rates. For background The rate at which pixel brightness changes, For the future The rate at which pixel brightness changes, is the mean value of background brightness variation, is the total number of pixels, is the foreground pixel index; The target region construction submodule calls the background-removed target region, reconstructs foreground pixels, marks the target region boundary, and generates a target region map after background filtering.

5. The AI-based video content analysis system according to claim 3, characterized in that: The target motion trajectory calculation module includes a target area extraction submodule, a target edge extraction submodule, and a motion trajectory screening submodule; The target region extraction submodule extracts dynamic change information in the target region based on the target region map after background filtering, identifies and filters the position change between frames of the target region, and generates the coordinate value of the target region; The target edge extraction submodule extracts edge feature points according to the target area coordinate values, captures the target edge position, records the position change of the edge feature points, and obtains the target edge coordinate data; The motion trajectory screening submodule analyzes and calculates the smoothness value of the target motion trajectory according to the target edge coordinate data, and screens the trajectory that conforms to the global trend using the formula: ； Analyze and optimize the accuracy of the motion trajectory to obtain the target motion trajectory data; in, Represents the smoothness value of the target motion trajectory, and Representative The horizontal and vertical coordinates of the frame feature points, and Represents the horizontal and vertical coordinates of the previous frame. The frame sequence number representing the feature point, Represents the total number of feature points.

6. The AI-based video content analysis system according to claim 4, characterized in that: The perspective conversion and trajectory compensation module includes a trajectory recording submodule, a perspective switching identification submodule, and a trajectory compensation calculation submodule; The trajectory recording submodule records the change in the target position and the change in speed according to the target motion trajectory data, stores the trajectory data, and obtains the trajectory speed change record; The perspective switching identification submodule calculates the target trajectory mutation point based on the trajectory speed change record, analyzes the trajectory direction adjustment trend, identifies the perspective switching moment, and obtains the perspective switching moment sequence; The trajectory compensation calculation submodule calls the view switching time sequence, analyzes the trajectory error before and after the view conversion, and adjusts the trajectory prediction direction using the formula: ； Calculate the perspective compensation distance, correct the trajectory based on the target trajectory trend, and obtain the trajectory prediction result after perspective compensation; in, Represents the viewing angle compensation distance, Representative moments The spatial location of the target, represents the spatial position of the target at the previous moment, Representative moments The target's movement speed, Indicates the target's moving speed at the previous moment. Represents the time series length of the trajectory record.

7. The AI-based video content analysis system according to claim 5, characterized in that: The abnormal behavior analysis module includes a trend change calculation submodule, an abnormal deviation classification submodule, and an abnormal pattern recognition submodule; The trend change calculation submodule analyzes the target's motion trend change rate in the time series based on the trajectory prediction result after the perspective compensation, determines the mutation of the motion mode, and obtains the trend change characteristics; The abnormal deviation classification submodule calls the trend change characteristics to identify the deviation angle, position error and speed change of the target at the differentiated time points, using the formula: ； Calculate the abnormal deviation and obtain the deviation type at the differentiated time points; in, represents the degree of abnormal deviation, and Represents adjacent time points and The target location, and Represents adjacent time points and The target speed, and Represents adjacent time points and The target acceleration; The abnormal pattern recognition submodule analyzes the time distribution of the deviation based on the deviation type of the differentiated time point, calculates the time density of the abnormal change, screens the abnormal area, and obtains the analysis result of the abnormal change of the video content.

Images